Json.NET 11.0 Release 1 - .NET Standard 2.0, JsonConverters, JSON Path and more

.NET Standard 2.0

The big new feature in Json.NET 11 Release 1 is targeted support for .NET Standard 2.0.

There are two main benefits of a library like Json.NET targeting .NET Standard 2.0. The first is more APIs: Json.NET with .NET Standard 2.0 almost matches Json.NET on the traditional Windows .NET Framework in features.

For example, fans of serializing DataSets to and from JSON rejoice, .NET Core now supports your pro-DataSet lifestyle:

DataTable dt = new DataTable();
dt.Columns.Add("PackageId", typeof(string));
dt.Columns.Add("Version", typeof(string));
dt.Columns.Add("ReleaseDate", typeof(DateTime));

dt.Rows.Add("Newtonsoft.Json", "11.0.1", new DateTime(2018, 2, 17));
dt.Rows.Add("NUnit", "3.9.0", new DateTime(2017, 11, 10));

string json = JsonConvert.SerializeObject(dt, Formatting.Indented);

Console.WriteLine(json);
// [
//   {
//     "PackageId": "Newtonsoft.Json",
//     "Version": "11.0.1",
//     "ReleaseDate": "2018-02-17T00:00:00"
//   },
//   {
//     "PackageId": "Newtonsoft.Json",
//     "Version": "10.0.3",
//     "ReleaseDate": "2017-06-18T00:00:00"
//   }
// ]

The other benefit of .NET Standard 2.0 is developers are no longer spammed with NuGet dependencies. UWP app authors for example saw NuGet pull in over 100 packages when referencing Json.NET. UWP supporting .NET Standard 2.0 and consuming a netstandard2.0 package eliminates that problem.

JsonConverter Stuff

Json.NET 11 adds a generic JsonConverter<T>. If you only care about serializing one specific type then JsonConverter<T> eliminates boiler plate code from your converter and keeps everything strongly typed.

Also new is a UnixDateTimeConverter. There is no standard for serializing dates in JSON and so UnixDateTimeConverter is a useful converter for anyone who want to store time as an integer Unix epoch in their JSON.

JSON Path Stuff

Json.NET 11 adds support for the regular expression operator in JSON Path queries. JSON Path has no formal specification (other than a blog post) but one common addition to it is support for querying with regular expressions.

JArray packages = JArray.Parse(@"[
  {
    ""PackageId"": ""Newtonsoft.Json"",
    ""Version"": ""11.0.1"",
    ""ReleaseDate"": ""2018-02-17T00:00:00""
  },
  {
    ""PackageId"": ""NUnit"",
    ""Version"": ""3.9.0"",
    ""ReleaseDate"": ""2017-11-10T00:00:00""
  }
]");

List<JToken> newtonsoftPackages = packages.SelectTokens(@"$.[?(@.PackageId =~ /^Newtonsoft\.(.*)$/)]").ToList();

Console.WriteLine(newtonsoftPackages.Count);
// 1

JSON Path has also seen many smaller bug fixes this release to improve its performance and support of escaped characters.

And lots more

Serializing and deserializing enumerations by name is faster and more accurate, JsonReader.SupportMultipleContent supports reading multiple comma delimited fragments of JSON together, JObject exposes ContainsKey, error messages have improved in many exceptional situations, and dozens of other new features and bug fixes.

Changes

Here is a complete list of what has changed since Json.NET 10.0 Release 3.

  • New feature - Added netstandard2.0 build
  • New feature - Added generic JsonConverter<T>
  • New feature - Added UnixDateConverter for converting Unix timestaps
  • New feature - Added support for regex operator in JSON Paths
  • New feature - Added JsonObjectAttribute.ItemNullValueHandling
  • New feature - Added JsonObjectContract.ItemNullValueHandling
  • New feature - Improved performance when resolving serialization contracts by using ConcurrentDictionary
  • New feature - Improved performance of JToken.Path with a faster reverse
  • New feature - Improved performance of parsing Int32 JSON integer values
  • New feature - Improved performance of parsing and writing enum names
  • New feature - Added IgnoreIsSpecifiedMembers to DefaultContractResolver
  • New feature - Added IgnoreShouldSerializeMembers to DefaultContractResolver
  • New feature - Added support for reading multiple comma delimited values with JsonReader.SupportMultipleContent
  • New feature - Improved error message when an object is reused with PreserveReferencesHandling
  • New feature - Added IConvertible support to netstandard1.3
  • New feature - Added INotifyPropertyChanging support to netstandard1.3
  • New feature - Optimized internal buffering when writing very large strings
  • New feature - JObject.ContainsKey is now public
  • New feature - Improved the error message when serialized ByRef properties
  • New feature - Improved the error message when serializing FileInfo/DictionaryInfo without ISerializable
  • New feature - Improved the error message when failing conversion in JsonReader.ReadAsInt32 and JsonReader.ReadAsDecimal
  • New feature - Improved the error message when deserializing badly formatted regex strings
  • Change - Types that implement ISerializable but don't have [SerializableAttribute] are not serialized using ISerializable
  • Change - Changed JsonProperty.MemberConverter to obsolete
  • Change - Changed camel casing logic for all caps words to not leave last character capitalized
  • Change - Changed enum serialization in dictionary keys to use EnumMemberAttribute
  • Fix - Fixed converting default datetime in JSON to XML on netstandard1.3
  • Fix - Fixed deserializing ObservableCollection in .NET Core 2.0
  • Fix - Fixed incorrectly allowing static IsSpecified properties
  • Fix - Fixed not preserving trailing zeros when deserializing decimals
  • Fix - Fixed JValue.CompareTo with number and string values
  • Fix - Fixed not erroring when reading undefined for nullable long
  • Fix - Fixed serializing immutable collections when .NET 4.0 or previous Newtonsoft.Json assembly is GACed
  • Fix - Fixed writing null values by XmlNodeConverter and RegexConverter
  • Fix - Fixed deserializing with a TraceWriter when the reader is not at the start
  • Fix - Fixed TraceJsonWriter.WriteValue(object) writing value twice
  • Fix - Fixed deserializing with constructor and readonly collection property
  • Fix - Fixed error when writing unknown null value as JSON
  • Fix - Fixed merging null string values
  • Fix - Fixed missing CancellationToken usages in JsonTextWriter.WriteAsync
  • Fix - Fixed error with JsonSerializer.Populate and comments in JSON array
  • Fix - Fixed error handling when deserializing certain dictionary and lists
  • Fix - Fixed serializing collection that is nullable
  • Fix - Fixed JsonTextReader sync read in async method
  • Fix - Fixed JsonConverter not called when target type is list and token is a string
  • Fix - Fixed serializing a property that is defined in multiple interfaces
  • Fix - Fixed error when deserializing null value into ConcurrentDictionary
  • Fix - Fixed escaping special characters in generated JSON Paths
  • Fix - Fixed reading escaped special characters in JSON Paths
  • Fix - Fixed using extension data with dictionary that explicitly implements Add method
  • Fix - Fixed parsing enum name to exact casing value before falling back to case-insensitive value
  • Fix - Fixed DataContractAttribute not forcing object serialization of classes
  • Fix - Fixed bug when deserializing into existing non-IList collection
  • Fix - Fixed bug when deserializing into existing non-IDictionary collection
  • Fix - Fixed JsonReader.ReadAsInt32 with BigInteger values

Links

Json.NET GitHub Project

Json.NET 11.0 Release 1 Download - Json.NET source code and assemblies

Json.NET 10.0 Release 1 - Async, performance, documentation and more

Async support

The headline feature in Json.NET 10 is enabling asyncronously reading and writing JSON with JsonReader and JsonWriter, and asyncronously loading JObject, JArray and friends.

Async support means that reading or writing JSON to the file system or network will never block while waiting on IO. Client applications will be more responsive and web applications more scalable.

JArray largeJson;

// read asynchronously from a file
using (FileStream asyncFileStream = new FileStream(@"large.json", FileMode.Open, FileAccess.Read, FileShare.Read, 4096, true))
{
    largeJson = await JArray.LoadAsync(new JsonTextReader(new StreamReader(asyncFileStream)));
}

JToken user = largeJson.SelectToken("$[?(@.name == 'Woodard Caldwell')]");
user["isActive"] = false;

// write asynchronously to a file
using (FileStream asyncFileStream = new FileStream(@"large.json", FileMode.Open, FileAccess.Write, FileShare.Write, 4096, true))
{
    await largeJson.WriteToAsync(new JsonTextWriter(new StreamWriter(asyncFileStream)));
}

Although it looks simple on the surface, adding async support to Json.NET was a huge task. Special thanks to Jon Hanna for contributing most of this work.

Performance

Json.NET now supports reading double and decimal values without internally converting the value to a string first. In addition to a small performance boost when deserializing double and decimal values, not creating a string for each double and decimal means less garbage for the garbage collector to clean up.

Obsoleting and removing

In an effort to slim down Json.NET the BSON functionality has been marked as obsolete. A Newtonsoft.Json.Bson package is on NuGet and contains a copy of BsonReader and BsonWriter. Obsolete classes in Newtonsoft.Json will eventually be removed after a few major versions.

Some previously obsoleted types and methods have been removed in Json.NET 10. These APIs aren't commonly used by end users, and all have been obsolete for multiple major versions. Check the detailed release notes below for more information.

Finally the Portable Class Library assembly that targets .NET 4 has been removed from the NuGet package.

And lots more

TypeConverters, BigInteger, ISerializable and XmlDocument are now supported in .NET Core applications, there have been multiple improvements to JSONPath querying of JObject/JArray, dozens of typos have been fixed in Json.NET's documentation, and many, many bugs have been reported and fixed across every part of Json.NET.

Changes

Here is a complete list of what has changed since Json.NET 9.0 Release 1.

  • New feature - Added async read support to JsonReader
  • New feature - Added async write support to JsonWriter
  • New feature - Added async support for loading JObject and JArray
  • New feature - Added non-allocating parsing of double and decimal values
  • New feature - Added support for TypeConverters to netstandard1.0+
  • New feature - Added support for BigInteger to netstandard1.3+
  • New feature - Added support for ISerializable to netstandard1.3+
  • New feature - Added support for XmlDocument to netstandard1.3+
  • New feature - Added support for SerializableAttribute and NonSerializedAttribute to netstandard1.3+
  • New feature - Added ISerializationBinder
  • New feature - Added SerializationBinder properties to JsonSerializer and JsonSerializerSettings
  • New feature - Added TypeNameAssemblyFormatHandling
  • New feature - Added TypeNameAssemblyFormatHandling properties to JsonSerializer and JsonSerializerSettings
  • New feature - Added support for root object selector in JSONPath filters
  • New feature - Added support for multiple names in JSONPath scan filter
  • New feature - Added support for any combination of paths and values in JSONPath filter expressions
  • New feature - Added support for extension data names to NamingStrategy
  • New feature - Added ProcessExtensionDataNames flag to NamingStrategy
  • New feature - Added JsonWriter.AutoCompleteOnClose to control whether JSON is auto-completed on JsonWriter.Close
  • New feature - Added JsonReaderException constructor with path, line number, line position
  • New feature - Added JsonWriterException constructor with path
  • New feature - Added JsonReader.SetToken method overload with option not to update position array index
  • New feature - Added support for automatically calling single constructor on immutable structs
  • Change - Removed .NET 4 portable class library target from NuGet package
  • Change - Removed obsolete JsonConverter.GetSchema method
  • Change - Removed obsolete constructor from DefaultContractResolver
  • Change - Removed obsolete async methods from JsonConvert
  • Change - Removed obsolete OnDeserialized, OnDeserializing, OnSerialized, OnSerializing, OnError from JsonContract
  • Change - Removed obsolete JsonDictionaryContract.PropertyNameResolver
  • Change - Removed obsolete ConstructorParameters, OverrideConstructor, ParametrizedConstructor from JsonObjectContract
  • Change - Obsoleted TypeNameAssemblyFormat properties on JsonSerializer and JsonSerializerSettings
  • Change - Obsoleted SerializationBinder
  • Change - Obsoleted Binder properties on JsonSerializer and JsonSerializerSettings
  • Change - Obsoleted FormatterAssemblyStyle in non-full .NET targets
  • Change - Obsoleted Newtonsoft.Json.Bson (moved to new NuGet package)
  • Change - Improved constructor parameter binding on .NET Core and portable builds
  • Change - Improved error when attempting to convert root JSON object property to an XML attribute
  • Change - Changed exception thrown when parsing invalid Unicode escape sequence to JsonReaderException
  • Change - Changed StringEnumConverter.AllowIntegerValues to also reject reading integer strings
  • Change - Changed StringEnumConverter.AllowIntegerValues to also reject writing enum values with no name
  • Fix - Fixed JsonTextReader.ReadAsXXX methods not erroring on missing commas between values
  • Fix - Fixed error when serializing F# lists with F# 4.1+
  • Fix - Fixed StringEnumConverter reading specified enum names that contain a comma
  • Fix - Fixed using a TraceWriter with nullable bytes
  • Fix - Fixed deserializing null Regex values
  • Fix - Fixed deserializing DataTable with error handling
  • Fix - Fixed DynamicReflectionDelegateFactory creating typed value type constructors
  • Fix - Fixed deserializing with type naming and complex nested generic types
  • Fix - Fixed error when converting certain JSON to XML on .NET Core
  • Fix - Fixed error handling skipping reading the next object value
  • Fix - Fixed date XML precision when converting certain JSON to XML on portable builds
  • Fix - Fixed not serializing readonly properties as references if they have corresponding creator parameters
  • Fix - Fixed type name being written with nullable structs and TypeNameHandling.Auto
  • Fix - Fixed deserializing a duplicate dictionary key from a parameterized constructor
  • Fix - Fixed not erroring when deserializing incomplete JSON with a parameterized constructor
  • Fix - Fixed incorrectly parsing scan filter when name is quoted
  • Fix - Fixed return type for SByte and SByteNullable in JToken.ToObject
  • Fix - Fixed deserializing generic only IList types with a constructor override
  • Fix - Fixed null reference error with ReadAsBytes and an empty JSON object
  • Fix - Fixed StringEnumConverter when EnumMember defines members that differ only by case
  • Fix - Fixed serializing protected override properties
  • Fix - Fixed merging null with complex type and MergeNullValueHandling.Ignore
  • Fix - Fixed roundtripping double.MaxValue and float.MaxValue as dictionary keys
  • Fix - Fixed JsonValidatingReader not closing underlying reader
  • Fix - Fixed writing duplicate nulls to trace with TraceJsonWriter.WriteValue for object, Uri, byte[]
  • Fix - Fixed BsonWriter.WriteValue erroring for null Uri or byte[]
  • Fix - Fixed not calling nullable WriteValue overloads in TraceJsonWriter
  • Fix - Fixed serializer not throwing an error when there is a comment followed by additional content with CheckAdditionalContent
  • Fix - Fixed JObject/JArray Parse not throwing an error when there is a comment followed by additional content
  • Fix - Fixed deserializing non-string values in some XML nodes
  • Fix - Fixed converting XML to JSON when json:Array and xmlns:json attributes are used directly on the array's element
  • Fix - Fixed error when attempting to populate values into read-only collection after creating object from non-default constructor

Links

Json.NET GitHub Project

Json.NET 10.0 Release 1 Download - Json.NET source code and assemblies

Json.NET 8.0 Release 1 - Allocations and bug fixes

Memory, Allocations and Performance

There is a big push across the .NET eco-system on performance. In modern .NET apps one of the biggest culprits of poor performance is allocating too many objects. The more objects and memory you allocate, the more the garbage collector needs to clean up. Best case garbage collection will slow your application; worst cause the app will halt execution until GC is finished. GC is a great way to kill pages per second on the server and frames per second on the client.

To reduce allocations and memory usage when serializing Json.NET 8.0 adds a new IArrayPool interface. Json.NET is already very lean when it comes to allocations, working with raw characters on array buffers instead of allocated strings, but those buffers can easily grow large, and a new buffer is created each time JSON is read or written. IArrayPool allows array buffers to be reused, similar to connection pooling with a database, or thread pooling in .NET.

public class JsonArrayPool : IArrayPool<char>
{
    public static readonly JsonArrayPool Instance = new JsonArrayPool();
 
    public char[] Rent(int minimumLength)
    {
        // get char array from System.Buffers shared pool
        return ArrayPool<char>.Shared.Rent(minimumLength);
    }
 
    public void Return(char[] array)
    {
        // return char array to System.Buffers shared pool
        ArrayPool<char>.Shared.Return(array);
    }
}

The example implemention above uses the upcoming System.Buffers package to manage pooling. Using the array pool just involves setting a property on JsonTextReader/JsonTextWriter.

IList<int> value;
 
JsonSerializer serializer = new JsonSerializer();
using (JsonTextReader reader = new JsonTextReader(new StringReader(@"[1,2,3,4]")))
{
    // reader will get buffer from array pool
    reader.ArrayPool = JsonArrayPool.Instance;
 
    value = serializer.Deserialize<IList<int>>(reader);
}

IArrayPool is a somewhat experimental feature. Unless you have extreme performance requirements or your own object pooling system already in-place you can wait until a default pool is automatically included with Json.NET.

JArray and Comments

Previously when loading JSON that contained comments into JArrays the comment would be added as an item in the JArray. This would be a common cause of errors: most developers would expect a commented out value in a JArray to disappear, instead it becomes a comment token, and an error would be thrown if the app looped over the array, casting its values.

In Json.NET 8.0 comments are ignored by default. If you want the old behaviour then set CommentHandling.Load on JsonLoadSettings.

Bug Fixes. Bug Fixes Everywhere.

There are two dozen minor bug fixes in Json.NET 8.0, ranging from fixes serializing DataSets and XML (brave souls still use them), to serializing F# discriminated unions in UWP applications.

Changes

Here is a complete list of what has changed since Json.NET 7.0 Release 1.

  • New feature - Added IArrayPool and a setting on JsonTextReader and JsonTextWriter
  • New feature - Added JsonReader.ReadAsDouble
  • New feature - Added JsonLoadSettings and CommentHandling to control loading comments in LINQ to JSON
  • New feature - Added JsonLoadSettings.LineInfoHandling
  • New feature - Added support for JsonConstructorAttribute on list and dictionary collections
  • New feature - Added support for deserializing string to Version
  • New feature - Added ShouldDeserialize to JsonProperty
  • New feature - Added Required.DisallowNull
  • New feature - Added support for converting JSON to XML with invalid XML name chracters
  • New feature - Improved case-insensitive deserialization performance
  • New feature - Improved date parsing performance
  • Change - Changed ReadAsDateTime, ReadAsDateTimeOffset, ReadAsBytes, ReadAsString, ReadAsInt32 on JsonReader from abstract to virtual
  • Change - Changed parsing JArrays to not include comments by default
  • Change - Changed JTokenWriter to use the last property instead of erroring when there are duplicate property names
  • Change - Changed Uri JValues to use OriginalString when written to JSON
  • Change - Changed DateTimeOffset JValues to return TypeCode.Object from IConvertible.GetTypeCode()
  • Fix - Fixed converting JSON metadata array value to XML
  • Fix - Fixed not including line information with some XML conversion errors
  • Fix - Fixed error when writing certain JSON with escaped characters
  • Fix - Fixed PopulateObject error when JSON starts with a comment
  • Fix - Fixed incorrect IJsonLineInfo line position after the first line
  • Fix - Fixed JSONPath when querying against Uri, Guid and Date values
  • Fix - Fixed JsonReader.Path when the path is escaped
  • Fix - Fixed writing JRaw twice when a TraceWriter is set
  • Fix - Fixed getting the wrong value when reading certain large integers
  • Fix - Fixed deserializing DataSet with a null DataTable
  • Fix - Fixed deserializing a null DataSet
  • Fix - Fixed error when serializing F# discriminated unions in Windows Store apps
  • Fix - Fixed error serializing some types when there are conflicting interface properties
  • Fix - Fixed error when settings ReferenceResolver is set to null
  • Fix - Fixed DateTimeZoneHandling not being used when writing DateTime dictionary keys
  • Fix - Fixed bug when converting an integer JValue to a nullable enum
  • Fix - Fixed converting null string JValue to XML
  • Fix - Fixed Mono generic class private field serialization bug
  • Fix - Fixed error when deserializing ignored property with mismatched type
  • Fix - Fixed deserializing to a type when setting JToken extension data
  • Fix - Fixed setting default values onto properties already set in constructor
  • Fix - Fixed casting dynamic JValue to JToken
  • Fix - Fixed serializing non-zero based arrays
  • Fix - Fixed reading 24 hour midnight ISO dates

Links

Json.NET GitHub Project

Json.NET 8.0 Release 1 Download - Json.NET source code and assemblies

Json.NET 7.0 Release 1 - Documentation, bug fixes, performance

Documentation

The biggest improvements in Json.NET 7.0 have been to user documentation. The old documentation design with its 2003 era HTML (iframes + inline JavaScript) has been replaced with a lightweight, fast to load design.

Old and Busted (left) vs New Hotness (right):

It takes two to lie: one to lie and one to listen   It's Patty who chose a life of celibacy. Selma had celibacy thrust upon her

It has been a couple of years since the docs were properly updated. New features added since then like extension data, annotations and JSONPath now have documentation and code samples. The new code samples brings the total up to 116!

Finally the documentation has been professionally proofread. My most embarrassing grammatical errors have been fixed.

NuGet Logo

Json.NET has a NuGet logo. Check it:

Kill my boss? Do I dare live out the American dream?

DiscriminatedUnionConverter performance improvements

Json.NET’s F# discriminated union support has been rewritten. Serializing very large collections of large discriminated unions was noticeably slow. The new implementation caches reflection and type data, and significantly improves performance.

What’s the point of going out? We’re just gonna wind up back home anyway.

That’s a 3200% improvement. If you’re using F# then you don’t need to do anything other than update Json.NET.

And everything else

Json.NET 7.0 includes 30 changes from 6 months of user feature requests and bug reports.

Changes

Here is a complete list of what has changed since Json.NET 6.0 Release 8.

  • New feature - DiscriminatedUnionConverter performance improvements
  • New feature - Added JsonRequiredAttribute
  • New feature - Added JsonSerializerSettings.ReferenceResolverProvider property
  • New feature - Added DefaultContractResolver.ResolveDictionaryKey
  • New feature - Added JsonDictionaryContract.DictionaryKeyResolver
  • New feature - Added support for reading GUID strings as bytes in JsonTextReader
  • New feature - Added EqualityComparer to JsonSerializer
  • Change - Changed reading GUIDs as bytes to only support 00000000-0000-0000-0000-000000000000 format
  • Change - Renamed aspnetcore50 target to dnxcore50
  • Change - Marked JsonSchema as obsolete
  • Change - Marked DefaultContractResolver(bool) as obsolete
  • Change - Marked JsonSerializerSettings.ReferenceResolver as obsolete
  • Change - Marked JsonDictionaryContract.PropertyNameResolver as obsolete
  • Fix - Fixed deserializing empty strings in Hashtables
  • Fix - Fixed incorrect JTokenReader.Path in certain situations
  • Fix - Fixed error when serializing certain objects in medium trust
  • Fix - Fixed deserializing large nullable UInt64 values
  • Fix - Fixed writing large UInt64 JValues
  • Fix - Fixed converting unmatched namespace prefixes in JSON to XML
  • Fix - Fixed IsoDateTimeConverter on DateTime properties with DateTimeFormatHandling.DateTimeOffset
  • Fix - Fixed preserving object references with read only properties
  • Fix - Fixed error when deserializing large JSON integers to XML
  • Fix - Fixed serializing extension data properties with no setter
  • Fix - Fixed serializing discriminated unions with type name or reference tracking enabled
  • Fix - Fixed DataTableConverter not using JsonSerializer settings
  • Fix - Fixed resolving properties from nested interfaces
  • Fix - Fixed deserializing classes derived from ConcurrentDictionary
  • Fix - Fixed passing default values to constructors
  • Fix - Fixed serializing root references from JsonConverters
  • Fix - Fixed empty strings coerced to null not erroring with Required.Always
  • Fix - Fixed invalid Required.Always error with constructor property name casing
  • Fix - Fixed empty string coerce check with Required.Always and constructor

Links

Json.NET GitHub Project

Json.NET 7.0 Release 1 Download - Json.NET source code and assemblies