Efficient JSON with Json.NET – Reducing Serialized JSON Size

Latest version of this guide: Reducing Serialized JSON Size

---

One of the common problems encountered when serializing .NET objects to JSON is that the JSON ends up containing a lot of unwanted properties and values. This can be especially important when returning JSON to the client. More JSON means more bandwidth and a slower website.

To solve the issue of unwanted JSON Json.NET has a range of built in options to fine tune what gets written from a serialized object.

JsonIgnoreAttribute and DataMemberAttribute

By default Json.NET will include all of a classes public properties and fields in the JSON it creates. Adding the JsonIgnoreAttribute to a property tells the serializer to always skip writing it to the JSON result.

public class Car

  // included in JSON

  public string Model { get; set; }

  public DateTime Year { get; set; }

  public List<string> Features { get; set; }

  // ignored

  [JsonIgnore]

  public DateTime LastModified { get; set; }

If a class has many properties and you only want to serialize a small subset of them then adding JsonIgnore to all the others will be tedious and error prone. The way to tackle this scenario is to add the DataContractAttribute to the class and DataMemberAttributes to the properties to serialize. This is opt-in serialization, only the properties you mark up with be serialized, compared to opt-out serialization using JsonIgnoreAttribute.

[DataContract]

public class Computer

  // included in JSON

  [DataMember]

  public string Name { get; set; }

  [DataMember]

  public decimal SalePrice { get; set; }

  // ignored

  public string Manufacture { get; set; }

  public int StockCount { get; set; }

  public decimal WholeSalePrice { get; set; }

  public DateTime NextShipmentDate { get; set; }

Formatting

JSON written by the serializer with an option of Formatting.Indented produces nicely formatted, easy to read JSON – great when you are developing. Formatting.None on the other hand keeps the JSON result small, skipping all unnecessary spaces and line breaks to produce the most compact and efficient JSON possible.

NullValueHandling

NullValueHandling is an option on the JsonSerializer and controls how the serializer handles properties with a null value. By setting a value of NullValueHandling.Ignore the JsonSerializer skips writing any properties that have a value of null.

public class Movie

  public string Name { get; set; }

  public string Description { get; set; }

  public string Classification { get; set; }

  public string Studio { get; set; }

  public DateTime? ReleaseDate { get; set; }

  public List<string> ReleaseCountries { get; set; }

Movie movie = new Movie();

movie.Name = "Bad Boys III";

movie.Description = "It's no Bad Boys";

string included = JsonConvert.SerializeObject(movie,

  Formatting.Indented,

  new JsonSerializerSettings { });

// {

//   "Name": "Bad Boys III",

//   "Description": "It's no Bad Boys",

//   "Classification": null,

//   "Studio": null,

//   "ReleaseDate": null,

//   "ReleaseCountries": null

// }

string ignored = JsonConvert.SerializeObject(movie,

  Formatting.Indented,

  new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });

// {

//   "Name": "Bad Boys III",

//   "Description": "It's no Bad Boys"

// }

NullValueHandling can also be customized on individual properties using the a JsonPropertyAttribute. The JsonPropertyAttribute value of NullValueHandling will override the setting on the JsonSerializer for that property.

DefaultValueHandling

DefaultValueHandling is an option on the JsonSerializer and controls how the serializer handles properties with a default value. Setting a value of DefaultValueHandling.Ignore will make the JsonSerializer skip writing any properties that have a default value to the JSON result. For object references this will be null. For value types like int and DateTime the serializer will skip the default unitialized value for that value type.

Json.NET also allows you to customize what the default value of an individual property is using the DefaultValueAttribute. For example if a string property called Department always returns an empty string in its default state and you didn't want that empty string in your JSON then placing the DefaultValueAttribute on Department with that value will mean Department is no longer written to JSON unless it has a value.

public class Invoice

  public string Company { get; set; }

  public decimal Amount { get; set; }

  // false is default value of bool

  public bool Paid { get; set; }

  // null is default value of nullable

  public DateTime? PaidDate { get; set; }

  // customize default values

  [DefaultValue(30)]

  public int FollowUpDays { get; set; }

  [DefaultValue("")]

  public string FollowUpEmailAddress { get; set; }

Invoice invoice = new Invoice

  Company = "Acme Ltd.",

  Amount = 50.0m,

  Paid = false,

  FollowUpDays = 30,

  FollowUpEmailAddress = string.Empty,

  PaidDate = null

};

string included = JsonConvert.SerializeObject(invoice,

  Formatting.Indented,

  new JsonSerializerSettings { });

// {

//   "Company": "Acme Ltd.",

//   "Amount": 50.0,

//   "Paid": false,

//   "PaidDate": null,

//   "FollowUpDays": 30,

//   "FollowUpEmailAddress": ""

// }

string ignored = JsonConvert.SerializeObject(invoice,

  Formatting.Indented,

  new JsonSerializerSettings { DefaultValueHandling = DefaultValueHandling.Ignore });

// {

//   "Company": "Acme Ltd.",

//   "Amount": 50.0

// }

DefaultValueHandling can also be customized on individual properties using the a JsonPropertyAttribute. The JsonPropertyAttribute value of DefaultValueHandling will override the setting on the JsonSerializer for that property.

IContractResolver

For more flexibility the IContractResolver provides an interface to customize almost every aspect of how a .NET object gets serialized to JSON, including changing serialization behavior at runtime.

public class DynamicContractResolver : DefaultContractResolver

  private readonly char _startingWithChar;

  public DynamicContractResolver(char startingWithChar)

    _startingWithChar = startingWithChar;

  protected override IList<JsonProperty> CreateProperties(JsonObjectContract contract)

    IList<JsonProperty> properties = base.CreateProperties(contract);

    // only serializer properties that start with the specified character

    properties =

      properties.Where(p => p.PropertyName.StartsWith(_startingWithChar.ToString())).ToList();

    return properties;

public class Book

  public string BookName { get; set; }

  public decimal BookPrice { get; set; }

  public string AuthorName { get; set; }

  public int AuthorAge { get; set; }

  public string AuthorCountry { get; set; }

Book book = new Book

                BookName = "The Gathering Storm",

                BookPrice = 16.19m,

                AuthorName = "Brandon Sanderson",

                AuthorAge = 34,

                AuthorCountry = "United States of America"

};

string startingWithA = JsonConvert.SerializeObject(book, Formatting.Indented,

  new JsonSerializerSettings { ContractResolver = new DynamicContractResolver('A') });

// {

//   "AuthorName": "Brandon Sanderson",

//   "AuthorAge": 34,

//   "AuthorCountry": "United States of America"

// }

string startingWithB = JsonConvert.SerializeObject(book, Formatting.Indented,

  new JsonSerializerSettings { ContractResolver = new DynamicContractResolver('B') });

// {

//   "BookName": "The Gathering Storm",

//   "BookPrice": 16.19

// }

Oct 11 2009

Json.NET 3.5 Release 5 – .NET 2.0 Support, Error Handling & Databinding
.NET 2.0 Support

Json.NET now has .NET 2.0 support! The great thing about 2.0 support is Json.NET 1.3.1 can finally be retired. Going on 2 years old now, 1.3.1 is chock full of bugs that I have fastidiously ignored. Upgrade recommended!

If you’re interested in the details I have previously blogged about adding .NET 2.0 support to Json.NET here.

Error Handling

Json.NET now has an Error event on the JsonSerializer and an OnErrorAttribute for handling serialization errors. When an error occurs you can access the exception, see details about the object and member the error occurred on and choose between setting the error as handled and letting it be thrown to your application.
List<string> errors = new List<string>();

List<DateTime> c = JsonConvert.DeserializeObject<List<DateTime>>(@"[

""2009-09-09T00:00:00Z"",

""I am not a date and will error!"",

[

    1

],

""1977-02-20T00:00:00Z"",

null,

""2000-12-01T00:00:00Z""

]",

new JsonSerializerSettings

    {

      Error = delegate(object sender, ErrorEventArgs args)

        {

          errors.Add(args.ErrorContext.Error.Message);

          args.ErrorContext.Handled = true;

        },

      Converters = { new IsoDateTimeConverter() }

    });

// 2009-09-09T00:00:00Z

// 1977-02-20T00:00:00Z

// 2000-12-01T00:00:00Z

// The string was not recognized as a valid DateTime. There is a unknown word starting at index 0.

// Unexpected token parsing date. Expected String, got StartArray.

// Cannot convert null value to System.DateTime.
In this example we are deserializing a JSON array to a collection of DateTimes. On the JsonSerializerSettings a handler has been assigned to the Error event which will log the error message and mark the error as handled.

The result of deserializing the JSON is three successfully deserialized dates and three error messages: one for the badly formatted string, "I am not a date and will error!", one for the nested JSON array and one for the null value since the list doesn't allow nullable DateTimes. The event handler has logged these messages and Json.NET has continued on deserializing the JSON because the errors were marked as handled.

Find more documentation about error handling here.

DataBinding

Lots of small additions have been made to LINQ to JSON to enable easy databinding.
- Added type descriptor objects under the Newtonsoft.Json.Linq.ComponentModel namespace
- Added ITypedList and IBindingList implementations to JContainer
- Added INotifyPropertyChanging and INotifyPropertyChanged implementations to JObject
As someone who is traditionally a web programmer it has been really interesting looking into the world of WPF databinding in more detail.

Release vs Beta

Finally one a minor detail the eagle eyed might have already noticed: from now on Json.NET updates are going to be called Releases rather than Betas.

Changes

Here is a complete list of what has changed since Json.NET 3.5 Beta 4.
- New feature - Added .NET 2.0 support
- New feature - Added serialization error handling via Error event on JsonSerializer and OnErrorAttribute
- New feature - Added implicit conversions from primitive values to JValue
- New feature - Added type descriptor objects to support LINQ to JSON databinding
- New feature - Added ITypedList and IBindingList to JContainer
- New feature - Added INotifyPropertyChanging and INotifyPropertyChanged to JObject
- New feature - Added ConstructorHandling to allow object creation using non-public default constructors
- New feature - Added roundtrip ADO.NET Entities serialization and deserialization
- New feature - Added set indexer to JToken
- Change - IsRequired now accepts null properties
- Change - JsonSerializer now creates JObject/JArray when deserializing complex JSON to an object type
- Change - Empty string to be deserialized as null for nullable types
- Change - CloneNode on JToken renamed to CloneToken
- Change - JProperty constructor that doesn't take a value made internal
- Fix - Schema generation with negative enum values
- Fix - Exception when loading JObject with multiple properties of the same name. Last property will be used
- Fix - Exception when deserializing null value to List<Nullable<T>>
- Fix - Generic type deserialization with type name tracking enabled
Links

Json.NET CodePlex Project

Json.NET 3.5 Release 5 Download – Json.NET source code, documentation and binaries
Tweet
Oct 6 2009

LogCategory

Tweet
Oct 4 2009

Sunday Podcasts 3

The Penny Arcade Podcast – For The Vin

The authors of the comic website Penny Arcade record their thought process as they write a comic strip.

In this episode they discuss their experiences of being molested at Disneyland by costumed characters, Swedes doing time in the big house for IP infringement, fantasy books and recruiting Vin Diesel into their DnD party.

This American Life – The Fix Is In

Corporate crime is fascinating. Your average burglar isn’t the sharpest tool in the shed and neither is the crime he commits so it is interesting to see what happens when smart people decide that the law doesn’t apply to them. Examples: Enron scandal, Madoff’s $65 billion Ponzi scheme.

The Fix Is In is a rerun show from 2000 about a global price fixing conspiracy and the executive, Mark Whitacre, who cooperated with the FBI in exposing it. Whitacre goes to amazing lengths to gather evidence and was described by the FBI as the most cooperative witness in their history. A film has even been made about his exploits: The Informant! – appropriately a comedy.

Awesome.

Tweet
Sep 24 2009

TechEd 2009 New Zealand In A NutShell

Last night I co-presented an Intergen Twilight at our Wellington office, talking to clients about TechEd this year. My partner in crime for the talk was Daniel Horwood who was the man behind the Hands on Labs at TechEd this year.

I think it went really well and we got a lot of good feedback talking to customers after the presentation.

Download the slides of our talk here (30megs).

Tweet
Sep 9 2009

Json.NET on the iPhone

Word on the twitter is that Json.NET runs on the iPhone via Monotouch. Very cool.

Tweet
Sep 2 2009

Backstage With Office 2010 Stats

In the 6 weeks since it went live the Backstage With Office 2010 website has had 100,000 video views, 500 comments and zero bugs. Not bad!

Another video has been added – The Making of Office Web Application. Check it out here.

Update:

The irony of advertising a Microsoft video in an Apple player just occurred to me. Heh.

Update update:

After looking it up I’m reasonably confident that that was a correct use of the word irony. These things keep me up at night.

Tweet
Aug 16 2009

Bizzaro Referendum

There is a referendum currently being voted on in New Zealand under the strangest circumstances.

Background

A couple of years ago New Zealand politicians voted to remove “reasonable force” as a defence for violence by a parent against their child. Both sides were very passionate with cries of “Won’t someone think of the children?” from supporters of the law change and rendering of clothing and gnashing of teeth by opponents, who dubbed it the “anti-smacking bill”.

Referendum

Fast forward to now and opponents to the bill have gathered enough signatures to force a non-binding referendum to be put to New Zealanders. The referendum question is:

Should a smack as part of good parental correction be a criminal offence in New Zealand?

Opponents of the law change to remove reasonable force are urging people to vote no it shouldn’t be a criminal offence, supporters of the law change to remove reasonable force are urging people to vote yes, make it a criminal offence.

What makes the referendum bizzaro is when the reasonable force clause was removed as a defence, an additional clause was added.

To avoid doubt, it is affirmed that the Police have the discretion not to prosecute complaints against a parent of a child or person in the place of a parent of a child in relation to an offence involving the use of force against a child, where the offence is considered to be so inconsequential that there is no public interest in proceeding with a prosecution.

I read that as if you the parent smack your child, as part of a good parental correction, the New Zealand police aren’t going to do anything. What the referendum is asking for is already exists! (there is some difference but the end result is the same: you don’t go to jail for a light smack).

Bizzaro

So the bizzaro situation we are in is that opponents of the current law (reasonable force is not allowed as a defence) are showing their opposition to the current law by voting to keep the current law. Supporters of the current law are showing their support for the current law with a vote for change (all smacking illegal).

Bizzare!

Tweet
Aug 2 2009

.NET 2.0 support added to Json.NET
“There are those who said this day would never come. What are they to say now?” – Prophet of Truth, Halo 2

As of the latest source code commit, Json.NET 3.5 supports running on the .NET 2.0 framework. Yes you read that right all you dot-net-two-point-oh-ers, next release expect a hot, hot Newtonsoft.Json.Net20.dll coming your way.

LINQ enabling a .NET 2.0 application

The main barrier to getting Json.NET running on .NET was LINQ – both usages internally inside Json.NET and unsurprisingly for LINQ to JSON. Fortunately getting LINQ running in a .NET 2.0 application is just a couple of steps.

1. Set the project target framework

The first thing you will need is VS2008 – it has the syntax support for extension methods and lambda expressions. VS2008 also has the ability to target a project against a specific version of the .NET framework. By setting the target framework to .NET 2.0 Visual Studio will remove all the assemblies from that project that have been added post 2.0 such as WCF, WPF, LINQ to SQL, LINQ to XML, etc.

2. Reference LINQBridge

Although VS2008 projects targeting .NET 2.0 still support extension methods and lambda expressions in code, without the LINQ assemblies added in 3.5 none of the LINQ extension methods (Select, Where, OrderBy) are available to use.

The solution is LINQBridge, an open source reimplementation of the LINQ classes and assembly methods. LINQBridge looks and acts exactly the same as LINQ in .NET 3.5, right down to the System.Linq namespace. Just add a reference to LINQBridge.dll to the project and you’re done. Enjoy LINQ in your .NET 2.0 application!

3. Preprocessor #if tests – Optional

This step is only necessary if you’re targeting multiple framework versions using multiple projects. The preprocessor lets you enable and disable portions of code based on conditional compilation symbols defined in a project. The Newtonsoft.Json.Net20 project defines the NET20 symbol and is used to disable portions of code that deal with functionality not available in .NET 2.0.

4. Merge LINQBridge.dll into your application – Optional

If you don’t want to distribute LINQBridge.dll with your application then ILMerge is a free command line app from Microsoft for merging .NET assemblies together. In my Json.NET build script I have used ILMerge to build LINQBridge.dll into Newtonsoft.Json.Net20.dll. The end user will never have to worry about the additional file.
ilmerge.exe /internalize /out:Newtonsoft.Json.Net20.Merged.dll Newtonsoft.Json.Net20.dll LinqBridge.dll
And that’s it. The next version of Json.NET will include the new 2.0 assembly or download the source code for yourself now.
Tweet
Jul 30 2009

JSON Standard Business Card

Front:

Back:

Awesome.

I think it is pretty cool that the entire standard can be put on the back of a business card (and a quarter of that is for defining what a number is!).

From here.

Tweet
Jul 28 2009

Implicit Conversions and LINQ to JSON
I have been doing a lot of work with Json.NET recently and have really been using LINQ to JSON in anger for the first time.

Pretty quickly I became annoyed with having to constantly wrap .NET primitive objects (int, long, string, DateTime, etc) in JValue whenever they’re added to a JSON object or array.
JObject moss = new JObject();

moss["FirstName"] = new JValue("Maurice");

moss["LastName"] = new JValue("Moss");

moss["BirthDate"] = new JValue(new DateTime(1977, 12, 30));

moss["Department"] = new JValue("IT");

moss["JobTitle"] = new JValue("Support");

Console.WriteLine(moss.ToString());

//{

// "FirstName": "Maurice",

// "LastName": "Moss",

// "BirthDate": "\/Date(252241200000+1300)\/",

// "Department": "IT",

// "JobTitle": "Support"

//}
Gross. Why?

For all the LINQy goodness I’ve added to Json.NET to work everything has to inherit from a common object: JToken. Container objects (JObject and JArray) inherit from JToken and all their child values must also be JTokens. For that reason JObject implements IDictionary<string, JToken> and JArray implements IList<JToken>.

Because the containers only allow JTokens, basic primitive values need to be wrapped in an object which inherits from JToken: JValue.

Each time we want to set a value on an object or add a value to an array it has to be manually wrapped in a JValue. Ugly!

Implicit Conversions

The solution is really simple: implicit conversions. Adding an implicit conversion operator allows .NET to automatically convert from the primitive to JValue without any of the above unpleasantness.
///<summary>

/// Performs an implicit conversion from <see cref="String"/> to <see cref="JToken"/>.

///</summary>

///<param name="value">The value to create a <see cref="JValue"/> from.</param>

///<returns>The <see cref="JValue"/> initialized with the specified value.</returns>

public static implicit operator JToken(string value)

{

return new JValue(value);

}
And here is the new end user experience:
JObject jen = new JObject();

jen["FirstName"] = "Jen";

jen["LastName"] = "Barber";

jen["BirthDate"] = new DateTime(1978, 3, 15);

jen["Department"] = "IT";

jen["JobTitle"] = "Manager";
Much better I think you’ll agree. I only wish I had done it earlier [:)]
Tweet
Jul 20 2009

Reductio ad Hitlerum

Not only does this Wikipedia article exist, it is going on 5 years old and has been edited over 600 times. I find this fantastic.

Tweet
Jul 14 2009

Backstage With Office 2010 Website Live!
The past two weeks I have been working long hours with a small band from Intergen on the Backstage With Office 2010 website. I can happily say that today Backstage went live with the unveiling of Microsoft Office 2010!

Backstage is full of videos from the Office 2010 team who go behind the scenes of Office development and talk about what they have been busy working on since 2007. The main Backstage website is 100% Silverlight shows off lots of neat features.
- 100% Silverlight design
- Smooth streaming – videos will scale from HD for awesome broadband connections down to speed optimized resolutions for mobile devices
- Embedded social links to those sites all the cool kids are using (Twitter, Facebook, Delicious, Digg, Live)
- Deep linking directly to videos
- Commenting and rating videos, video view counts
As well as the main site we have created an RSS feed of recently added videos. Not only can you subscribe to it an RSS reader but the feed also works with iTunes and other video podcast clients.

Finally visiting mobile users will automatically be redirect to an HTML mobile version of the site, complete with low resolution videos for 3G mobile devices. It looks fantastic on my iPhone.

Its been a crazy ride getting everything done in time for the launch but I think the end result was worth it. Check out Backstage With Office 2010 here.
Tweet

Jul 11 2009

ADO.NET Entity Framework support accidently added to Json.NET

I sat down today resolved to finally add support for serializing ADO.NET Entities objects… and instead I came away surprised to find that it already worked. My initial “Lets prove it breaks first and then fix it” test passed first time and it wasn’t until I debugged through the test that I was convinced NUnit wasn’t the one that was broken.

My simple test Entities model of a file system:

Marge: "Homer, the plant called. They said if you don't show up tomorrow don't bother showing up on Monday." Homer: "Woo-hoo. Four-day weekend!"

And the JSON serialized with a couple of folders and files in it:

Folder rootFolder = CreateEntitiesTestData();

string json = JsonConvert.SerializeObject(rootFolder, Formatting.Indented);

Console.WriteLine(json);

//{

//  "$id": "1",

//  "FolderId": "a4e8ba80-eb24-4591-bb1c-62d3ad83701e",

//  "Name": "Root folder",

//  "Description": null,

//  "CreatedDate": "\/Date(978087000000)\/",

//  "Files": [],

//  "ChildFolders": [

//    {

//      "$id": "2",

//      "FolderId": "484936e2-7cbb-4592-93ff-b2103e5705e4",

//      "Name": "Child folder",

//      "Description": null,

//      "CreatedDate": "\/Date(978087000000)\/",

//      "Files": [

//        {

//          "$id": "3",

//          "FileId": "cc76d734-49f1-4616-bb38-41514228ac6c",

//          "Name": "File 1.txt",

//          "Description": null,

//          "CreatedDate": "\/Date(978087000000)\/",

//          "Folder": {

//            "$ref": "2"

//          }

//        }

//      ],

//      "ChildFolders": [],

//      "ParentFolder": {

//        "$ref": "1"

//      }

//    }

//  ],

//  "ParentFolder": null

//}

That looks pretty good!

Features added in Beta 4, reference tracking and support for the DataContract/DataMember attributes from WCF, means that instead of Json.NET exploding in a heap of timber (complete with lone wagon wheel spinning off into the distance) at the mere site of an ADO.NET Entities object, serializing now works out of the box!

I love it when things come together [:)]

Jul 9 2009

OpenXmlDeveloper.org – An Introduction to the OpenXML SDK 2.0
Some time ago I wrote an article on version 2.0 of the OpenXML SDK, focusing on the tools that come with it to help you develop OpenXML applications.
- Open XML Classes Explorer
- Open XML Diff
- Document Reflector
Check it out yo.
Tweet
Jul 6 2009

Json.NET 3.5 Beta 4 – JsonSerializer Improvements Part Deux
1. Best. Serializer. Ever.
2. ???
3. Profit!
JsonSerializer Improvements Part Deux

A huge amount has changed in the Json.NET serializer since beta 3. As well as a general refactor (the JsonSerializer.cs file was pushing 1000 lines last release), tons and tons of new features have been added.

New to the serializer is reference tracking, type tracking, serialization callback events, MetadataTypeAttribute support for when working with code generation, additional property level control over serialization settings, a BinaryConverter to convert byte arrays, a StringEnumConverter to convert enum values to their string name, a Populate method for populating JSON values onto an existing object, additional programmatic control over serialization via the IReferenceResolver interface (it replaces IMappingResolver), and much more…

There is too much to cover in one blog post so I’m going to split everything up over a couple of weeks. If you’re really eager for the inside scoop then check out the Json.NET help.
I’m going to be adding more as I go. The first topic to be covered is my favourite new feature, one that very few serializers support and no JSON serializers anywhere as far as I know: Object reference tracking

Preserving Object References

By default Json.NET will serialize all objects it encounters by value. If a list contains two Person references, and both references point to the same object then the JsonSerializer will write out all the names and values for each reference.
Person p = new Person

{

    BirthDate = new DateTime(1980, 12, 23, 0, 0, 0, DateTimeKind.Utc),

    LastModified = new DateTime(2009, 2, 20, 12, 59, 21, DateTimeKind.Utc),

    Name = "James"

};

List<Person> people = new List<Person>();

people.Add(p);

people.Add(p);

string json = JsonConvert.SerializeObject(people, Formatting.Indented);

//[

// {

//    "Name": "James",

//    "BirthDate": "\/Date(346377600000)\/",

//    "LastModified": "\/Date(1235134761000)\/"

// },

// {

//    "Name": "James",

//    "BirthDate": "\/Date(346377600000)\/",

//    "LastModified": "\/Date(1235134761000)\/"

// }

//]
In most cases this is the desired result but in certain scenarios writing the second item in the list as a reference to the first is a better solution. If the above JSON was deserialized now then the returned list would contain two completely separate Person objects with the same values. Writing references by value will also cause problems on objects where a circular reference occurs.

PreserveReferencesHandling

Settings PreserveReferencesHandling will track object references when serializing and deserializing JSON.
string json = JsonConvert.SerializeObject(people, Formatting.Indented,

new JsonSerializerSettings { PreserveReferencesHandling = PreserveReferencesHandling.Objects });

//[

// {

//    "$id": "1",

//    "Name": "James",

//    "BirthDate": "\/Date(346377600000)\/",

//    "LastModified": "\/Date(1235134761000)\/"

// },

// {

//    "$ref": "1"

// }

//]

List<Person> deserializedPeople = JsonConvert.DeserializeObject<List<Person>>(json,

new JsonSerializerSettings { PreserveReferencesHandling = PreserveReferencesHandling.Objects });

Console.WriteLine(deserializedPeople.Count);

// 2

Person p1 = deserializedPeople[0];

Person p2 = deserializedPeople[1];

Console.WriteLine(p1.Name);

// James

Console.WriteLine(p2.Name);

// James

bool equal = Object.ReferenceEquals(p1, p2);

// true
The first Person in the list is serizlied with the addition of an object Id. The second Person in JSON is now only a reference to the first.

With PreserveReferencesHandling on now only one Person object is created on deserialization and the list contains two references to it, mirroring what we started with.

IsReference on JsonObjectAttribute, JsonArrayAttribute and JsonPropertyAttribute

The PreserveReferencesHandling setting on the JsonSerializer will change how all objects are serialized and deserialized. For fine grain control over which objects and members should be serialized there is the IsReference property on the JsonObjectAttribute, JsonArrayAttribute and JsonPropertyAttribute.

Setting IsReference on JsonObjectAttribute or JsonArrayAttribute to true will mean the JsonSerializer will always serialize the type the attribute is against as a reference. Setting IsReference on the JsonPropertyAttribute to true will serialize only that property as a reference.
[JsonObject(IsReference = true)]

public class EmployeeReference

{

public string Name { get; set; }

public EmployeeReference Manager { get; set; }

}
IReferenceResolver

To customize how references are generated and resolved the IReferenceResolver interface is available to inherit from and use with the JsonSerializer.

Changes

Here is a complete list of what has changed since Json.NET 3.5 Beta 3.
- New feature - Added StringEnumConverter to convert enum values to and from their string name rather than number value
- New feature - Added BinaryConverter which converts byte array's, Binary and SqlBinary values to and from base64 text.
- New feature - Added NullValueHandling, DefaultValueHandling and ReferenceLoopHandling to JsonPropertyAttribute
- New feature - Added MetadataTypeAttribute support when searching for attributes
- New feature - JsonSerializer now looks for DataContractAttribute and DataMemberAttribute on a type
- New feature - Now able to explicitly serialize private members when marked up with JsonPropertyAttribute or DataMemberAttribute
- New feature - Added CustomCreationConverter. Used to customize creation of an object before the serializer populates values
- New feature - Added Populate method to JsonSerializer. Pass existing object to serializer and have current object's values populated onto it
- New feature - Added IsReference to JsonContainerAttribute and JsonPropertyAttribute
- New feature - Added PreserveReferencesHandling to JsonSerializer
- New feature - Added IReferenceResolver (replacing IMappingResolver) to JsonSerializer
- New feature - JsonObjectAttribute will now force a collection class to be serialized as an object
- New feature - Added JsonContract, JsonObjectContract, JsonArrayContract and JsonDictionaryContract
- New feature - Added support for OnSerializing, OnSerialized, OnDeserializing, OnDeserialized callback methods
- Change - Rename JsonTokenReader, JsonTokenWriter, JsonTokenType to JTokenReader, JTokenWriter, JTokenType
- Change - DefaultDateTimeFormat on IsoDateTimeConverter no longer displays milliseconds zeros
- Change - JObject now enumerates over KeyValuePair<string, JToken> rather than JToken
- Change - Moved serialize stack used to check for reference loops from JsonWriter (yuck) to JsonSerializerWriter (yay)
- Change - Renamed JsonMemberMapping to JsonProperty
- Fix - JToken now successfully casts to a float or decimal value
- Fix - Serializer now handles comments encountered in JSON while deserializing
- Fix - Fixed (hopefully) cache threading issues
- Fix - Uri objects are now correctly serizlized on Silverlight/Compact Framework
- Fix - Whole decimals will now always be written with a decimal place
Links

Json.NET CodePlex Project

Json.NET 3.5 Beta 4 Download – Json.NET source code, documentation and binaries
Tweet

Jul 3 2009

Simple .NET Profanity Filter

A website I am working on right now accepts public comments, and one of the requirements is to do a basic check for dirty language. Surprisingly for such a common problem I wasn’t able to find any code on the net that did what I wanted and so I’ve ended up writing my own.

The Censor class is pretty simple: you give it a list of words you want to censor, either simple text or with wildcards, and the censor will star out any matches it finds.

IList<string> censoredWords = new List<string>

  "gosh",

  "drat",

  "darn*",

};

Censor censor = new Censor(censoredWords);

string result;

result = censor.CensorText("I stubbed my toe. Gosh it hurts!");

// I stubbed my toe. **** it hurts!

result = censor.CensorText("The midrate on the USD -> EUR forex trade has soured my day. Drat!");

// The midrate on the USD -> EUR forex trade has soured my day. ****!

result = censor.CensorText("Gosh darnit, my shoe laces are undone.");

// **** ******, my shoe laces are undone.

The first example is a simple whole word match on gosh. The second example replaces drat but doesn’t star out the drat in midrate. The final example shows the censor starting out multiple matches and also matching darnit against the wildcard darn*.

I’m passing a collection of strings in my examples but it is easy enough to find a list of swear words on the net, put them in a text file and call something like File.GetAllLines to get an array of words to filter on.

The code:

public class Censor

  public IList<string> CensoredWords { get; private set; }

  public Censor(IEnumerable<string> censoredWords)

    if (censoredWords == null)

      throw new ArgumentNullException("censoredWords");

    CensoredWords = new List<string>(censoredWords);

  public string CensorText(string text)

    if (text == null)

      throw new ArgumentNullException("text");

    string censoredText = text;

    foreach (string censoredWord in CensoredWords)

      string regularExpression = ToRegexPattern(censoredWord);

      censoredText = Regex.Replace(censoredText, regularExpression, StarCensoredMatch,

        RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);

    return censoredText;

  private static string StarCensoredMatch(Match m)

    string word = m.Captures[0].Value;

    return new string('*', word.Length);

  private string ToRegexPattern(string wildcardSearch)

    string regexPattern = Regex.Escape(wildcardSearch);

    regexPattern = regexPattern.Replace(@"\*", ".*?");

    regexPattern = regexPattern.Replace(@"\?", ".");

    if (regexPattern.StartsWith(".*?"))

      regexPattern = regexPattern.Substring(3);

      regexPattern = @"(^\b)*?" + regexPattern;

    regexPattern = @"\b" + regexPattern + @"\b";

    return regexPattern;

Jun 22 2009

.NET Ajax Survey 2009

Json.NET usage up 193% *

* Chris immediately started to point out the statistical flaws of the survey. My response was that regardless, Json.NET’s share of the flawed survey market has increased [;)]

Tweet
Jun 21 2009

Sunday Podcasts 2

This American Life - Classifieds

This American Life is a fantastic weekly podcast about everyday aspects of life. Each week’s show has a theme and then a number of stories around that theme.

This episode is based on stories gathered from classified ads, all found from a single single newspaper on a single day. A man who posted an ad about a missing dog, a personnel ad to win back a lost love, and my favourite: assembling a disparate group of musicians from classified ads to form a band for one day only and record a song – Rocket Man.

Jon Langford and the One Day Band - Rocket Man.mp3:

Hanselminutes – Visiting Fog Creek Software and Joel Spolsky

Scott Hanselman interviews Joel Spolsky about technology, business, blogging and shark jumping. I can’t say I agree with many of Spolsky’s opinions on programming (eww) but he has a good head for business and marketing.

While Fog Creek is most known for a project management tool called Fogbuz they also have created a remote desktop service called Copilot. A DVD I brought not long after I started my first software development job is a documentary about Copilot called Aardvark'd: Twelve Weeks with Geeks. Aardvark is a fun little movie about the software and the small group of interns who made it. Recommended.

Tweet
Jun 18 2009

ThreadSafeStore
Developers and multithreading fall into 3 camps:
- Doesn’t know anything. This developer avoids thinking about other threads at all costs. Followers of the HttpContext. 90% of developers.
- Knows everything. This developer is either a savant or writes operating systems for fun. Most likely sports a sweet hippy beard. 3.14159% of developers.
- Doesn’t know everything but knows enough. This developer will happily write multithreaded applications. It turns out however he/she doesn’t know enough and said applications are full of hard to find, intermittent bugs just waiting to corrupt data and deadlock. The rest us.
I fall into the third category: knows enough to be dangerous.

The latest example of my thread safety failure is knowing that two threads modifying a dictionary at the same time is very bad, but then not considering that getting from a dictionary that is being modified is also not a terribly good idea. Thanks to Amir for the pointer.

ThreadSafeStore

I’ve written a simple helper class that wraps a Dictionary called ThreadSafeStore. ThreadSafeStore treats its internal Dictionary as immutable. Each time a new value is added the wrapper will create a new Dictionary with the new value and reassigns the internal reference. There is no chance a thread could access the Dictionary while it is being modified. ThreadSafeStore is aimed towards read performance with no lock on read required.

The downside to this approach is a new Dictionary is being created with every add. An improvement would be to introduce a second level Dictionary and buffer up new values before adding them to the main store, reducing total object’s allocated. For now I’ve left it simple. Suggestions welcome [:)]
public class ThreadSafeStore<TKey, TValue>

{

private Dictionary<TKey, TValue> _store;

private readonly Func<TKey, TValue> _creator;

public ThreadSafeStore(Func<TKey, TValue> creator)

{

    if (creator == null)

      throw new ArgumentNullException("creator");

    _creator = creator;

}

public TValue Get(TKey key)

{

    if (_store == null)

      return AddValue(key);

    TValue value;

    if (!_store.TryGetValue(key, out value))

      return AddValue(key);

    return value;

}

private TValue AddValue(TKey key)

{

    lock (this)

    {

      TValue value;

      if (_store == null)

      {

        _store = new Dictionary<TKey, TValue>();

        value = _creator(key);

        _store[key] = value;

      }

      else

      {

        // double check locking

        if (_store.TryGetValue(key, out value))

          return value;

        Dictionary<TKey, TValue> newStore = new Dictionary<TKey, TValue>(_store);

        value = _creator(key);

        newStore[key] = value;

        _store = newStore;

      }

      return value;

    }

}

}
Tweet
Jun 15 2009

CodeCamp 2009 - .NET 4.0 Code Contracts

Over the weekend I presented a session on .NET 4.0 Code Contracts at Code Camp 2009.

One of my favourite announcements at PDC2008, I volunteered too talk about Code Contracts to force myself into learning everything I could about them [:D]

The slides for my presentation can be downloaded here.

Thanks to the organisers and other speakers for all the great content. Stand out session goes to Ivan for not only covering everything new in C# 4.0 in 4 minutes but also showing us all some sweet dance moves.

Tweet
May 28 2009

Json.NET @ Twitter

Undoubtedly I’m 6 months behind the times but I just discovered Twitter search. It is rather nifty being able to search public conversations in real time and I begin to see now why Google is interested in them.

The nice thing about Json.NET having a unique name is that it makes it very easy to search for.

Tweet
May 21 2009

Woooo

A cool milestone. Not as elite as this one but a good deal more round: Fifty thousand Json.NET downloads since hosting on CodePlex a year and a half ago.

Json.NET CodePlex Stats

Tweet
May 14 2009

Best. Serializer. Ever.
I have really enjoyed some of the discussion on the Json.NET CodePlex forum recently. Lots of people have posed interesting scenarios and it has been fun coming up with solutions that makes Json.NET a better tool for everyone.

With practically every request these days being something around serialization I have set a goal: make Json.NET the best serializer for .NET

See that underline? That just happened. No pressure, no diamonds folks.

Ideas Wanted

I think the JsonSerializer is most of the way there already but I know there are places where it could do better.

What I’m most interested in are:
- What it is missing compared the old XmlSerializer and new DataContractSerializer (FYI I just added tracking object references)
- What common scenarios it handles poorly
- What popular frameworks it has trouble with
- Anything that is already there but you think is stupid and dumb
- General ideas or suggestions
- Useful JsonConverters you might have written that would be worth including for everyone
Most ideas come from you guys (I have no imagination) and all feedback is welcome.
Tweet
May 2 2009

Json.NET Beta versus Final

Some Json.NET users have asked me whether it is ok to use Json.NET while it is in beta, or have asked if the a final version will be released soon because they need a recently added feature and their employer has a policy against beta software.

For Json.NET the only difference between beta and final is completed documentation.

Json.NET’s testing is completely automated. Approaching 400 unit tests and 90% code coverage (not 100% but good enough) they are the beginning and the end of Json.NET’s test policy. A final release is no different to beta or even a regular source code check-in when it comes to stability.

I think one regression over 3 years and many releases is doing pretty well [:)]

Tweet
Apr 25 2009

No bandwidth MSDN: Reflector
The low bandwidth version of MSDN has gotten a lot of attention recently. Allow me to suggest an alternative for when you're trying to figure out how a class or method works, one which happily doesn’t use any bandwidth at all: Reflector.

Reflector is a .NET assembly browser from Red Gate. It free and weighs in at a svelte 1MB.

What Reflector does is it disassembles any .NET dll on the fly and provides a nice interface to navigate, search, and analyze the assembly’s class hierarchy. Quickly searching over the types in a dll is useful but the really cool thing about Reflector is it goes a step further and decompiles the contents of properties and methods, displaying them in C#.

The reason why I prefer Reflector over MSDN is simple: What better way of discovering what a class or method does than by reading the code itself?

I don’t say that to be masochistic and make things hard or to lord superiority over anyone who does use MSDN but because I think that after writing code, the next best thing you can do to become a better developer is to read it.
- Code comprehension. Being comfortable with understanding other peoples code is a key skill for a developer to have (we spend far more time reading code than writing it). Digging through raw undocumented code in Reflector and figuring out what it does is certainly a great way to practice.
- New ideas. Whether you’re checking out how the LINQ where statement works or you’re browsing what goes on inside the .NET configuration classes, you will see a lot of different problems being solved in different ways. The more you see something solved the better you’ll be able to tell a good approach from a bad one.
- .NET isn’t magic. Seeing a chart of the ASP.NET page lifecycle is one thing, but opening up the Page class in Reflector and seeing it happen in code in the ProcessRequestMain() is another [:)]
There are cases when checking out MSDN is the right thing to do to figure out how something works but your first port of call should be Reflector.

Download Reflector and give it a try. You’ll be a better developer for it.
Tweet
Apr 20 2009

TextGlow CodePlex Project

By popular demand TextGlow now has a CodePlex Project.

TextGlow CodePlex Project

Go go forth and, er, collaborate!

Tweet
Apr 18 2009

Blog Driven Design
It is time I coined a programming term: Blog Driven Design

Blog Driven Design is discovering code is too verbose to reasonably include in a blog post and is the additional work put into the design to trim it down to a blogable size.

Example

A recent beta of Json.NET added schema support. While writing the blog post showing the new feature in action I discovered there was no quick way go from a string to a JsonSchema object.
JsonSchema schema;

using (JsonTextReader reader = new JsonTextReader(new StringReader(schemaJson)))

{

JsonSchemaBuilder builder = new JsonSchemaBuilder(new JsonSchemaResolver());

schema = builder.Build(reader);

}
Yuck. What looked ok in a unit test is now excessive and ugly. A short example is a good example in my opinion so the JsonSchema.Parse(string) method was added to hide that work away. The end result looked like this:
JsonSchema schema = JsonSchema.Parse(schemaJson);
Not only is the code example in the blog post much smaller and simpler but end users can use the new Parse method as well. BDD in action!

Behaviour Driven Development watch out, Blog Driven Design is coming for your initials [:D]
Tweet
Apr 12 2009

Native JSON in IE8, Firefox 3.5 plus Json.NET
Native JSON is a new feature to IE8 and Firefox 3.5. Built in serialization and deserialization in the browser makes evaling JSON text a thing of the past.

This post looks at how to use native JSON and how it can complement Json.NET’s server side JSON support by parsing and generating JSON on the browser.

Native JSON API

Using the new native JSON API is pretty simple. To turn JSON into a JavaScript object simply pass a string to JSON.parse:
var jsonText = '{"name":"Frodo","address":"Hobbiton, The Shire"}';

var person = JSON.parse(jsonText);

alert(person.name);

// Frodo
And to go from an object back to a JSON string use the interestingly named JSON.stringify function:
var jsonText = JSON.stringify(person);

// {"name":"Frodo","address":"Hobbiton, The Shire"}
Because native JSON is based upon the popular json2.js script you can support old browsers by including that file in your webpage. Browsers that don’t support native JSON will now work while browsers with built in JSON support will automatically use their much faster native implementations.

Native JSON + Json.NET

Native JSON and Json.NET work almost flawlessly together. The one wrinkle is the usual suspect when it comes to JSON: dates. Fortunately getting native JSON, Json.NET and dates to work together is a two step process.

The first thing we want to do is make Json.NET write dates in the ISO format. IsoDateTimeConverter to the rescue.
Person p = new Person

            {

              Name = "Keith",

              BirthDate = new DateTime(1980, 3, 8),

              LastModified = new DateTime(2009, 4, 12, 20, 44, 55),

            };

string jsonText = JsonConvert.SerializeObject(p, new IsoDateTimeConverter());

// {

// "Name": "Keith",

// "BirthDate": "1980-03-08T00:00:00",

// "LastModified": "2009-04-12T20:44:55"

// }
Now that we have JSON being generated on the server using Json.NET we want to deserialize it on the client using JSON.parse. Because there is no standard for representing dates in JSON we need to tell the JSON.parse function how to recognise and deserialize a date. Luckily JSON.parse has an optional reviver parameter that we can use to turn ISO formatted date strings into a JavaScript Date objects.
function isoDateReviver(key, value) {

    if (typeof value === 'string') {

        var a = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)(?:([\+-])(\d{2})\:(\d{2}))?Z?$/.exec(value);

        if (a) {

            var utcMilliseconds = Date.UTC(+a[1], +a[2] - 1, +a[3], +a[4], +a[5], +a[6]);

            return new Date(utcMilliseconds);

        }

    }

    return value;

}

var person = JSON.parse(jsonText, isoDateReviver);

alert(person.BirthDate.toUTCString());

// Sat, 8 Mar 1980 00:00:00 UTC

alert(person.LastModified.toUTCString());

// Sun, 12 Apr 2009 20:44:55 UTC
The reviver function uses a regular expression to tests if a string matches an ISO formatted date. If it does a JavaScript Date object is created and it is inserted into the JavaScript object instead of the string.

Conclusion

And that’s it. Server generated JSON using Json.NET will now be parsed natively in the browser, dates and all. The other direction, client generated JSON using JSON.stringify then being read by Json.NET doesn’t require any modification. It works by default. Sweet!
Tweet
Apr 6 2009

TextGlow SharePoint Web Part

Jeroen Ritmeijer has taken the TextGlow source code and integrated it into a SharePoint web part. Check out the article he wrote about the new web part here.

Tweet
Apr 1 2009

Announcing Json.NET Pro

Json.NET is entering an exciting new phase in its lifecycle with the release of a Pro edition!

To celebrate the commercial release for today only there is a special of $195 per developer. To find out more details about Json.NET Pro, and what it will mean for existing users, visit the new storefront here.

Tweet
Mar 29 2009

Adoreably Awesome

These sorts of positive externalities must be supported and maintained!

Via Greg Mankin.

Tweet
Mar 29 2009

TextGlow Open Sourced

The source code for TextGlow, a Silverlight application I wrote and was launched a year ago at MIX08 to much fanfare, has been released as open source under the Microsoft Public License.

TextGlow was one of the first Silverlight 2.0 applications released and it showcased many of the new features and possibilities of Office Open XML and Silverlight.

The source code for TextGlow has been released along with an article published at OpenXML Developer, TextGlow - View Open XML Word documents with Silverlight. The article is a high level overview of the internals of TextGlow to help developers get started with their own applications.

I feel compelled to put in a disclaimer that the source code is somewhat… rough in certain places. I was the sole developer on the project and had just a couple of months to try and recreate as much as Word as possible. Fighting to keep what I had written against the constant stream of pre-MIX Silverlight 2.0 betas didn’t help either [:)]

You can find a link to the source code at the bottom of the article.

Tweet
Mar 24 2009

Json.NET Audio Interview

dev{shaped} has a CodePlex Project of the month feature where they interview the developer of an open source project.

Json.NET is their March project and I did an audio interview with Derek Hatchard, talking about what JSON is, how Json.NET makes using it in .NET easy and the trials and tribulations of managing an open source project.

Listen:

Or download the mp3 here.

If you’re trying to pick the accent, it is New Zealand. Listening to the Kiwi and American accents is very Flight of the Conchordesc.

Tweet
Mar 23 2009

Building Accessible RIAs in Microsoft Silverlight

Chris Auld, Intergen director, and Reed Shaffner, Microsoft Office technical product manager, did a presentation at MIX09 showing off ButtercupReader. The mini breakout session was called Building Accessible RIAs in Microsoft Silverlight and focuses on Silverlight’s accessibility features.

I have worked quite closely with Chris over the last few years on various projects and Reed and I presented a session at TechEd NZ last year. It is great to see them together and the video is well worth a look if you’re interested in Silverlight development.

Tweet
Mar 21 2009

Json.NET 3.5 Beta 3 – JsonSerializer improvements
An oldie but a goodie Your favourite song, and mine Ladies and gentlemen Json.NET!

JsonSerializer Improvements

This release is a rollup of the various changes since beta 2. The majority of the changes are enhancements to the way Json.NET serializes and deserializes JSON.

The most notable enhancement is the addition of IMappingResolver, previously blogged about here: Thoughts on improving JsonSerializer: IMappingResolver?

The IMappingResolver interface provides a way to globally customize how the JsonSerializer works, without any modification to your objects. I should note that the the JsonProperty attribute is still fully supported and is a great way to customize a single property. IMappingResolver is an addition and any existing Json.NET code will continue to work flawlessly.

DefaultMappingResolver

Two implementations of IMappingResolver are included in this release of Json.NET. DefaultMappingResolver is fairly self explanatory. I have tried to structure it in a way that provides many avenues of extensibility. Most of the methods on the class are protected and virtual.

CamelCaseMappingResolver

CamelCaseMappingResolver is the second implementation. This class inherits from DefaultMappingResolver and simply overrides the JSON property name to be camelcase.
Product product = new Product

                    {

                      ExpiryDate = new DateTime(2010, 12, 20, 18, 1, 0, DateTimeKind.Utc),

                      Name = "Widget",

                      Price = 9.99m,

                      Sizes = new[] {"Small", "Medium", "Large"}

                    };

string json =

JsonConvert.SerializeObject(

    product,

    Formatting.Indented,

    new JsonSerializerSettings { MappingResolver = new CamelCaseMappingResolver() }

);

//{

// "name": "Widget",

// "expiryDate": "\/Date(1292868060000)\/",

// "price": 9.99,

// "sizes": [

//    "Small",

//    "Medium",

//    "Large"

// ]

//}
Note that camelcased property names in the serialized JSON.

CamelCaseMappingResolver is just one example of what can be achieve using IMappingResolver. Off the top of my head some other examples could be modifying the JsonSerializer to serialize private fields, or adding additional properties to a serialized JSON object like the .NET type name.

Overall I’m really happy with the way IMappingResolver has turned out. I think it provides a great halfway point between using attributes to control serializing an object, and writing a custom JsonConverter.

Simple things should be simple. Complex things should be possible. –Alan Kay

Changes

Here is a complete list of what has changed since Json.NET 3.5 Beta 2.
- New feature - Added IMappingResolver interface and DefaultMappingResolver class. IMappingResolver property added to JsonSerializer, JsonSerializerSettings, JsonSchemaGenerator
- New feature - Added CamelCaseMappingResolver
- New feature - Added DeserializeNode overload to JsonConvert that allows a root element to be added to the XML
- New feature - Added support for deserializing to IEnumerable<T>, ICollection<T>, IList<T> and IDictionary<TKey, TValue> members
- New feature – Deserializer will now populate an object’s members with unused values after creating an object from a non-default constructor
- New feature - Deserializer now attempts a case insensitive match to a member if the exact case match fails
- New feature - Added a ToString(Formatting, JsonConverters[]) overload to JToken
- New feature - Added AddAfterSelf, AddBeforeSelf methods to JToken
- Change - JsonSerializer now ignores missing members by default
- Fix - Made the error message when attempting to deserialize to an interface or abstract class clearer
- Fix - Fixed the whitespace issues when writing a raw JValue token
- Fix - XmlNodeConverter now handles nested arrays when converting JSON to XML
- Fix - Ensure JavaScriptDateTimeConverter converts nullable DateTime members
- Fix - Fix possible thread safety issues by adding double check locking to static reflection cache
Links

Json.NET CodePlex Project

Json.NET 3.5 Beta 3 Download – Json.NET source code, documentation and binaries
Tweet
Mar 20 2009

Obligatory

Via this site and this application.

Download the DAISY book here.

Tweet
Mar 19 2009

ButtercupReader – A Silverlight Digital Talking Book Reader

For the second Mix in a row Intergen is launching a cool new Silverlight application: ButtercupReader!

ButtercupReader is a free Silverlight 2.0 application for viewing and playing digital talking books (DAISY) on the web by blind and partially sighted users.

As well as using Silverlight to render text and play DAISY document audio, ButtercupReader also showcases many of Silverlight’s accessibility features including screen reader support, shortcut keys, different contrast settings and zoomability. Andrew Tokeley, a member of the ButtercupReader team, has a great blog post on those features here.

I only worked briefly on this project, spiking out functionality at the beginning. It is great to see how it has all come together.

Tweet
Mar 15 2009

Sunday Podcasts

Hanselminutes - Mo Interviews Scott Hanselman

One of my highlights of TechEd NZ 2008 was meeting Scott Hanselman. Chatting about random techie things in the speakers room and working side by side to prepare our presentations was a lot of fun. Here Mo, Scott’s wife, interviews him about what it is like being The HanselMan.

Econtalk - Wales on Wikipedia

Wikipedia is a fantastic example of harnessing the cloud to collaboratively create something amazing. Jimmy Wales talks about what has made Wikipedia successful, dealing with conflict and how Wikipedia is a self managing entity.

Dungeons & Dragons Podcast – Penny Arcade Episode 1

The Penny Arcade guys are always funny and listening to them play D&D is hilarious. The first of episode of many.

Tweet
Mar 1 2009

Thoughts on improving JsonSerializer. IMappingResolver?
JsonSerializer is pretty darn flexible. It has a half dozen options on the class to customize how an object is serialized, attributes for more fine grained control and finally JsonConverters if you need to do something completely different.

The most popular request I get these days is control over how JsonSerializer determines what members on a type to serialize (e.g. serialize private as well as public properties) and control over the name of the written property (e.g. instead of “FirstName” write the property camelcase style: “firstName”).

Controlling this behaviour is actually already possible by inheriting from JsonSerializer and overriding GetMemberMappings. The MemberMapping class contains all the information about how a .NET member gets mapped to a JSON property (name, readable, writable, converter, etc) and is used internally by JsonSerializer. The problem is it isn’t terribly discoverable to the average user.

IMappingResolver

What I am thinking of doing is adding is an IMappingResolver interface.
public interface IMappingResolver

{

JsonMemberMappingCollection ResolveMappings(Type type);

}
JsonSerializer would have a MappingResolver property and the instance of the object assigned to it would determine how the members on a .NET class get mapped to a JSON object (if you’re hip with the GOF urban street lingo this is an example of the strategy pattern).

Examples of resolvers that could come with Json.NET or you could create yourself: CamelCaseMappingResolver, TypeDescriptorMappingResolver, PrivateMembersMappingResolver and so on.

I like this over inheritance because it is obvious and in a user’s face. MappingResolver could also be added to the JsonSerializerSettings class and used from the serialization helper methods on JsonConvert.

Yay or nay? Suggestions welcome [:)]
Tweet
Feb 24 2009

All those moments will be lost in time... like tears in rain... Time to die.

IE7 has been out for two and a half years. It is time to stop living in the past and retire IE6.

IETester

As an aside I highly recommend IETester (see the screenshot above) to anyone needing to test websites in old versions of IE. It renders IE 5.5, 6, 7 and 8 side by side in one application. It even supports tabs!

Tweet
Feb 22 2009

Add Twitter to your blog the right way with caching
Twitter (read more about Twitter here) offers a simple JavaScript API to add your status updates to your blog or other website.

The Problem

Injecting HTML into a webpage like this works well enough for the most part put it can produce problems if the website you are getting HTML from, Twitter in this case, is slow or unreliable. An inline script reference will hang the rendering of the page at that point until a response is returned.

If Twitter.com is down, not an uncommon occurrence, and you have an inline script reference like the one above then your website is essentially broken until Twitter fixes itself.

To get around this problem Twitter advises that the script reference should be placed at the end of a page. That fixes one issue but creates another: The content from Twitter pops into the page only once the page is completely loaded. It looks a little hacky at best and could cause the content on the page to suddenly move and reflow depending upon where the Twitter statuses are being inserted.

An example of content pop-in can be seen in the sidebar of MajorNelson’s blog. Each time the page reloads there is a noticeable delay before the Twitter statues are populated.

The Solution

My solution is simple: cache the result from Twitter in a cookie. Placing the script reference at the bottom of the page gets around the issue of a broken website and caching the result means no content pop-in after the first page view.

The JavaScript is pretty simple.
<script type="text/javascript">

function setCookie(name, value, expires) {

    document.cookie = name + "=" + escape(value) + "; path=/" + ((expires == null) ? "" : "; expires=" + expires.toGMTString());

}

function getCookie(name) {

    if (document.cookie.length > 0) {

      var start = document.cookie.indexOf(name + "=");

      if (start != -1) {

        start = start + name.length + 1;

        var end = document.cookie.indexOf(";", start);

        if (end == -1) {

          end = document.cookie.length;

        }

        return unescape(document.cookie.substring(start, end));

      }

    }

    return "";

}

function twitterCachedCallback(c) {

    // this will create the HTML. Function found inside blogger.js

    twitterCallback2(c);

    var content = document.getElementById("twitter_update_list").innerHTML;

    // expire cookie after 30 minutes

    var exp = new Date();

    exp.setTime(exp.getTime() + (1000 * 60 * 30));

    setCookie('twitter_content', content, exp);

}

// set content immediately if cached

var cachedContent = getCookie('twitter_content');

if (cachedContent) {

    document.getElementById("twitter_update_list").innerHTML = cachedContent;

}

</script>
Add this script immediately after the Twitter content. If cached content is found then it will be inserted into the page straight away, improving user experience.

The only other change that needs to be made is to the script reference to Twitter. Rename the callback function from twitterCallback2 to twitterCachedCallback in the querystring and you’re done.

I use this technique on my blog (look to the sidebar on the right). You can also see an example webpage I threw together here.
Tweet
Feb 21 2009

Good (Date)Times with Json.NET
Latest version of this guide: Serializing Dates in JSON

---

I’ve said it before, I’ll say it again: Dates in JSON are hard.

The problem comes from the JSON spec itself, there is no literal syntax for dates in JSON. The spec has objects, arrays, strings, integers and floats, but it defines no standard for what a date looks like.

The default format used by Json.NET for dates is the same one used by Microsoft: "\/Date(1198908717056)\/". You can read more about it here.

JsonConverters

With no standard for dates in JSON, the number of possible different formats when interoping with other systems is endless. Fortunately Json.NET has a solution to deal with reading and writing custom dates: JsonConverters. A JsonConverter is used to override how a type is serialized.
public class LogEntry

{

public string Details { get; set; }

public DateTime LogDate { get; set; }

}

[Test]

public void WriteJsonDates()

{

LogEntry entry = new LogEntry

{

LogDate = new DateTime(2009, 2, 15, 0, 0, 0, DateTimeKind.Utc),

Details = "Application started."

};

string defaultJson = JsonConvert.SerializeObject(entry);

// {"Details":"Application started.","LogDate":"\/Date(1234656000000)\/"}

string javascriptJson = JsonConvert.SerializeObject(entry, new JavaScriptDateTimeConverter());

// {"Details":"Application started.","LogDate":new Date(1234656000000)}

string isoJson = JsonConvert.SerializeObject(entry, new IsoDateTimeConverter());

// {"Details":"Application started.","LogDate":"2009-02-15T00:00:00Z"}

}
Simply pass the JsonConverter you wish to use to the Json.NET serializer.

JavaScriptDateTimeConverter

The JavaScriptDateTimeConverter class is one of the two DateTime JsonConverters that come with Json.NET. This converter serializes a DateTime as a JavaScript Date object.
new Date(1234656000000)
Technically this is invalid JSON according to the spec but all browsers, and some JSON frameworks including Json.NET, support it.

IsoDateTimeConverter

IsoDateTimeConverter seralizes a DateTime to an ISO 8601 formatted string.
"2009-02-15T00:00:00Z"
The IsoDateTimeConverter class has a property, DateTimeFormat, to further customize the formatted string.

One final thing to note is all date values returned by Json.NET are in UTC time.

JSON makes working with dates harder than it needs to be but with a little work it is a problem that is easily solved. Good times.
Tweet
Feb 14 2009

Writing JSON to a file using Json.NET
There are a couple of different ways you can write to a file using Json.NET. The first is really simple:
Person person = GetPerson();

string json = JsonConvert.SerializeObject(person, Formatting.Indented);

File.WriteAllText(@"c:\person.json", json);
In this C# example we get the string returned from SerializeObject and pass it to WriteAllText, a helper method on the very useful System.IO.File class. The JSON contents of the string is written to the file.

If greater control is required over how the file is written, or the JSON you are writing is large and you don’t want the overhead of having the entire JSON string in memory, a better approach is to use JsonSerializer directly.
Person person = GetPerson();

using (FileStream fs = File.Open(@"c:\person.json", FileMode.CreateNew))

using (StreamWriter sw = new StreamWriter(fs))

using (JsonWriter jw = new JsonTextWriter(sw))

{

jw.Formatting = Formatting.Indented;

JsonSerializer serializer = new JsonSerializer();

serializer.Serialize(jw, person);

}
More code but also more efficient: the JSON is written directly to the file stream. The other bonus to using this approach is that it can be adapted to write to any .NET Stream: file streams, web response streams, memory streams, etc.

Isn’t polymorphism cool? [:)]
Tweet
Feb 9 2009

Googleman

Googleman is a fun and pointless application for launching Google.com in the style of the Batman 60s TV show, complete with spinning Google logo and sound effects. Now when a friend or co-worker says “To Google!” you can open your favourite search engine in style.

Launch Google either from the Googleman tray icon or by hitting the Windows+G hotkey.

Download Googleman

Video Preview

Technical Stuff

Googleman is my first WPF application and it is extremely simple. The Batman 60s splash screen effect is achieved by launching a fullscreen window with two images. The background image is a screenshot of the desktop with a rotate animation applied to it. The logo zoom is done using a scale animation. Once the splash screen animation is complete the window hides itself and launches Google.com.

Kind of ruins the magic when you know how it works.

Background

We have been discussing patterns a fair bit at work so I wrote Googleman using the MVP pattern. I am working on an ASP.NET MVC website at the moment and Googleman was so I could understand the differences between the two patterns. Hopefully I have gotten it right…

Thanks to JD for help with testing and Tokes for insights into MVP.

Disclaimer

Google is serious business. Googleman is not a toy and should only be used in serious business situations. Or for lulz.

Tweet
Feb 8 2009

Teach a man to Google

Give a man a link, you have answered him for today. Teach a man to Google, and he will stop bothering you for a lifetime.

- James Newton-King*, 2009

Tweet
Feb 2 2009

A .NET Open Source Project Retrospective: Json.NET

I have always enjoyed the video game post-mortem articles on Gamesutra. Written by developers for developers, they reflect on how the project went, what they learnt and what they would do differently. Hearing peoples thoughts on what they have created is very interesting to me (I *love* DVD commentries) and have never seen the same done for an open source project.

This post on Json.NET, a retrospective rather than a post-mortem since Json.NET is still alive, follows the same format as Gamesutra: What went right and what went wrong and lessons I have learnt along the way.

Background

Json.NET grew out of projects I was working on in late 2005 involving JavaScript, AJAX and .NET. At the time there were no libraries for working with JSON in .NET so I began to grow my own.

This is the first open source project I publically released and the most popular with over 50,000 downloads.

What Went Right?

1. .NET

The whole purpose was to write a JSON library for .NET, so there was never a choice of which platform to use. However in saying that, .NET has been great to work with. The base class library and the C# programming language are well thought out and are (mostly) a joy to work with. The .NET framework has gotten better with every new release.

The .NET tools stack I use with Json.NET likewise provides an excellent experience. Visual Studio when combined with NUnit, TestDriven.NET and VisualSVN allows for rapid testing and development. Coding, compiling, testing, debugging and manage source control all within the IDE is a great boon for productivity.

2. Incremental releases

I like the quote ‘incremental improvement is better than delayed perfection’. The 1.0 release of Json.NET came from just a couple of weekends of effort and after a flurry of releases to fix the initial bugs, a new major version of Json.NET has been released approximately every 6 months.

While not every feature I wanted was in version 1.0, users soon gave feedback on what they wanted most and I went from there. I haven’t been overly precious about maintaining 100% compatibility between releases but the core behaviour of Json.NET: reading, writing and serializing between .NET and JSON has remained unchanged since day one. New features like LINQ to JSON have slotted in side by side with existing classes.

3. User driven design

Confession time: I have not used JSON in anger on a project in years. Almost every change, bug fix and new feature since Json.NET 1.0 has come from user feedback.

Although it sounds like a negative, I think it turned out to be a great way to develop. The end result is a focus on what users want most rather than relying on your own experiences.

I think a danger to keep in mind with this approach is that user feature requests are generally very specific and will only address one persons need. Make sure you take care to take a step back and look at their intent and then create a solution that works for everyone.

4. Unit Testing

Json.NET, back in 2005, was the first project I used unit testing on and it has been a great success (although the test project is a bit of a mess…). Good unit testing coverage over the entire project has meant I could make wholesale changes to the internal workings of Json.NET to support a new feature while having a good level of security that I wasn’t breaking what was already there.

Each time a bug is reported I have a grown into a pattern where I recreate the bug in a unit test, assert that it is broken, fix the bug and then assert that it is fixed. It is a good way to get coverage of all those edge cases you never thought of.

5. CodePlex

CodePlex is an open source project hosting website from Microsoft. Overall I have found it a great place to host releases, source control and discussion forums for Json.NET. These days it also has good Subversion support and it provides all kinds of interesting statistics.

Probably the thing I like most about using a site like CodePlex is that it makes source code public and easy to download in a zip file. When I deal with a bug and check in the updated files I can straight away tell a user that fixed source code is available and a link to where they can download it.

What Went Wrong?

1. Poor documentation

A well designed API and XML intellisense documentation isn’t enough for the average user. Until Json.NET 2.0 was released, which included CHM documentation, a lot of questions were asked about how to achieve very basic tasks.

When it comes to documentation, it is hard to ever have too much. I think getting started guides and high level overviews of a feature are especially useful to users.

2. No central knowledge base

Json.NET suffers from having too many places to search if you are looking for information… old and new blog posts, a project page, a forum on CodePlex and the CHM documentation. Not everyone is a Google ninja and expecting users to search out a reply you made to someone else asking the same question a couple of years ago is asking a lot.

Directing users to one place to look or ask for help is key. In future I might look to move a lot of Json.NET’s documentation to CodePlex and use its wiki functionality.

Conclusion

I’m really happy with how Json.NET has gone as a project. JSON’s popularity in programming has grown markedly over the last 3 years and Json.NET’s has grown with it. I look forward to what new possibilities .NET 4.0 might bring.

Stats

Numer of developers: 1

Budget: $0

Development Time: 2 1/2 years (and counting)

Initial Release Date: June 2006

Platform: .NET, Silverlight, Compact Framework

Software: Visual Studio, NUnit, TestDriven.NET, VisualSVN, TortoiseSVN, Sandcastle

Size of Project:

Files: 180

Lines of code (including tests): 21,000

Tweet
Jan 4 2009

Json.NET 3.5 Beta 2 – JSON schema validation
Notice:

The JSON Schema functionality in Json.NET has been moved to its own package. Please visit http://www.newtonsoft.com/jsonschema for more details.

I’ve been hard at work on Json.NET over the holidays. I spent most of December setting up environments, writing PowerShell scripts and deploying applications so it was nice to do a little .NET development for a change.

JSON Schema

The big new feature in Beta 2 is JSON schema validation. JSON Schema is a specification for defining the structure of JSON. Think of it as XML Schema for JSON except it is actually readable.
{

"description": "A person",

"type": "object",

"properties":

{

    "name": {"type":"string"},

    "hobbies": {

      "type": "array",

      "items": {"type":"string"}

    }

}

}
I have put together a demo page of the new JSON Schema features: JSON Schema Validation Demo

There are a couple of different ways of validating using JSON Schema in Json.NET. The simplest is using an extension method for the LINQ to JSON objects. IsValid in this example just returns a flag but there are other overloads that let you also capture validation messages.
JsonSchema schema = JsonSchema.Parse(schemaJson);

JObject person = JObject.Parse(@"{

""name"": ""James"",

""hobbies"": ["".NET"", ""Blogging"", ""Reading"", ""Xbox"", ""LOLCATS""]

}");

bool valid = person.IsValid(schema);

// true
As well as validation, for fun, and to help with testing, I wrote a class called JsonSchemaGenerator. This class looks at a .NET type and using reflection it generates a schema for it. The schemas on the demo page are all generated at runtime.

JSON line and position Information

JsonTextReader now reports line and position information. Line and position information was needed for the schema validator to help point out to a user in an error message which part the JSON was invalid. The nice flow on effect of this feature is now when you give Json.NET bad JSON, the resulting exception will tell you the location of the invalid character.
JObject person = JObject.Parse(@"{

""name"": ""James"",

]!#$THIS IS: BAD JSON![{}}}}]

}");

// Invalid property identifier character: ]. Line 3, position 3.
Changes

Here is a complete list of what has changed since Json.NET 3.5 Beta 1.
- New feature - Added JSON Schema implementation
- New feature - Added IJsonLineInfo. Implemented by JsonTextReader, JsonTokenReader, JsonValidatingReader, JToken
- New feature - Added line details to JsonTextReader exception messages and JsonValidatingReader errors
- New feature - Added JsonContainerAttribute with Id, Title and Description members. JsonObject and JsonArray inherit from this attribute
- New feature - Added JsonArrayAttribute. Has flag to control whether array can contain null items
- New feature - Added IsRequired to JsonProperty
- New feature - Added Load(JsonReader) to JProperty, JConstructor
- New feature - Added the ability for JsonTokenWriter to write individual values as well as arrays and objects
- New feature - Added CreateReader to JToken
- New feature - Added FromObject to JToken
- New feature - Added ReadFrom to JToken
- Change - Renamed JavaScriptConvert to JsonConvert
- Change - Value<T> on JObject supports getting nullable values
- Change - Type values now serialize and deserialize to the type name string
- Change - GetSerializableMembers has been removed and replaced with GetMemberMappings on JsonSerializer
- Fix - JsonConvert now always write a floating point number with a decimal place
- Fix - JsonSerializer now correctly skips missing members
- Fix - JsonWriter now allows objects and arrays to be written in a constructor
- Fix - QuoteChar not getting set when parsing property name
- Fix - JProperty now correctly loads content from a collection
Links

Json.NET CodePlex Project

Json.NET 3.5 Beta 2 Download – Json.NET source code, documentation and binaries
Tweet

Archives