Give a man a link, you have answered him for today. Teach a man to Google, and he will stop bothering you for a lifetime.

- James Newton-King*, 2009

I have always enjoyed the video game post-mortem articles on Gamesutra. Written by developers for developers, they reflect on how the project went, what they learnt and what they would do differently. Hearing peoples thoughts on what they have created is very interesting to me (I *love* DVD commentries) and have never seen the same done for an open source project.

This post on Json.NET, a retrospective rather than a post-mortem since Json.NET is still alive, follows the same format as Gamesutra: What went right and what went wrong and lessons I have learnt along the way.

Background

Json.NET grew out of projects I was working on in late 2005 involving JavaScript, AJAX and .NET. At the time there were no libraries for working with JSON in .NET so I began to grow my own.

This is the first open source project I publically released and the most popular with over 50,000 downloads.

What Went Right?

1. .NET

The whole purpose was to write a JSON library for .NET, so there was never a choice of which platform to use. However in saying that, .NET has been great to work with. The base class library and the C# programming language are well thought out and are (mostly) a joy to work with. The .NET framework has gotten better with every new release.

The .NET tools stack I use with Json.NET likewise provides an excellent experience. Visual Studio when combined with NUnit, TestDriven.NET and VisualSVN allows for rapid testing and development. Coding, compiling, testing, debugging and manage source control all within the IDE is a great boon for productivity.

2. Incremental releases

I like the quote ‘incremental improvement is better than delayed perfection’. The 1.0 release of Json.NET came from just a couple of weekends of effort and after a flurry of releases to fix the initial bugs, a new major version of Json.NET has been released approximately every 6 months.

While not every feature I wanted was in version 1.0, users soon gave feedback on what they wanted most and I went from there. I haven’t been overly precious about maintaining 100% compatibility between releases but the core behaviour of Json.NET: reading, writing and serializing between .NET and JSON has remained unchanged since day one. New features like LINQ to JSON have slotted in side by side with existing classes.

3. User driven design

Confession time: I have not used JSON in anger on a project in years. Almost every change, bug fix and new feature since Json.NET 1.0 has come from user feedback.

Although it sounds like a negative, I think it turned out to be a great way to develop. The end result is a focus on what users want most rather than relying on your own experiences.

I think a danger to keep in mind with this approach is that user feature requests are generally very specific and will only address one persons need. Make sure you take care to take a step back and look at their intent and then create a solution that works for everyone.

4. Unit Testing

Json.NET, back in 2005, was the first project I used unit testing on and it has been a great success (although the test project is a bit of a mess…). Good unit testing coverage over the entire project has meant I could make wholesale changes to the internal workings of Json.NET to support a new feature while having a good level of security that I wasn’t breaking what was already there.

Each time a bug is reported I have a grown into a pattern where I recreate the bug in a unit test, assert that it is broken, fix the bug and then assert that it is fixed. It is a good way to get coverage of all those edge cases you never thought of.

5. CodePlex

CodePlex is an open source project hosting website from Microsoft. Overall I have found it a great place to host releases, source control and discussion forums for Json.NET. These days it also has good Subversion support and it provides all kinds of interesting statistics.

Probably the thing I like most about using a site like CodePlex is that it makes source code public and easy to download in a zip file. When I deal with a bug and check in the updated files I can straight away tell a user that fixed source code is available and a link to where they can download it.

What Went Wrong?

1. Poor documentation

A well designed API and XML intellisense documentation isn’t enough for the average user. Until Json.NET 2.0 was released, which included CHM documentation, a lot of questions were asked about how to achieve very basic tasks.

When it comes to documentation, it is hard to ever have too much. I think getting started guides and high level overviews of a feature are especially useful to users.

2. No central knowledge base 

Json.NET suffers from having too many places to search if you are looking for information… old and new blog posts, a project page, a forum on CodePlex and the CHM documentation. Not everyone is a Google ninja and expecting users to search out a reply you made to someone else asking the same question a couple of years ago is asking a lot.

Directing users to one place to look or ask for help is key. In future I might look to move a lot of Json.NET’s documentation to CodePlex and use its wiki functionality.

Conclusion

I’m really happy with how Json.NET has gone as a project. JSON’s popularity in programming has grown markedly over the last 3 years and Json.NET’s has grown with it. I look forward to what new possibilities .NET 4.0 might bring.

Stats

Numer of developers: 1

Budget: $0

Development Time: 2 1/2 years (and counting)

Initial Release Date: June 2006

Platform: .NET, Silverlight, Compact Framework

Software: Visual Studio, NUnit, TestDriven.NET, VisualSVN, TortoiseSVN, Sandcastle

Size of Project:

Files: 180

Lines of code (including tests): 21,000

Notice:

The JSON Schema functionality in Json.NET has been moved to its own package. Please visit http://www.newtonsoft.com/jsonschema for more details.

I’ve been hard at work on Json.NET over the holidays. I spent most of December setting up environments, writing PowerShell scripts and deploying applications so it was nice to do a little .NET development for a change.

JSON Schema

The big new feature in Beta 2 is JSON schema validation. JSON Schema is a specification for defining the structure of JSON. Think of it as XML Schema for JSON except it is actually readable.

{

  "description": "A person",

  "type": "object",

  "properties":

  {

    "name": {"type":"string"},

    "hobbies": {

      "type": "array",

      "items": {"type":"string"}

    }

  }

}

I have put together a demo page of the new JSON Schema features: JSON Schema Validation Demo

There are a couple of different ways of validating using JSON Schema in Json.NET. The simplest is using an extension method for the LINQ to JSON objects. IsValid in this example just returns a flag but there are other overloads that let you also capture validation messages.

JsonSchema schema = JsonSchema.Parse(schemaJson);

JObject person = JObject.Parse(@"{

  ""name"": ""James"",

  ""hobbies"": ["".NET"", ""Blogging"", ""Reading"", ""Xbox"", ""LOLCATS""]

}");

bool valid = person.IsValid(schema);

// true

As well as validation, for fun, and to help with testing, I wrote a class called JsonSchemaGenerator. This class looks at a .NET type and using reflection it generates a schema for it. The schemas on the demo page are all generated at runtime.

JSON line and position Information

JsonTextReader now reports line and position information. Line and position information was needed for the schema validator to help point out to a user in an error message which part the JSON was invalid. The nice flow on effect of this feature is now when you give Json.NET bad JSON, the resulting exception will tell you the location of the invalid character.

JObject person = JObject.Parse(@"{

  ""name"": ""James"",

  ]!#$THIS IS: BAD JSON![{}}}}]

}");

// Invalid property identifier character: ]. Line 3, position 3.

Changes

Here is a complete list of what has changed since Json.NET 3.5 Beta 1.

• New feature - Added JSON Schema implementation
• New feature - Added IJsonLineInfo. Implemented by JsonTextReader, JsonTokenReader, JsonValidatingReader, JToken
• New feature - Added line details to JsonTextReader exception messages and JsonValidatingReader errors
• New feature - Added JsonContainerAttribute with Id, Title and Description members. JsonObject and JsonArray inherit from this attribute
• New feature - Added JsonArrayAttribute. Has flag to control whether array can contain null items
• New feature - Added IsRequired to JsonProperty
• New feature - Added Load(JsonReader) to JProperty, JConstructor
• New feature - Added the ability for JsonTokenWriter to write individual values as well as arrays and objects
• New feature - Added CreateReader to JToken
• New feature - Added FromObject to JToken
• New feature - Added ReadFrom to JToken
• Change - Renamed JavaScriptConvert to JsonConvert
• Change - Value<T> on JObject supports getting nullable values
• Change - Type values now serialize and deserialize to the type name string
• Change - GetSerializableMembers has been removed and replaced with GetMemberMappings on JsonSerializer
• Fix - JsonConvert now always write a floating point number with a decimal place
• Fix - JsonSerializer now correctly skips missing members
• Fix - JsonWriter now allows objects and arrays to be written in a constructor
• Fix - QuoteChar not getting set when parsing property name
• Fix - JProperty now correctly loads content from a collection

Links

Json.NET CodePlex Project

Json.NET 3.5 Beta 2 Download – Json.NET source code, documentation and binaries

I am considering renaming the JavaScriptConvert class to JsonConvert.

When Json.NET was first released it contained a number of data classes that were prefixed with JavaScript... JavaScriptObject, JavaScriptArray, JavaScriptConstructor. JavaScriptConvert was used to convert the data objects from text and back again. In 2.0 these classes were removed and replaced by the LINQ to JSON API: JObject, JArray, etc, but JavaScriptConvert remained.

The current state of the nation is that just about every Json.NET class is prefixed with Json (JsonReader, JsonWriter, JsonSerializer) except JavaScriptConvert, a situation which I think is somewhat confusing and gets in the way of learning and discovering the Json.NET API through intellisense.

The downside of renaming JavaScriptConvert is that existing code that uses it will need to be updated when moving to a new version of Json.NET. A pain but nothing that Find and Replace couldn’t fix in 5 minutes. A namespace alias would also fix existing code.

using JavaScriptConvert = Newtonsoft.Json.JsonConvert;

Thoughts? Objections?