Penny Arcade TV – Ping Pong 1/Ping Pong 2

The guys at Penny Arcade have branched out from podcasts to a web TV show with predictably hilarious results.

There are a lot of good episodes (PA’s crazy hire process!) but I really love this duo covering Penny Arcade’s obsession with Ping Pong and a rematch game against Bungie, the makers of my beloved Halo.

Smack talk, slow-mo ping pong action shots and an agonist sporting a sinister English accent – what’s not to like?

NPR: Planet Money Podcast – We Bought A Toxic Asset!

I wouldn’t call Planet Money a pure economics podcast – it definitely has a bit of an opinion about some economic social issues so viewer discretion is advised – but that doesn’t mean it isn’t interesting.

This episode the people at Planet Money buy a toxic asset and explain in simple language what a toxic asset is and why and how you would buy one. It is a lot more entertaining than I ever would have thought possible.

Oh and bonus points for playing both The Gambler and Danger Zone in a single podcast!

Binary data and text file formats (JSON, XML) don’t tend get along. To be included in JSON or XML binary data has to be encode into a text friendly format, generally base64, which creates overhead both in the time spent encoding/decoding binary, and the extra size of the text encoded data in the message.

{

  "binary": "SGVsbG8gV29ybGQ="

}

The Test

In our test we’ll compare serializing a message with binary data using common .NET serialization methods and compare the result. First our message…

public class Image

{

  public string FileName { get; set; }

  public string Author { get; set; }

  public string Caption { get; set; }

  public byte[] Data { get; set; }

}

And some test data…

FileName: bunny_pancake.jpg Author: Hironori Akutagawa Caption: I have no idea what you are talking about so here’s a bunny with a pancake on its head.

The Results

Json.NET and the DataContractSerializer are about equal to each other in the size of the resulting message – each is encoding the image byte array as base64 encoded text. The small edge to Json.NET comes from the smaller size of the JSON metadata compared to the DataContractSerializer’s XML.

Json.NET BSON and the BinaryFormatter are binary formats so they can include the image data directly in the message. They’re approximately 25% smaller than JSON/XML by saving the overhead of base64. Again the Json.NET BSON format is more efficient than the BinaryFormatter making it slightly smaller.

Finally as you can see the WCF DataContractJsonSerializer is by far the least efficient. When I looked into why it performed so badly it turns out that the WCF JsonSerializer serializes binary data as a JSON array of integers, one integer representing each byte. It goes without saying that you should never use the WCF JsonSerializer with binary data.

Read more about optimizing Json.NET performance here: Json.NET performance

---

A lot of work around performance went into the latest release of Json.NET, with big improvements in both serializing and deserializing over the previous version.

I can happily say that Json.NET is now faster than both the JavaScriptSerializer and the WCF DataContractJsonSerializer over all scenarios.

Other .NET Serializers

The latest performance improvements now puts Json.NET on the same level as the XML based DataContractSerializer which is pretty remarkable.

Also worth noting is Json.NET over binary (BSON) is considerably faster than the .NET BinaryFormatter.

Serialization Result Data Size

Finally one of the benefits of JSON is its smaller size when compared to equivalent XML. The output of Json.NET is less than half the size of the XML that DataContractSerializer produces (strings have been encoded to UTF8 bytes).

In this test the BSON result is marginally smaller than the JSON result. BSON really shines when serializing byte data (i.e. images, movies, files, etc) where its output will come in at least 30% smaller over JSON because there is no base64 encoding of byte data.

You can find the latest version of Json.NET, including source code with the performance tests, here.

A new Json.NET release! As well as the new features below this release also adds documentation for common questions and I’ve spent some time having fun working hard on performance to make this the fastest version ever. Expect a post with sweet Excel graphs in the near future.

Binary JSON (BSON) support

Json.NET now supports reading and writing binary JSON (BSON).

BSON is a standard for binary JSON. BSON supports all JSON types (objects, arrays, strings, integers, etc). Anything that can be represented in JSON can also be represented in BSON.

BSON supports some additional types, most notably binary data, making it a superset of JSON. Json.NET handles reading all BSON and will translate any unsupported types to JSON.

The two major benefits of BSON over JSON are size and speed. BSON stores binary data directly, avoiding the time overhead and the additional size of base64 encoded text that regular JSON has with binary data. Reading and writing BSON is also faster than JSON.

Product p = new Product();

p.ExpiryDate = DateTime.Parse("2009-04-05T14:45:00Z");

p.Name = "Carlos' Spicy Wieners";

p.Price = 9.95m;

p.Sizes = new[] { "Small", "Medium", "Large" };

MemoryStream ms = new MemoryStream();

JsonSerializer serializer = new JsonSerializer();

// serialize product to BSON

BsonWriter writer = new BsonWriter(ms);

serializer.Serialize(writer, p);

Console.WriteLine(BitConverter.ToString(ms.ToArray()));

// 7C-00-00-00-02-4E-61-6D-65-00-16-00-00-00-43-61-72-6C-

// 6F-73-27-20-53-70-69-63-79-20-57-69-65-6E-65-72-73-00-

// 09-45-78-70-69-72-79-44-61-74-65-00-E0-51-BD-76-20-01-

// 00-00-01-50-72-69-63-65-00-66-66-66-66-66-E6-23-40-04-

// 53-69-7A-65-73-00-2D-00-00-00-02-30-00-06-00-00-00-53-

// 6D-61-6C-6C-00-02-31-00-07-00-00-00-4D-65-64-69-75-6D-

// 00-02-32-00-06-00-00-00-4C-61-72-67-65-00-00-00

ms.Seek(0, SeekOrigin.Begin);

// deserialize product from BSON

BsonReader reader = new BsonReader(ms);

Product deserializedProduct = serializer.Deserialize<Product>(reader);

Console.WriteLine(deserializedProduct.Name);

// Carlos' Spicy Wieners

Decoupling

Json.NET no longer has dependencies on the Entity Framework, LINQ to SQL or data annotations assemblies. It should now run under the .NET 4 Client Profile – a subset of the .NET framework designed for client applications. Read more about the .NET 4 Client Profile here.

DataSetConverter

Json.NET now automatically serializes and deserializes DataSets and DataTables.

A DataSet is serialized as an object with table names as its properties. A DataTable is serialized as an array of table rows, each row an object of name/values using the column name and column value for that row.

DataSet ds = new DataSet();

ds.Tables.Add(CreateDataTable("FirstTable", 2));

ds.Tables.Add(CreateDataTable("SecondTable", 1));

string json = JsonConvert.SerializeObject(ds, Formatting.Indented, new IsoDateTimeConverter());

// {

//   "FirstTable": [

//     {

//       "StringCol": "Item Name",

//       "Int32Col": 1,

//       "BooleanCol": true,

//       "TimeSpanCol": "10.22:10:15.1000000",

//       "DateTimeCol": "2000-12-29T00:00:00Z",

//       "DecimalCol": 64.0021

//     },

//     {

//       "StringCol": "Item Name",

//       "Int32Col": 2,

//       "BooleanCol": true,

//       "TimeSpanCol": "10.22:10:15.1000000",

//       "DateTimeCol": "2000-12-29T00:00:00Z",

//       "DecimalCol": 64.0021

//     }

//   ],

//   "SecondTable": [

//     {

//       "StringCol": "Item Name",

//       "Int32Col": 1,

//       "BooleanCol": true,

//       "TimeSpanCol": "10.22:10:15.1000000",

//       "DateTimeCol": "2000-12-29T00:00:00Z",

//       "DecimalCol": 64.0021

//     }

//   ]

// }

DataSet deserializedDs = JsonConvert.DeserializeObject<DataSet>(json, new IsoDateTimeConverter());

LINQ to JSON SelectToken

SelectToken provides a method to query LINQ to JSON using a single string path to a desired JToken. SelectToken is handy for dynamic queries easy because the entire query is defined in a string.

string name = (string)o.SelectToken("Manufacturers[0].Name");

SelectToken is a method on JToken and takes a string path to a child token. SelectToken returns the child token or a null reference if a token couldn't be found at the path's location.

The path is made up of property names and array indexes separated by periods. Array indexes can use either square or round brackets. Both of the following are valid paths and are equivalent to each other: Manufacturers[0].Name and Manufacturers(0).Name.

JObject o = JObject.Parse(@"{

  ""Stores"": [

    ""Lambton Quay"",

    ""Willis Street""

  ],

  ""Manufacturers"": [

    {

      ""Name"": ""Acme Co"",

      ""Products"": [

        {

          ""Name"": ""Anvil"",

          ""Price"": 50

        }

      ]

    },

    {

      ""Name"": ""Contoso"",

      ""Products"": [

        {

          ""Name"": ""Elbow Grease"",

          ""Price"": 99.95

        },

        {

          ""Name"": ""Headlight Fluid"",

          ""Price"": 4

        }

      ]

    }

  ]

}");

string name = (string)o.SelectToken("Manufacturers[0].Name");

// Acme Co

decimal productPrice = (decimal)o.SelectToken("Manufacturers[0].Products[0].Price");

// 50

string productName = (string)o.SelectToken("Manufacturers[1].Products[0].Name");

// Elbow Grease

Changes

Here is a complete list of what has changed since Json.NET 3.5 Release 5.

• New feature - Added reading and writing binary JSON (BSON) support via BsonReader, BsonWriter
• New feature - Added support for reading and writing byte arrays to JsonReader, JsonWriter and LINQ to JSON classes
• New feature - Added ReadAsBytes to JsonReader
• New feature - Added DataSetConverter, DataTableConverter
• New feature - Added JPath and SelectToken to JToken
• New feature - Added ObjectCreationHandling to JsonPropertyAttribute
• New feature - Added IValueProvider, ReflectionValueProvider, DynamicValueProvider
• New feature - Added serialization event support to Silverlight version
• New feature - Added DBNull support to JValue
• Change - Removed dependency on Entity Framework, LINQ to SQL and data annotations assemblies
• Change - Upgraded Silverlight project to Silverlight 3
• Change - Changed IsRequired to Required enum on JsonPropertyAttribute with options Default, AllowNull and Always
• Change - Change converter to be resolved from new Converter property on JsonProperty
• Change - Improved error message when JSON array encountered for object and vice-versa when deserializing
• Change - Improved error message for converting types when deserializing
• Change - Improved error message from getting and setting properties when serializing and deserializing
• Change - Class converter now resolved from contract
• Change - Built in serializers now resolved when contract is created
• Change - JsonRaw removed and replaced with JRaw
• Fix - Schema id not being written for collections
• Fix - TimeSpan serialization incorrect on Compact Framework
• Fix - Unicode line break string encoding when writing JSON
• Fix - Empty string being incorrectly converted to null when deserializing
• Fix - Unclosed object when including type name with array
• Fix - Serializing objects with generic methods failing on Compact Framework
• Fix - Remove DateTimeOffset stand-in from Compact Framework build
• Fix - Modified .NET 2.0 build to run on environments without .NET 2.0 SP1
• Fix - Changed deserialization to always use a new value created from a JsonConverter
• Fix - XmlNodeConverter not converting comments in JSON
• Fix - ToString culture inconsistency when serializing dictionary key
• Fix - JTokenWriter not writing comments
• Fix - Properties on existing derived objects not being populated during deserialization

Links

Json.NET CodePlex Project

Json.NET 3.5 Release 6 Download – Json.NET source code, documentation and binaries