Clever Geek Handbook

Battle of C # JSON serializers for .NET Core 3

Hello. In anticipation of the start of the "C # Developer" course , we have prepared an interesting translation for you, and we also offer you to watch the lesson record for free: "Design Pattern State (State)"









The recently released .NET Core 3 brought with it a number of innovations. In addition to C # 8 and support for WinForms and WPF, the latest release has added a new JSON (de) serializer - System.Text.Json , and as its name suggests, all of its classes are in this namespace.



This is a major innovation. JSON serialization is an important factor in web applications.Most of today's REST API relies on it. When your javascript client sends JSON in the body of a POST request, the server uses JSON deserialization to convert it to a C # object. And when the server returns an object in response, it serializes this object to JSON so that your javascript client can understand it. These are large operations that are performed on every request with objects. Their performance can significantly affect the performance of applications, which I am going to demonstrate now.



If you have experience with .NET, then you must have heard of the excellent Json.NET serializer , also known as Newtonsoft.Json . So why do we need a new serializer when we already have the lovely Newtonsoft.Json ? While Newtonsoft.Json is undoubtedly great, there are some good reasons to replace it:



  • Microsoft was keen to use new types, such as Span<T>, to improve performance. Modifying a huge library like Newtonsoft without breaking functionality is very difficult.
  • , HTTP, UTF-8. String .NET — UTF-16. Newtonsoft UTF-8 UTF-16, . UTF-8.
  • Newtonsoft , .NET Framework ( BCL FCL), . ASP.NET Core Newtonsoft, .


In this article, we're going to run a few benchmarks to see how much better the new serializer is in terms of performance. In addition, we'll also compare Newtonsoft.Json and System.Text.Json with other well-known serializers and see how they handle in relation to each other.



Battle of serializers



Here's our ruler:



  • Newtonsoft.Json (also known as Json.NET ) is the serializer currently the industry standard. It was integrated into ASP.NET, although it was third-party. NuGet package # 1 of all time. A multi-award winning library (I probably don't know for sure).
  • System.Text.Json — Microsoft. Newtonsoft.Json. ASP.NET Core 3. .NET, NuGet ( ).
  • DataContractJsonSerializer — , Microsoft, ASP.NET , Newtonsoft.Json.
  • Jil — JSON Sigil



  • ServiceStack — .NET JSON, JSV CSV. .NET ( ).



  • Utf8Jso n is another self-proclaimed fastest C # to JSON serializer. Works with zero memory allocation and reads / writes directly to UTF8 binary for better performance.





Note that there are non-JSON serializers out there that are faster. In particular, protobuf-net is a binary serializer that should be faster than any of the compared serializers in this article (which hasn't been tested by benchmarks though).



Benchmark structure



Serializers are not easy to compare. We will need to compare serialization and deserialization. We will need to compare different types of classes (small and large), lists and dictionaries. And we will need to compare different serialization targets: strings, streams and character arrays (UTF-8 arrays). This is a fairly large test matrix, but I'll try to keep it as organized and concise as possible.



We will be testing 4 different functionality:



  • Serialization to string
  • Serialization to a stream
  • Deserialize from string
  • Requests per second on ASP.NET Core 3 app


For each, we will be testing different types of objects (which you can see on GitHub ):



  • Small class with only 3 primitive type properties.
  • Large class with about 25 properties, DateTime and a couple of enums
  • List of 1000 elements (small class)
  • Dictionary of 1000 elements (small class)


These are not all necessary benchmarks, but, in my opinion, they are enough to get a general idea.

For all the benchmarks I used BenchmarkDotNet on the following system: BenchmarkDotNet=v0.11.5, OS=Windows 10.0.17134.1069 (1803/April2018Update/Redstone4) Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores. .NET Core SDK=3.0.100. Host : .NET Core 3.0.0 (CoreCLR 4.700.19.46205, CoreFX 4.700.19.46214), 64bit RyuJIT. You can find the benchmark project itself on GitHub .


All tests will only run on .NET Core 3 projects.



Benchmark 1: Serialization to String



The first thing we'll check is serializing our sample of objects to a string.



The benchmark code itself is pretty simple (see on GitHub ): 



public class SerializeToString<T> where  T : new()
{
 
    private T _instance;
    private DataContractJsonSerializer _dataContractJsonSerializer;
 
    [GlobalSetup]
    public void Setup()
    {
        _instance = new T();
        _dataContractJsonSerializer = new DataContractJsonSerializer(typeof(T));
    }
 
    [Benchmark]
    public string RunSystemTextJson()
    {
        return JsonSerializer.Serialize(_instance);
    }
 
    [Benchmark]
    public string RunNewtonsoft()
    {
        return JsonConvert.SerializeObject(_instance);
    }
 
    [Benchmark]
    public string RunDataContractJsonSerializer()
    {
        using (MemoryStream stream1 = new MemoryStream())
        {
            _dataContractJsonSerializer.WriteObject(stream1, _instance);
            stream1.Position = 0;
            using var sr = new StreamReader(stream1);
            return sr.ReadToEnd();
        }
    }
 
    [Benchmark]
    public string RunJil()
    {
        return Jil.JSON.Serialize(_instance);
    }
 
    [Benchmark]
    public string RunUtf8Json()
    {
        return Utf8Json.JsonSerializer.ToJsonString(_instance);
    }
 
    [Benchmark]
    public string RunServiceStack()
    {
        return SST.JsonSerializer.SerializeToString(_instance);
    }   
}


The above test class is generic, so we can test all our objects with the same code, for example: 



BenchmarkRunner.Run<SerializeToString<Models.BigClass>>();


After running all test classes with all serializers, we got the following results:





More accurate indicators can be found here


  • Utf8Json is the fastest to date, more than 4x faster than Newtonsoft.Json and System.Text.Json . This is a striking difference.
  • Jil is also very fast, about 2.5x faster than Newtonsoft.Json and System.Text.Json.
  • In most cases, the new System.Text.Json serializer performs better than Newtonsoft.Json by about 10%, except for Dictionary, where it ended up being 10% slower.
  • The older DataContractJsonSerializer is much worse than the others.
  • ServiceStack sits right in the middle, showing that it is no longer the fastest text serializer. At least for JSON.


Benchmark 2: Serialization to Stream



The second set of tests is pretty much the same, except we're serializing to a stream. The benchmark code is here . Results:







More accurate figures can be found here . Thanks to Adam Sitnik and Ahson Khan for helping me get System.Text.Json to work.


The results are very similar to the previous test. Utf8Json and Jil are 4 times faster than others. Jil is very fast, second only to Utf8Json . The DataContractJsonSerializer is still the slowest in most cases. Newtonsoft works much like System.Text.Json in most cases , except for dictionaries where Newtonsoft has a noticeable advantage .



Benchmark 3: Deserializing From String



The next set of tests deals with deserialization from a string. The test code can be found here .







More accurate figures can be found here .


I'm having some difficulty running the DataContractJsonSerializer for this benchmark, so it is not included in the results. Otherwise, we see that Jil is the fastest in deserialization , Utf8Json is in second place. They are 2-3 times faster than System.Text.Json . And System.Text.Json is about 30% faster than Json.NET .



So far, it turns out that the popular Newtonsoft.Json and the new System.Text.Json have significantly worse performance than their competitors. This was a rather unexpected result for me due to the popularity of Newtonsoft.Jsonand all the hype around the new top performer Microsoft System.Text.Json . Let's check it out in an ASP.NET application.



Benchmark 4: The number of requests per second on the .NET server



As mentioned earlier, JSON serialization is very important because it is constantly present in REST APIs. HTTP requests to a server using the content type application/jsonwill need to serialize or deserialize the JSON object. When the server accepts the payload in a POST request, the server deserializes from JSON. When the server returns an object in its response, it serializes JSON. Modern client-server communication relies heavily on JSON serialization. Therefore, to test the "real" scenario, it makes sense to create a test server and measure its performance.



I was inspired by the Microsoft performance testin which they created an MVC server application and checked the requests per second. Microsoft benchmarks test System.Text.Json and Newtonsoft.Json . In this article we will do the same, except that we are going to compare them to Utf8Json , which has proven to be one of the fastest serializers in previous tests.

Unfortunately I was unable to integrate ASP.NET Core 3 with Jil, so the benchmark doesn't include it. I'm pretty sure it can be done with more effort, but alas.


The creation of this test proved to be more difficult than before. I first created an ASP.NET Core 3.0 MVC app, just like in the Microsoft benchmark. I added a controller for performance tests, similar to the one in the Microsoft test : 



[Route("mvc")]
public class JsonSerializeController : Controller
{
 
    private static Benchmarks.Serializers.Models.ThousandSmallClassList _thousandSmallClassList
        = new Benchmarks.Serializers.Models.ThousandSmallClassList();
    
    [HttpPost("DeserializeThousandSmallClassList")]
    [Consumes("application/json")]
    public ActionResult DeserializeThousandSmallClassList([FromBody]Benchmarks.Serializers.Models.ThousandSmallClassList obj) => Ok();
 
    [HttpGet("SerializeThousandSmallClassList")]
    [Produces("application/json")]
    public object SerializeThousandSmallClassList() => _thousandSmallClassList;
}


When the client calls the endpoint DeserializeThousandSmallClassList, the server will accept the JSON text and deserialize the content. This is how we test deserialization. When the client calls SerializeThousandSmallClassList, the server will return a list of 1000 SmallClassitems and thereby serialize the content to JSON.



Then we need to cancel the logging for each request so that it doesn't affect the result: 



public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureLogging(logging =>
        {
            logging.ClearProviders();
            //logging.AddConsole();
        })
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });


We now need a way to switch between System.Text.Json , Newtonsoft and Utf8Json . It's easy with the first two. For System.Text.Json you don't need to do anything at all. To switch to Newtonsoft.Json just add one line to ConfigureServices : 



public void ConfigureServices(IServiceCollection services)
{
    services.AddControllersWithViews()
    //   Newtonsoft.   -     System.Text.Json
    .AddNewtonsoftJson()
    ;


For Utf8Json, we need to add custom media formatters InputFormatterand OutputFormatter. It wasn't that easy, but in the end I found a good solution on the internet , and after digging around in the settings, it worked. There is also a NuGet package with formatters, but it doesn't work with ASP.NET Core 3. 



internal sealed class Utf8JsonInputFormatter : IInputFormatter
{
    private readonly IJsonFormatterResolver _resolver;
 
    public Utf8JsonInputFormatter1() : this(null) { }
    public Utf8JsonInputFormatter1(IJsonFormatterResolver resolver)
    {
        _resolver = resolver ?? JsonSerializer.DefaultResolver;
    }
 
    public bool CanRead(InputFormatterContext context) => context.HttpContext.Request.ContentType.StartsWith("application/json");
 
    public async Task<InputFormatterResult> ReadAsync(InputFormatterContext context)
    {
        var request = context.HttpContext.Request;
 
        if (request.Body.CanSeek && request.Body.Length == 0)
            return await InputFormatterResult.NoValueAsync();
 
        var result = await JsonSerializer.NonGeneric.DeserializeAsync(context.ModelType, request.Body, _resolver);
        return await InputFormatterResult.SuccessAsync(result);
    }
}
 
internal sealed class Utf8JsonOutputFormatter : IOutputFormatter
{
    private readonly IJsonFormatterResolver _resolver;
 
    public Utf8JsonOutputFormatter1() : this(null) { }
    public Utf8JsonOutputFormatter1(IJsonFormatterResolver resolver)
    {
        _resolver = resolver ?? JsonSerializer.DefaultResolver;
    }
 
    public bool CanWriteResult(OutputFormatterCanWriteContext context) => true;
 
    
    public async Task WriteAsync(OutputFormatterWriteContext context)
    {
        if (!context.ContentTypeIsServerDefined)
            context.HttpContext.Response.ContentType = "application/json";
 
        if (context.ObjectType == typeof(object))
        {
            await JsonSerializer.NonGeneric.SerializeAsync(context.HttpContext.Response.Body, context.Object, _resolver);
        }
        else
        {
            await JsonSerializer.NonGeneric.SerializeAsync(context.ObjectType, context.HttpContext.Response.Body, context.Object, _resolver);
        }
    }
}


Now for ASP.NET to use these formatter: 



public void ConfigureServices(IServiceCollection services)
{
    services.AddControllersWithViews()
 
    //   Newtonsoft
    //.AddNewtonsoftJson()
 
   //   Utf8Json
    .AddMvcOptions(option =>
    {
        option.OutputFormatters.Clear();
        option.OutputFormatters.Add(new Utf8JsonOutputFormatter1(StandardResolver.Default));
        option.InputFormatters.Clear();
        option.InputFormatters.Add(new Utf8JsonInputFormatter1());
    });
}


So this is the server. Now about the client.



C # client for measuring requests per second



I've also created a C # client application, although JavaScript clients will prevail in most real-world scenarios. It doesn't matter for our purposes. Here is the code: 



public class RequestPerSecondClient
{
    private const string HttpsLocalhost = "https://localhost:5001/";
 
    public async Task Run(bool serialize, bool isUtf8Json)
    {
        await Task.Delay(TimeSpan.FromSeconds(5));
        
        var client = new HttpClient();
        var json = JsonConvert.SerializeObject(new Models.ThousandSmallClassList());
 
       //  ,    
        for (int i = 0; i < 100; i++)
        {
            await DoRequest(json, client, serialize);
        }
 
        int count = 0;
 
        Stopwatch sw = new Stopwatch();
        sw.Start();
 
        while (sw.Elapsed < TimeSpan.FromSeconds(1))
        {
            count++;
            await DoRequest(json, client, serialize);
        }
        
        Console.WriteLine("Requests in one second: " + count);
    }
 
    
    private async Task DoRequest(string json, HttpClient client, bool serialize)
    {
        if (serialize)
            await DoSerializeRequest(client);
        else
            await DoDeserializeRequest(json, client);
    }
    
    private async Task DoDeserializeRequest(string json, HttpClient client)
    {
        var uri = new Uri(HttpsLocalhost + "mvc/DeserializeThousandSmallClassList");
        var content = new StringContent(json, Encoding.UTF8, "application/json");
        var result = await client.PostAsync(uri, content);
        result.Dispose();
    }
 
    private async Task DoSerializeRequest(HttpClient client)
    {
        var uri = HttpsLocalhost + "mvc/SerializeThousandSmallClassList";
        var result = await client.GetAsync(uri);
        result.Dispose();
    }
}


This client will continuously send requests for 1 second, counting them.



results



So, without further ado, here are the results:







More accurate indicators can be found here


Utf8Json outperformed other serializers by a huge margin. This was not a big surprise after previous tests.



In terms of serialization, Utf8Json is 2x faster than System.Text.Json and 4x faster than Newtonsoft . For deserialization, Utf8Json is 3.5 times faster than System.Text.Json and 6 times faster than Newtonsoft .



The only surprise for me here is how poorly Newtonsoft.Json works... This is probably due to the UTF-16 and UTF-8 issue. The HTTP protocol works with UTF-8 text. Newtonsoft converts this text to .NET string types, which are UTF-16. This overhead is not present in either Utf8Json or System.Text.Json , which work directly with UTF-8.



It is important to note that these benchmarks should not be 100% trusted as they may not fully reflect the actual scenario. And that's why:



  • I ran everything on my local machine - both client and server. In a real-world scenario, the server and client are on different machines.
  • . , . . - . , , . , , GC. Utf8Json, .
  • Microsoft ( 100 000). , , , , .
  • . , - - .


All things considered, these results are pretty incredible. It seems that the response time can be significantly improved by choosing the right JSON serializer. Switching from Newtonsoft to System.Text.Json will increase the number of requests by 2-7 times, and switching from Newtonsoft to Utf8Json will improve by 6-14 times. This is not entirely fair, because a real server will do a lot more than just accept arguments and return objects. It will probably do other things as well, like working with databases and therefore executing some business logic, so serialization time may be less important. However, these numbers are incredible.



conclusions



Let's summarize:



  • System.Text.Json , Newtonsoft.Json ( ). Microsoft .
  • , Newtonsoft.Json System.Text.Json. , Utf8Json Jil 2-4 , System.Text.Json.
  • , Utf8Json ASP.NET . , , , ASP.NET.


Does this mean that we should all switch to Utf8Json or Jil? The answer to that is ... perhaps. Remember, Newtonsoft.Json has stood the test of time and become the most popular serializer for a reason. It supports many features, has been tested with all types of edge cases, and has tons of documented solutions and workarounds. Both System.Text.Json and Newtonsoft.Json are very well supported. Microsoft will continue to invest resources and effort in System.Text.Json so you can count on great support. Whereas Jil and Utf8Jsonreceived very few commits in the last year. In fact, it looks like they haven't had much maintenance in the last 6 months.



One option is to combine multiple serializers in your application. Upgrade to faster serializers for ASP.NET integration for superior performance, but continue to use Newtonsoft.Json in your business logic to get the most out of its feature set.



I hope you enjoyed this article. Good luck)



Other benchmarks



Several Other Benchmarks Comparing Different Serializers



When Microsoft announced System.Text.Json, they showed their own benchmark comparing System.Text.Json and Newtonsoft.Json . In addition to serialization and deserialization, this benchmark tests the Document class for random access, Reader and Writer. They also demonstrated their Query Per Second test , which inspired me to create my own.



The .NET Core GitHub repository includes a set of benchmarks similar to those described in this article. I looked very closely at their tests to make sure I wasn't making mistakes myself. You can find them inMicro-benchmarks solution .



Jil has its own benchmarks that compare Jil , Newtonsoft , Protobuf, and ServiceStack .



Utf8Json has posted a set of benchmarks available on GitHub . They also test binary serializers.



Alois Kraus has done excellent in- depth testing of the most popular .NET serializers, including JSON serializers, binary serializers, and XML serializers. Its benchmark includes benchmarks for .NET Core 3 and .NET Framework 4.8.





Learn more about the course.





Read more:





More articles:


All Articles