Dealing with JSON interop between Java and C# REST APIs

I'm currently dealing with 2 systems that expose interop via their own RESTful JSON APIs. One is in C# with JSON.NET and one is Java Spring Boot Starter (Jackson JSON). I have full control over both systems.

Both systems need to transfer JSON data with reference handling. Whilst both JSON serialization frameworks support it, C# JSON.NET uses "$id" and "$ref" syntax to signify references whilst Java's Jackson uses something plainer with only "id".

I am much less familiar with Java than I am C# so I would more readily accept and understand any solution on getting JSON ref handling working both ways on the C# side. How can I get these two systems to interop with JSON refs?

C# JSON.NET reference handling documentation.

Example JSON coming from Java Jackson

Note that it is possible to mark up what class property Jackson uses as the reference. In this case I am using the Id variable as it will always locally unique to the type.

{
  "Resources": [
      {
          "Id": 0,
          "Name": "Resource 0"
      },
      {
          "Id": 1,
          "Name": "Resource 1"
      }
  ],
  "Tasks": [
      {
          "Id": 0,
          "Name": "Task 0",
          "Resource": 0
      },
      {
          "Id": 1,
          "Name": "Task 1",
          "Resource": 1
      },
      {
          "Id": 2,
          "Name": "Task 2",
          "Resource": 0
      },
      {
          "Id": 3,
          "Name": "Task 3",
          "Resource": 1
      },
      {
          "Id": 4,
          "Name": "Task 4",
          "Resource": 0
      }
  ]
}

Before reading, check out my other solution approach here, it may be simpler.
Keeping this post as well as I believe it is informative and may be consider a better approach by some.

Why is this complicated?

The issue is not the reference property names, for that you can use IReferenceResolver to override. Instead, the issue is two-fold:

1. The reference is from a property of an object in the Tasks list to an object in the Resources list. This is not the intention of the PreserveObjectReference feature. It was intended to not repeat objects in the same list as well as help prevent cyclic references.

2. The value in the Resource property of a Task is a number instead of a Resource object (which would not have worked anyways, due to item 1 above), e.g.

{
  "Id": 0,
  "Name": "Task 0",
  "Resource": {
    "$ref": 0
  }
}

3. IDs and references have to be strings, not numbers

Solution

Manually build the object and manually match the references:

1. Our DTOs:

public class Dto
{
    public Resource[] Resources { get; set; }
    public Task[] Tasks { get; set; }
}

public class Resource
{
    public long Id { get; set; }
    public string Name { get; set; }
}

public class Task
{
    public long Id { get; set; }
    public string Name { get; set; }
    public Resource Resource { get; set; }
}

2. Contract resolution:

/// <summary>
/// This is to resolve the Resource resolver for the Task
/// </summary>
internal class TaskResourceContractResolver : DefaultContractResolver
{
    private readonly IDictionary<long, Resource> _resources;

    public TaskResourceContractResolver(IDictionary<long, Resource> resources) => this._resources = resources;

    #region Overrides of DefaultContractResolver
    protected override JsonProperty CreateProperty(MemberInfo member, MemberSerialization memberSerialization)
    {
        var property = base.CreateProperty(member, memberSerialization);
        if (property.DeclaringType != typeof(Task) || property.PropertyName != nameof(Task.Resource))
            return property;

        property.Converter = new TaskResourceConverter(this._resources);
        property.IsReference = true;
        property.ValueProvider = new CurrentValueGetterValueProvider();
        return property;
    }
    #endregion Overrides of DefaultContractResolver

    /// <summary>
    /// This is to resolve the Resource for the Task
    /// </summary>
    private class TaskResourceConverter : JsonConverter<Task>
    {
        private readonly IDictionary<long, Resource> _resources;

        public TaskResourceConverter(IDictionary<long, Resource> resources) => this._resources = resources;

        #region Overrides of JsonConverter
        public override void WriteJson(JsonWriter writer, Task value, JsonSerializer serializer) => throw new NotImplementedException();
        public override Task ReadJson(JsonReader reader, Type objectType, Task existingValue, bool hasExistingValue, JsonSerializer serializer)
        {
            if (reader.Value is Resource resource) existingValue.Resource = resource;
            else if (reader.Value is long resourceRef)
            {
                if (!this._resources.TryGetValue(resourceRef, out resource)) throw new Exception($"Invalid resource reference '{resourceRef}'");
                existingValue.Resource = resource;
            }
            else throw new Exception($"Invalid resource reference '{reader.Value}'");

            return existingValue;
        }
        #endregion Overrides of JsonConverter
    }

    /// <summary>
    /// This is so we get the value of Task object to be set
    /// </summary>
    private class CurrentValueGetterValueProvider : IValueProvider
    {
        #region Implementation of IValueProvider
        public void SetValue(object target, object value) => throw new NotImplementedException();
        public object GetValue(object target) => target;
        #endregion Implementation of IValueProvider
    }
}

3. Implementation:

var input = Encoding.UTF8.GetString(Properties.Resources.input); // the posted Java-outputted JSON
var parsed = JObject.Parse(input);

var resources = parsed[nameof(Dto.Resources)]?.Children()
    .Select(token => token.ToObject<Resource>())
    .ToDictionary(r => r!.Id);

var serializer = new JsonSerializer() { ContractResolver = new TaskResourceContractResolver(resources) };

var dto = new Dto
{
    Resources = resources?.Values.ToArray(),
    Tasks = parsed[nameof(Dto.Tasks)]?.Children()
        .Select(token => token.ToObject<Task>(serializer))
        .ToArray()
};

Console.WriteLine($@"Distinct resources: {dto.Resources?.Distinct().Count()}");
Console.WriteLine($@"Distinct tasks: {dto.Tasks?.Distinct().Count()}");
Console.WriteLine($@"Distinct task resources: {dto.Tasks?.Select(t => t.Resource).Distinct().Count()}");

4. Output:

Distinct resources: 2
Distinct tasks: 5
Distinct task resources: 2

.NET Fiddle

Possibly a simpler approach to what I published here

Reasons are the same, but solution is different:

Use an interim serialization-only class

(this one has both serialization and deserialization solution)

1. DTOs are the same:

public class Dto
{
    public Resource[] Resources { get; set; }
    public Task[] Tasks { get; set; }
}

public class Resource
{
    public long Id { get; set; }
    public string Name { get; set; }
}

public class Task
{
    public long Id { get; set; }
    public string Name { get; set; }
    public Resource Resource { get; set; }
}

2. Interim serialization-only class:

/// <summary>
/// Helper class for Dto serialization
/// </summary>
internal class DtoSerializationHelper
{
    public Resource[] Resources { get; set; }

    /// <summary>
    /// To be used by application code (not for 
    /// </summary>
    [JsonIgnore]
    public Task[] Tasks { get; set; }

    /// <summary>
    /// Used by serializer
    /// </summary>
    [JsonProperty(nameof(Tasks))]
    private TaskSerializationHelper[] SerializationTasks { get; set; }

    [OnDeserialized]
    private void OnDeserialized(StreamingContext context)
    {
        var resourceLookup = this.Resources.ToDictionary(r => r.Id);
        this.Tasks = this.SerializationTasks.Select(t => t.ToTask(resourceLookup)).ToArray();
    }

    [OnSerializing]
    private void OnSerializing(StreamingContext context)
    {
        this.SerializationTasks = this.Tasks?.Select(t => new TaskSerializationHelper(t)).ToArray();
    }

    /// <summary>
    /// Converts from the helper to the Dto when casting
    /// </summary>
    /// <param name="helper"></param>
    public static implicit operator Dto(DtoSerializationHelper helper) => new Dto
    {
        Resources = helper.Resources,
        Tasks = helper.Tasks
    };

    /// <summary>
    /// Converts from the Dto to the helper when casting
    /// </summary>
    /// <param name="dto"></param>
    public static explicit operator DtoSerializationHelper(Dto dto) => new DtoSerializationHelper
    {
        Resources = dto.Resources,
        Tasks = dto.Tasks
    };

    /// <summary>
    /// A Task serialization helper class
    /// </summary>
    private class TaskSerializationHelper
    {
        public TaskSerializationHelper() { }

        public TaskSerializationHelper(Task task) : this()
        {
            this.Id = task.Id;
            this.Name = task.Name;
            this.Resource = task.Resource.Id;
        }

        public long Id { get; set; }
        public string Name { get; set; }
        public long Resource { get; set; }

        public Task ToTask(IDictionary<long, Resource> resourceLookup) =>
            new Task
            {
                Id = this.Id,
                Name = this.Name,
                Resource = resourceLookup is null || !resourceLookup.TryGetValue(this.Resource, out var resource)
                    ? throw new Exception($"Invalid resource {this.Resource}")
                    : resource
            };
    }
}

3. Implementation:

var input = Encoding.UTF8.GetString(Properties.Resources.input); // the posted Java-outputted JSON

var dtoSerializationHelper = JsonConvert.DeserializeObject<DtoSerializationHelper>(input);
var dto = (Dto)dtoSerializationHelper;

var deserializationResults = new
{
    distinctResources = dto.Resources?.Distinct().Count(),
    distinctTasks = dto.Tasks?.Distinct().Count(),
    distinctTaskResources = dto.Tasks?.Select(t => t.Resource).Distinct().Count()
};

Console.WriteLine($@"Distinct resources: {deserializationResults.distinctResources}");
Console.WriteLine($@"Distinct tasks: {deserializationResults.distinctTasks}");
Console.WriteLine($@"Distinct task resources: {deserializationResults.distinctTaskResources}");

if (deserializationResults.distinctResources != 2 ||
    deserializationResults.distinctTasks != 5 ||
    deserializationResults.distinctTaskResources != 2) throw new Exception("Deserialization failed");

Console.WriteLine();

var output = JsonConvert.SerializeObject((DtoSerializationHelper)dto);
var serializationResult = output == input;

Console.WriteLine($@"Input and output are same: {serializationResult}");
if (serializationResult) return;

Console.WriteLine($@"Output: {output}");
throw new Exception("Serialization failed");

4. Output:

Distinct resources: 2
Distinct tasks: 5
Distinct task resources: 2
Input and output are same: True

.NET Fiddle