Flattening marshalled JSON structs with anonymous members in Go

Given the following code: (reproduced here at play.golang.org.)

package main
import (
"encoding/json"
"fmt"
)
type User struct {
Id   int    `json:"id"`
Name string `json:"name"`
}
type Session struct {
Id     int `json:"id"`
UserId int `json:"userId"`
}
type Anything interface{}
type Hateoas struct {
Anything
Links map[string]string `json:"_links"`
}
func MarshalHateoas(subject interface{}) ([]byte, error) {
h := &Hateoas{subject, make(map[string]string)}
switch s := subject.(type) {
case *User:
h.Links["self"] = fmt.Sprintf("http://user/%d", s.Id)
case *Session:
h.Links["self"] = fmt.Sprintf("http://session/%d", s.Id)
}
return json.MarshalIndent(h, "", "    ")
}
func main() {
u := &User{123, "James Dean"}
s := &Session{456, 123}
json, err := MarshalHateoas(u)
if err != nil {
panic(err)
} else {
fmt.Println("User JSON:")
fmt.Println(string(json))
}
json, err = MarshalHateoas(s)
if err != nil {
panic(err)
} else {
fmt.Println("Session JSON:")
fmt.Println(string(json))
}
}

I'm attempting to have the rendered JSON look correct in my case that means something like:

User JSON:
{
"id": 123,
"name": "James Dean",
"_links": {
"self": "http://user/123"
}
}
Session JSON:
{
"id": 456,
"userId": 123,
"_links": {
"self": "http://session/456"
}
}

Unfortunately Go is treating the anonymous member as a real named thing, so it's taking the defined type (Anything) and naming the JSON thusly:

User JSON:
{
"Anything": {
"id": 123,
"name": "James Dean"
},
"_links": {
"self": "http://user/123"
}
}
Session JSON:
{
"Anything": {
"id": 456,
"userId": 123
},
"_links": {
"self": "http://session/456"
}
}

There's no clear docs on the handling of anonymous members in JSON, from the docs:

> Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph. An anonymous struct field with a name given in its JSON tag is treated as having that name, rather than being anonymous.
>
> Handling of anonymous struct fields is new in Go 1.1. Prior to Go 1.1, anonymous struct fields were ignored. To force ignoring of an anonymous struct field in both current and earlier versions, give the field a JSON tag of "-".

This doesn't make clear if there's a way to flatten out, or hint to the Marshaller what I am trying to do.

I'm certain that there might be, as there is a special case, magic name that has a special meaning to rename the root element of an XML document in the XML marshaller.

In this case, I'm also not attached to the code in any way, my use-case is to have a function that accepts interface{}, *http.Request, http.ResponseWriter and write back HATEOAS documents down the wire, switching on the type passed, to infer which links to write back into the JSON. (thus access to the request, for request host, port, scheme, etc, as well as to the type itself to infer the URL and known fields, etc)

Working playground link: http://play.golang.org/p/_r-bQIw347

The gist of it is this; by using the reflect package we loop over the fields of the struct we wish to serialize and map them to a map[string]interface{} we can now retain the flat structure of the original struct without introducing new fields.

Caveat emptor, there should probably be several checks against some of the assumptions made in this code. For instance it assumes that MarshalHateoas always receives pointers to values.

package main
import (
"encoding/json"
"fmt"
"reflect"
)
type User struct {
Id   int    `json:"id"`
Name string `json:"name"`
}
type Session struct {
Id     int `json:"id"`
UserId int `json:"userId"`
}
func MarshalHateoas(subject interface{}) ([]byte, error) {
links := make(map[string]string)
out := make(map[string]interface{})
subjectValue := reflect.Indirect(reflect.ValueOf(subject))
subjectType := subjectValue.Type()
for i := 0; i < subjectType.NumField(); i++ {
field := subjectType.Field(i)
name := subjectType.Field(i).Name
out[field.Tag.Get("json")] = subjectValue.FieldByName(name).Interface()
}
switch s := subject.(type) {
case *User:
links["self"] = fmt.Sprintf("http://user/%d", s.Id)
case *Session:
links["self"] = fmt.Sprintf("http://session/%d", s.Id)
}
out["_links"] = links
return json.MarshalIndent(out, "", "    ")
}
func main() {
u := &User{123, "James Dean"}
s := &Session{456, 123}
json, err := MarshalHateoas(u)
if err != nil {
panic(err)
} else {
fmt.Println("User JSON:")
fmt.Println(string(json))
}
json, err = MarshalHateoas(s)
if err != nil {
panic(err)
} else {
fmt.Println("Session JSON:")
fmt.Println(string(json))
}
}

sorry, but I think the JSON you're trying to generate is not a valid JSON object and thus it may be the reason the JsonMarshal is not playing game with you.

The object may not consumable via JavaScript as it contains two objects, unless you wrap the objects in an array.

[
{
"id": 123,
"name": "James Dean",
"_links": {
"self": "http://user/123"
}
},
{
"id": 456,
"userId": 123,
"_links": {
"self": "http://session/456"
}
}
]

Then you would be able to consume this JSON, example:

var user, session;
user = jsonString[0]; 
session = jsonString[1];

Consider giving your objects root names might be a better consideration, example:

{
"user": {
"id": 123,
"name": "James Dean",
"_links": {
"self": "http://user/123"
}
},
"session": {
"id": 456,
"userId": 123,
"_links": {
"self": "http://session/456"
}
}
}

and consumed as, example:

var user, session;
user = jsonString.user;
session = jsonString.session;

I hope this helps you

Make use of omitempty tags and a bit of logic so you can use a single type that produces the right output for different cases.

The trick is knowing when a value is considered empty by the JSON encoder. From the encoding/json documentation:

> The empty values are false, 0, any nil pointer or interface value, and
> any array, slice, map, or string of length zero.

Here is your program slightly modified to produce the output you wanted. It omits certain fields when their values are "empty" - specifically, the JSON encoder will omit ints with "0" as value and maps with zero-length.

package main
import (
"encoding/json"
"fmt"
)
type User struct {
Id     int               `json:"id"`
Name   string            `json:"name,omitempty"`
UserId int               `json:"userId,omitempty"`
Links  map[string]string `json:"_links,omitempty"`
}
func Marshal(u *User) ([]byte, error) {
u.Links = make(map[string]string)
if u.UserId != 0 {
u.Links["self"] = fmt.Sprintf("http://user/%d", u.UserId)
} else if u.Id != 0 {
u.Links["self"] = fmt.Sprintf("http://session/%d", u.Id)
}
return json.MarshalIndent(u, "", "    ")
}
func main() {
u := &User{Id: 123, Name: "James Dean"}
s := &User{Id: 456, UserId: 123}
json, err := Marshal(u)
if err != nil {
panic(err)
} else {
fmt.Println(string(json))
}
json, err = Marshal(s)
if err != nil {
panic(err)
} else {
fmt.Println(string(json))
}
}

Copy on play.golang.org.

One approach which I've worked through involves the use of methods on types. Please see http://play.golang.org/p/bPWB4ryDQn for more details.

Basically, you work the problem from the opposite angle -- instead of "encapsulating" a base type into a Hateoas type, you instead incorporate the required map in each of your base types. Then, implement a method on each of those base types, which is responsible for updating the Links field accordingly.

This produces the intended result, and with only marginal source code boiler-plate.

{
"id": 123,
"name": "James Dean",
"_links": {
"self": "http://user/123"
}
}
{
"id": 456,
"userId": 123,
"_links": {
"self": "http://session/456"
}
}

I believe any other way than this, particularly if you persue the embed-and-extend approach, will require implementing a custom marshaler (http://golang.org/pkg/encoding/json/#Marshaler) and will likely require the use of the reflect package as well, especially since anything is of type interface{}.

This is kinda tricky. One thing for certain, however, is that the doc addresses your sample with this:

> Interface values encode as the value contained in the interface

from the same link so there is nothing more to do. "Anything" while it is anonymous it is an interface variable and so I might expect the behavior in your sample.

I took your code and made some changes. This example works but has some side effects. In this case I needed to change the ID member names so that there was nome collision. But then I also changed the json tag. If I did not change the tag then the code seemed to take too long to run and the overlapping tags were both omitted. (here).

PS: I cannot say with any certainty but I would guess that there is a problem with the assumption. I would assume that anything that I was going to Marshal I would want to be able to UnMarshal. Your flattener just won't do that. If you want a flattener you might have to fork the JSON encoder and add some values to the tags (much like there is a 'omitempty' you might add a 'flatten'.

I needed to do something similar and tried your technical of the embedded interface and (no surprise) had the same issue.

I didn't want to have to alter all the possible structs that needed additional fields, but ultimately I compromised with this solution:

http://play.golang.org/p/asLFPx76jw

package main

import (
"encoding/json"
"fmt"
)
type HateoasFields struct {
Links []interface{} `json:"_links,omitempty"`
}
type User struct {
Id   int    `json:"id"`
Name string `json:"name"`
HateoasFields
}
type Session struct {
Id     int `json:"id"`
UserId int `json:"userId"`
HateoasFields
}
func main() {
u := &User{Id: 123, Name: "James Dean"}
s := &Session{Id: 456, UserId: 123}
u.Links = []interface{}{fmt.Sprintf("http://user/%d", u.Id)}
s.Links = []interface{}{fmt.Sprintf("http://session/%d", s.Id)}
uBytes, _ := json.Marshal(u)
sBytes, _ := json.Marshal(s)
fmt.Println(string(uBytes))
fmt.Println(string(sBytes))
}

Which outputs:

{"id":123,"name":"James Dean","_links":["http://user/123"]}
{"id":456,"userId":123,"_links":["http://session/456"]}

Rather than embedding the original structs in a struct with the additional fields, I did the opposite.

When looking at your original code, I don't think this solution is that awesome. How is this better than adding the Links property to the original struct? But for my app needs it was the best thing.

I like this solution if you only have maps to worry about.

// Flatten takes a map and returns a new one where nested maps are replaced
// by dot-delimited keys.
func Flatten(m map[string]interface{}) map[string]interface{} {
o := make(map[string]interface{})
for k, v := range m {
switch child := v.(type) {
case map[string]interface{}:
nm := Flatten(child)
for nk, nv := range nm {
o[k+"."+nk] = nv
}
default:
o[k] = v
}
}
return o
}