JSON field set to null vs field not there

Is there a way, in golang, to see if I can differentiate between a json field being set to null vs a json field not being there when unmarshalled into a struct? Because both set the value in the struct to be nil, but I need to know if the field was there to begin with and to see if someone set it to null.

{
  "somefield1":"somevalue1",
  "somefield2":null
}

VS

{
  "somefield1":"somevalue1",
}

Both jsons will be nil when unmarshalled into a struct.
Any useful resources will be very appreciated!

Use json.RawMessage to "delay" the unmarshaling process to determine the raw byte before deciding to do something:

var data = []byte(`{
        "somefield1":"somevalue1",
        "somefield2": null
}`)

type Data struct {
	SomeField1 string          
	SomeField2 json.RawMessage
}

func main() {
	d := &Data{}
	
	_ = json.Unmarshal(data, &d)
	
	fmt.Println(d.SomeField1)
	
	if len(d.SomeField2) > 0 {
        if string(d.SomeField2) == "null" {
		    fmt.Println("somefield2 is there but null")
        } else {
            fmt.Println("somefield2 is there and not null")
            // Do something with the data
        }
	} else {
        fmt.Println("somefield2 doesn't exist")
    }
}

See the playground https://play.golang.org/p/Wganpf4sbO

If you're on Go 1.18+ you can use a simple generic struct to know when a JSON value is undefined or null:

type Optional[T any] struct {
	Defined bool
	Value   *T
}

// UnmarshalJSON is implemented by deferring to the wrapped type (T).
// It will be called only if the value is defined in the JSON payload.
func (o *Optional[T]) UnmarshalJSON(data []byte) error {
	o.Defined = true
	return json.Unmarshal(data, &o.Value)
}

That's all, you can then just use this type in your structs:

type Payload struct {
	Field1 Optional[string] `json:"field1"`
	Field2 Optional[bool]   `json:"field2"`
	Field3 Optional[int32]  `json:"field3"`
}

And you'll be able to use the Defined field to know if the field was null or undefined.

Check this playground link for a full example: https://go.dev/play/p/JZfZyVVUABz

Original answer (pre-generics)

Another way to do this, with a custom type:

// OptionalString is a struct that represents a JSON string that can be
// undefined (Defined == false), null (Value == nil && Defined == true) or 
// defined with a string value
type OptionalString struct {
	Defined bool
	Value   *string
}

// UnmarshalJSON implements the json.Unmarshaler interface.
// When called, it means that the value is defined in the JSON payload.
func (os *OptionalString) UnmarshalJSON(data []byte) error {
	// UnmarshalJSON is called only if the key is present
	os.Defined = true
	return json.Unmarshal(data, &os.Value)
}


// Payload represents the JSON payload that you want to represent.
type Payload struct {
	SomeField1 string         `json:"somefield1"`
	SomeField2 OptionalString `json:"somefield2"`
}

You can then just use the regular json.Unmarshal function to get your values, for example:

	var p Payload
	_ = json.Unmarshal([]byte(`{
        	"somefield1":"somevalue1",
        	"somefield2":null
        }`), &p)
	fmt.Printf("Should be defined == true and value == nil: \n%+v\n\n", p)


	p = Payload{}
	_ = json.Unmarshal([]byte(`{"somefield1":"somevalue1"}`), &p)
	fmt.Printf("Should be defined == false \n%+v\n\n", p)
	
	p = Payload{}
	_ = json.Unmarshal([]byte(`{
        	"somefield1":"somevalue1",
        	"somefield2":"somevalue2"
        }`), &p)
	fmt.Printf("Parsed should be defined == true and value != nil \n%+v\n", p)
	if p.SomeField2.Value != nil {
	  fmt.Printf("SomeField2's value is %s", *p.SomeField2.Value)
	}

Should give you this output:

Should be defined == true and value == nil: 
{SomeField1:somevalue1 SomeField2:{Defined:true Value:<nil>}}

Should be defined == false 
{SomeField1:somevalue1 SomeField2:{Defined:false Value:<nil>}}

Parsed should be defined == true and value != nil 
{SomeField1:somevalue1 SomeField2:{Defined:true Value:0xc000010370}}
SomeField2's value is somevalue2

Link to the playground with the full example: https://play.golang.org/p/AUDwPKHBs62

Do note that you will need one struct for each type you want to wrap, so, if you need an optional number you'll need to create an OptionalFloat64 (all JSON numbers can be 64 bit floats) struct with a similar implementation.
If/when generics land in Go, this could be simplified to a single generic struct.

Good question.

I believe you can use https://golang.org/pkg/encoding/json/#RawMessage as:

type MyMessage struct {
  somefield1 string
  somefield2 json.RawMessage
}

So after unmarshalling you should have []byte("null") in case of null and nil if missing.

Here is a playground code: https://play.golang.org/p/UW8L68K068

If you unmarshall the object into a map[string]interface{} then you can just check if a field is there

type unMarshalledObject map[string]interface{}
json.Unmarshal(input, unMarshalledObject)
_, ok := unMarshalledObject["somefield2"]

Go Playground

If struct field is a pointer, JSON decoder will allocate new variable if the field is present or leave it nil if not. So I suggest to use pointers.

type Data struct {
    StrField *string
    IntField *int
}
...
if data.StrField != nil {
    handle(*data.StrField)
}

By using github.com/golang/protobuf/ptypes/struct and jsonpb github.com/golang/protobuf/jsonpb, you can do like this:

func TestFunTest(t *testing.T) {
	p := &pb.KnownTypes{}
	e := UnmarshalString(`{"val":null}`, p)
	fmt.Println(e, p)
	p = &pb.KnownTypes{}
	e = UnmarshalString(`{"val":1}`, p)
	fmt.Println(e, p)
	p = &pb.KnownTypes{}
	e = UnmarshalString(`{"val":"string"}`, p)
	fmt.Println(e, p)
	p = &pb.KnownTypes{}
	e = UnmarshalString(`{}`, p)
	fmt.Println(e, p)
}

Output:

[ `go test -test.run="^TestFunTest$"` | done: 1.275431416s ]
	<nil> val:<null_value:NULL_VALUE > 
	<nil> val:<number_value:1 > 
	<nil> val:<string_value:"string" > 
	<nil> 
	PASS