Transmitting global data, or run-specific data, to a Beam DoFn in go

I'm writing a Beam ParDo transform in Go for a streaming Dataflow pipeline, as a DoFn. I'm trying to find a way to add a map that was computed at runtime, but pre-pipeline, to every DoFn. Putting it in using the state API seems not quite right, as it's constant data for the duration of the pipeline. But I can't seem to pass in a pre-initialized DoFn to do this. I tried

type EngineMap struct {
    Map map[string]string 
}

type ResultProcessor struct {
	engineMap EngineMap
}

... (ProcessElement defined, initialization)


processor := ResultProcessor{}
processor.engineMap.Map = make(map[string]string)
for k, v := range engines.Map {
	processor.engineMap.Map[k] = v
}
register.DoFn2x1[context.Context, []byte, []string](&processor)

... (pipeline initialized, input "lines" defined)

result := beam.ParDo(s, &processor, lines)

but when I run this, the map in engineMap is still empty when the ProcessElement() method runs, even though it isn't after the for loop. I could pass this data as a side input, but this seems unnecessarily complicated for a fairly small map that is constant at pipeline run time, especially for a streaming pipeline.

Is there another way to pass the data along?

The root cause is that the engineMap field is unexported, so it's data cannot be serialized. Only Exported Fields (say, EngineMap can be serialized. This is a property of "general" coders which rely on reflection, like JSON or Beam Schema Row encodings.

https://beam.apache.org/documentation/programming-guide/#user-code-serializability

It's not necessary and not recommended to register the same instance of a DoFn as used in the pipeline. (If that could work, we wouldn't need to register at all). DoFn Registration should happen in an init block or at least prior to beam.Init() is called in main.

If I understand correctly, the map you are using is just a DoFn member variable which remains constant after the DoFn initialization? In that case, I would suggest to make the member variable engineMap public for it to be serialized and deserialized in DoFn instances created for bundles.

The StartBundle has the same elements as that of ProcessElement method. https://github.com/apache/beam/blob/b68d38e32c2aac51170da16c4d9c479420754009/sdks/go/pkg/beam/pardo.go#L240

An example with StartBundle (This is a fairly large example so I would suggest focusing on a single DoFn): https://github.com/apache/beam/blob/67e6726ffeb47d2ada0122369fa230833ce0f026/sdks/go/examples/large_wordcount/large_wordcount.go#L207