Initializing Go AppEngine app with Cloud Datastore

in init() function for GAE's golang app, how can I set-up initial values for my application ?

How to read from Cloud Datastore in the init() function or immediately after applications start-up ? As I understand, server cannot write to the local filesystem and Cloud Datastore is the only option ?

I need some global variables and slices of data..

Using static files

On AppEngine you don't have access to the file system of the host operating system, but you can access files of your web application (you have read-only permission, you can't change them and you can't create new files in the app's folder).

So the question is: can your application's code change the data that you want to read and use for initialization? Or is it fine if it is deployed "statically" with your app's code?

If you don't need to change it (or only when you redeploy your app), easiest is to store it as a "static" file as part of your webapp. You may refer to files of your app using relative paths, where the current or working directory is your app's root. E.g. if your app contains a data folder in its root (where app.yaml resides), and there is an init_values.txt file inside the data folder, you can refer to it with the path: data/init_values.txt.

One important note: not every file is readable by code, this depends on the app configuration. Quoting from Configuring with app.yaml / Static file handlers:

> If you have data files that need to be read by the application code, the data files must be application files, and must not be matched by a static file pattern.

Using the Datastore

You can't use AppEngine services that require a Context outside of handlers (because the creation of a Context requires an *http.Request value). This by nature means you can't use them in package init() functions either.
_{Note that you can use them from cron jobs and tasks added to task queues, because tasks and cron jobs are executed by issuing HTTP GET requests.}

You have to restructure your code so that your initialization (e.g. reading from the Datastore) gets called from a handler.

Example of achieving this with Once.Do():

var once = sync.Once{}

func MainHandler(w http.ResponseWriter, r *http.Request) {
	ctx := appengine.NewContext(r)
	once.Do(func() { mysetup(ctx) })
	// do your regular stuff here
}

func mysetup(ctx appengine.Context) {
	// This function is executed only once.
	// Read from Datastore and initialize your vars here.
}

"Utilizing" warmup requests

Yes, this may cause first requests to take considerably longer to serve. For this purpose (to avoid this) I recommend you to utilize Warmup requests. A warmup request is issued to a new instance before it goes "live", before it starts serving user requests. In your app.yaml config file you can enable warmup requests by adding -warmup to the inbound_services directive:

inbound_services:
  - warmup

This will cause the App Engine infrastructure to first issue a GET request to /_ah/warmup. You can register a handler to this URL and perform initialization tasks. As with any other request, you will have an http.Request in the warmup handler.

But please note that:

> ..you may encounter loading requests, even if warmup requests are enabled in your app.

Which means that in rare cases it may happen a new instance will not receive a warmup request, so its best to check initialization state in user handlers too.

Related questions:

https://stackoverflow.com/questions/34629056/how-do-i-store-the-private-key-of-my-server-in-google-app-engine

https://stackoverflow.com/questions/29912318/fetching-a-url-from-the-init-func-in-go-on-appengine

https://stackoverflow.com/questions/33336850/environment-variables-specified-on-app-yaml-but-its-not-fetching-on-main-go