Organizing Environment Variables Golang

In Node.js I use the nconf module to house environment variables like S3 keys, GCM keys, etc for each of my projects.

I haven't been able to find a similar solution in Go.

What are the generally accepted tools to help manage environment variables for each Go project?

Thanks in advance.

I would strongly recommend using github.com/namsral/flag instead.
It's like the built in flag except you can also supply the parameters via environment variables.

For example, suppose you have this code:

package main

import "fmt"
import "github.com/namsral/flag"

func main() {
    var port = 3000
	flag.IntVar(&port, "port", port, "Port number")
    flag.Parse()
	fmt.Println("You seem to prefer", port)
}

Then you can supply the values with either a command line option or an environment variable:

:~/dev/GO$ go run dummy.go
You seem to prefer 3000
:~/dev/GO$ go run dummy.go -port=1234
You seem to prefer 1234
:~/dev/GO$ PORT=4321 go run dummy.go
You seem to prefer 4321
:~/dev/GO$ PORT=4321 go run dummy.go -port=5555
You seem to prefer 5555

This might matter when it's hard to supply command line args. For example, if you use gin to automatically restart a server you have no way to supply command line arguments since gin is just calling go run on the main code without any arguments passed along.

I did some reading on this a while back when I was getting started with Go. According to this link, http://peter.bourgon.org/go-in-production/, they recommend using CLI flags (parameters) instead of environment vars - they even convert environment vars to flags to their CLI apps.

It took some getting used to; but, I really do see the advantages of going pure CLI flags between development, staging and production environments - having specific scripts for each environment.

For example, here's a little web app I wrote recently:

// global flags
var isdebug bool
var port int
var cert string
var key string
var dbdsn string
var dbmaxidle int
var dbmaxopen int
var imguri string

// init is the entry point for the entire web application.
func init() {

	log.Println("Starting wwwgo ...")

	// setup the flags
	//flag.StringVar(&host, "host", "", "Specify a host to redirect to. Use this to redirect all traffic to a single url.")
	flag.IntVar(&port, "port", 8080, "Specify the port to listen to.")
	flag.BoolVar(&isdebug, "isdebug", false, "Set to true to run the app in debug mode.  In debug, it may panic on some errors.")
	flag.StringVar(&cert, "cert", "", "Enables listening on 443 with -cert and -key files specified.  This must be a full path to the certificate .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.")
	flag.StringVar(&key, "key", "", "Enables listening on 443 with -cert and -key files specified.  This must be a full path to the key .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.")
	flag.StringVar(&dbdsn, "dbdsn", "root:root@tcp(localhost:3306)/dev_db?timeout=5s&tls=false&autocommit=true", "Specifies the MySql DSN connection.")
	flag.IntVar(&dbmaxidle, "dbmaxidle", 0, "Sets the database/sql MaxIdleConns.")
	flag.IntVar(&dbmaxopen, "dbmaxopen", 500, "Sets the database/sql MaxOpenConns.")
	flag.StringVar(&imguri, "imguri", "/cdn/uploads/", "Set this to the full base uri of all images, for example on a remote CDN server or local relative virtual directory.")
	flag.Parse()

	// log our flags
	if isdebug != false {
		log.Println("DEBUG mode enabled")
	}
	if cert != "" && key != "" {
		log.Println("Attempting SSL binding with supplied cert and key.")
	}
	if dbdsn != "" {
		log.Printf("Using default dbdsn: %s", dbdsn)
	}
	
	...
}

This really becomes nice in staging/production environments.

A quick ./wwwgo -h for "what the heck was that parameter?" gives you full documentation:

admin@dev01:~/code/frontend/src/wwwgo [master]$ ./wwwgo -h
Usage of ./wwwgo:
  -cert="": Enables listening on 443 with -cert and -key files specified.  This must be a full path to the certificate .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.
  -dbdsn="root:root@tcp(localhost:3306)/dev_db?timeout=5s&tls=false&autocommit=true": Specifies the MySql DSN connection.
  -dbmaxidle=0: Sets the database/sql MaxIdleConns.
  -dbmaxopen=500: Sets the database/sql MaxOpenConns.
  -imguri="/cdn/uploads/": Set this to the full base uri of all images, for example on a remote CDN server or local relative virtual directory.
  -isdebug=false: Set to true to run the app in debug mode.  In debug, it may panic on some errors.
  -key="": Enables listening on 443 with -cert and -key files specified.  This must be a full path to the key .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.
  -port=8080: Specify the port to listen to.

Very nice to have many options at CLI, and no documentation required - it's built into the flags package.

You can clearly see the defaults immediately.

With this type of documentation, I tend to setup all the defaults for common "development environments" that the team uses. We all have root/root access to our local databases. We all are using port 8080 for this particular web app during development, etc. That way, you just have to run:

go build
./wwwgo

And the app runs with all defaults - defaults that are documented. In production, just override the defaults. The built-in flag parsers will panic the application if any parameters are in the wrong format, which is also very nice.

We have used this for a large scale microservice application. This doesn't involve use of any third party libraries, just plain go lang using reflection. It is very simple to use and follows DRY principle. Adding new environment variables and configuring defaults is quite easy. You can also set loading from a config file as a fallback by changing just 2 lines of code. Checkout github page for more info.

package config
import (
"fmt"
"os"
"reflect"
)
/* Tag names to load configuration from environment variable */
const (
ENV     = "env"
DEFAULT = "default"
)
type Configuration struct {
Port        string `env:"port" default:"3009"`
MongoURL    string `env:"MongoUrl" default:"mongodb://localhost:27017/test"`
UserService string `env:"UserService" default:"http://localhost:3005"`
AuthService string `env:"AuthService" default:"http://localhost:3050"`
Debug       string `env:"Debug" default:"true"`
}
/* Non-exported instance to avoid accidental overwrite */
var serviceConfig Configuration
func setConfig() {
// ValueOf returns a Value representing the run-time data
v := reflect.ValueOf(serviceConfig)
for i := 0; i < v.NumField(); i++ {
// Get the field tag value
tag := v.Type().Field(i).Tag.Get(ENV)
defaultTag := v.Type().Field(i).Tag.Get(DEFAULT)
// Skip if tag is not defined or ignored
if tag == "" || tag == "-" {
continue
}
a := reflect.Indirect(reflect.ValueOf(serviceConfig))
EnvVar, Info := loadFromEnv(tag, defaultTag)
if Info != "" {
fmt.Println("Missing environment configuration for '" + a.Type().Field(i).Name + "', Loading default setting!")
}
/* Set the value in the environment variable to the respective struct field */
reflect.ValueOf(&serviceConfig).Elem().Field(i).SetString(EnvVar)
}
}
func loadFromEnv(tag string, defaultTag string) (string, string) {
/* Check if the tag is defined in the environment or else replace with default value */
envVar := os.Getenv(tag)
if envVar == "" {
envVar = defaultTag
/* '1' is used to indicate that default value is being loaded */
return envVar, "1"
}
return envVar, ""
}
/*GetConfiguration :Exported function to return a copy of the configuration instance */
func GetConfiguration() Configuration {
return serviceConfig
}
func init() {
setConfig()
fmt.Printf("Service configuration : %+v\n ", serviceConfig)
}

Well I prefer go-arg for setting environment variables. It is easy to use and has nice features.

For example:

package configs

import (
	"fmt"
	"github.com/alexflint/go-arg"
)

type config struct {
	 DBHost string `arg:"env:DBHost, -D, --dbhost" help:"Host of the database" placeholder:"DBHost"`
}

var Config config

func main(){

  arg.MustParse(&Config)
  fmt.Println(Config.DBHost)

}

With this library either you can take the variable from your env or you can pass through args.

export DBHost=127.0.0.1

or

go run ./main.go --dbhost=127.0.0.1

In nodejs, nconf is a great tool. It gives more than storing secret key.<br>
In Golang, as far as i know there is two great package to use .env file easily , godotenv and viper , I prefer godotenv beacuse it's much easier .<br>

• Pros: Developers won't see your production secrets. You can use different secrets in dev, test, and production, without having to modify the code.
• Cons: Malicious code can read your secrets. The bulk of your application's code is probably open-source libraries. Bad code may creep in without you knowing it.

Example for using godotenv

First run this command inside your terminal,

go get github.com/joho/godotenv

In your .env file

S3_BUCKET=YOURS3BUCKET
SECRET_KEY=YOURSECRETKEYGOESHERE

In your main.go file

package main
import (
"github.com/joho/godotenv"
"log"
"os"
)
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal("Error loading .env file")
}
s3Bucket := os.Getenv("S3_BUCKET")
secretKey := os.Getenv("SECRET_KEY")
// now do something with s3 or whatever
}