Sharing a globally defined db conn with multiple packages

I've read a few StackOverflow answers on how we handling the db connection. Since it's a pool, we can define it globally and use it in multiple goroutines and it's safe.

The issue I'm having is that I have split my REST API into multiple packages. Each of these packages require a db connection, so I open a database connection in the startup. But even if I define the connection globally, it's only at the package level. What can I do to potentially share it among multiple packages?

For some context I'm using the PostgreSQL driver and gin-gonic in my application.

There is also the option of creating another package to hold your database connection-related settings. It can then have a package level global, which can be initialized in main and used in any package that is importing it.

This way, you can explicitly see that the database package is being imported. Here is some sample code.

package database

var (
	// DBCon is the connection handle
	// for the database
	DBCon *sql.DB
)

package main

import "myApp/database"

func main() {

	var err error
	database.DBCon, err = sql.Open("postgres", "user=myname dbname=dbname sslmode=disable")

}

package user

import "myApp/database"

func Index() {
    // database handle is available here
	database.DBCon

	...
}

Simple answer: pass an initialised connection pool to your packages' own globals.

e.g.

// package stuff

var DB *sql.DB

func GetAllStuff() (*Stuff, error) {
    err := DB.Query("...")
    // etc.
}

// package things

var DB *sql.DB

func GetAllThings() (*Thing, error) {
    err := DB.Query("...")
    // etc.
}

// package main

func main() {
    db, err := sql.Open("...")
    if err != nil {
        log.Fatal(err)
    }
    
    stuff.DB = db
    things.DB = db
    
    // etc.
}

We define package-level globals, make sure they're exported (capitalised) and then pass a pointer to our connection pool to them.

This is "okay", but can mask "where" things are being used. If you're looking at a handler it may not be clear where the connection is coming from, especially as your package grows. A more scalable approach might look like the below:

// package stuff

type DB struct {
    *sql.DB
}

func New(db *sql.DB) (*DB, error) {
    // Configure any package-level settings
    return &DB{db}, nil
}

func (db *DB) GetAllStuff() (*Stuff, error) {
    err := db.Query("...")
    // etc.
}

// package things

type DB struct {
    *sql.DB
}

func New(db *sql.DB) (*DB, error) {
    // Configure any package-level settings
    return &DB{db}, nil
}

func (db *DB) GetAllThings() (*Thing, error) {
    err := db.Query("...")
    // etc.
}

// package main

func main() {
    db, err := sql.Open("...")
    if err != nil {
        log.Fatal(err)
    }
    
    stuffDB, err := stuff.New(db)
    if err != nil {
        log.Fatal(err)
    }
    
    thingsDB, err := things.New(db)
    if err != nil {
        log.Fatal(err)
    }
    
    // Simplified.
    http.HandleFunc("/stuff/all", stuff.ShowStuffHandler(stuffDB))
    http.HandleFunc("/things/all", things.ShowThingsHandler(thingsDB))

    // etc.
}

func ShowStuffHandler(db *stuff.DB) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // We can use our stuff.DB here
        stuff, err := db.GetAllStuff()
        // etc.
    }
}

If you have more than just the DB connection as a dependency (e.g. config params, hostnames, etc.) , wrap them in an things.Env struct or a stuff.Env struct for each package.

An example would be to have a things.New("deps...") *Env function that returns a configured *things.Env that contains the dependencies used by your things package.

Update

I've ended up using the package config to initialize database, at the start of the application, It's pretty good, as you don't have to pass any unneeded param to functions.

Original

I'm still new to the golang world, but I'm handling this same problem, by declaring
local variables in the main package, in my case its package app

var mongoClient *mongo.Client
var mongoCtx context.Context
var mongoCancelCtx context.CancelFunc

then I'm passing them to a database file inside package config to connect to the database, and use the passed pointer to assign the result.

# app/app.go

var mongoClient *mongo.Client
var mongoCtx context.Context
var mongoCancelCtx context.CancelFunc
config.BootstrapDatabase(&mongoClient, &mongoCtx, &mongoCancelCtx)

# app/config/database.go

// BootstrapDatabase ...
func BootstrapDatabase(mongoClient **mongo.Client, ctx *context.Context, cancel *context.CancelFunc) {
	*ctx, *cancel = context.WithTimeout(context.Background(), 10*time.Second)

	client, err := mongo.Connect(*ctx, options.Client().ApplyURI(os.Getenv("MONGO_URI")))
	if err != nil {
		panic(err)
	}

	if err := client.Ping(*ctx, readpref.Primary()); err != nil {
		panic(err)
	}

	*mongoClient = client

	fmt.Println("Successfully connected and pinged.")
}

Then I'm creating a database reference and passing it to each package
that will make use of it.

# app/app.go

db := mongoClient.Database(os.Getenv("MONGO_DATABASE"))
repositories.BootstrapRepositories(db)

e := echo.New()
routes.BootstrapRoutes(e, db)

So the idea is to declare the database reference in the high level package which is my app/app.go in my app and pass it down as needed, for example in my package routes

#app/app.go


// BootstrapRoutes ...
func BootstrapRoutes(e *echo.Echo, db *mongo.Database) {

	// Set middleware
	e.Use(middleware.Logger())
	e.Use(middleware.Recover())

	// Users related routes
	e.POST("/api/users", controllers.CreateUser)

	// Starte web server
	e.Logger.Fatal(e.Start(":4242"))
}

I'm still experimenting with it, I don't know yet if this approach will skill
with many endpoints, or I need to try separating it into its own package, as suggested in the first thread of this issue.

Please share you thoughts I'll be happy to hear from you.

I am doing something like this. Each configuration file contains its own connection method and exports a global variable.

main.go

func main() {

	var cors = handlers.CORS(
		handlers.AllowedOrigins([]string{"*"}),
		handlers.AllowedHeaders([]string{"Content-Type", "x-api-key"}),
		handlers.AllowedMethods([]string{"POST","GET", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS"}),
	)

	server := &http.Server{
		Handler:      cors(apiRouter.Router),
		Addr:         "0.0.0.0:80",
		WriteTimeout: 15 * time.Second,
		ReadTimeout:  15 * time.Second,
	}

	if os.Getenv("STATUS") == "dev" {
		server.Addr = "0.0.0.0:3000"
	}

	// configuration
	configs.ConfigureS3()
	configs.ConnectRedis()
	configs.ConfigureDb()

	log.Fatal(server.ListenAndServe())
}

and in my configs package there are three files:<br>

configs/mysql.go

var Db *sql.DB

func ConfigureDb() {
	var err error

	mysqlConfig := &mysql.Config{
		User:   os.Getenv("MYSQL_USER"),
		Net: "tcp",
		Passwd: os.Getenv("MYSQL_PASSWORD"),
		Addr:   os.Getenv("MYSQL_HOST"),
		DBName: os.Getenv("MYSQL_DATABASE"),
	}

	Db, err = sql.Open(
		"mysql",
		mysqlConfig.FormatDSN(),
	)
	if err != nil {
		panic(err)
	}

	// Db connection configuration
	Db.SetConnMaxLifetime(time.Minute * 5)
	Db.SetMaxOpenConns(1000)
	Db.SetMaxIdleConns(1000) // I have to always set it to greater or equal to max open connection
}

configs/redis.go

var RedisPool *redis.Pool

func ConnectRedis() {
	RedisPool = &redis.Pool{
		MaxIdle:     3,
		IdleTimeout: 3 * time.Minute,
		Dial: func() (redis.Conn, error) {
			return redis.Dial(
				"tcp",
				os.Getenv("REDIS_HOST") + ":" + os.Getenv("REDIS_PORT"),
				redis.DialPassword(os.Getenv("REDIS_PASSWORD")),
			)
		},
	}
}

configs/aws.go

var S3Client *s3.Client

func ConfigureS3() {
	awsConfig, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		panic(err)
	}
	S3Client = s3.NewFromConfig(awsConfig)
}

For initialize a package-scoped value (here a database connection), just defined a variable with package scope.

In a init() function as main(), describe how to initialize the variable.

package database

var db *sql.DB

func init(){
  db, err = sql.Open("credentials")
  if err != nil{
    panic(err)
  }
}

// Getter for use everywhere else package
func DB()*sql.DB{
  return db
}

Go automatically execute init() function while importing the library and before main() function.

In you main.go file, import as dependancies as

package main

import _ "../folder/database" // init() have been executed, and initialize db

func main(){}