Golang - DTOs, Entities and Mapping

I am new to Go having come from a C# background, and I am just plain confused about structure a Go application.

Say I am building a REST API that will sit on top of a database. Also, say that, even after it is complete, this application could need to change somewhat frequently given the vicissitudes of the business, etc.

In C#, with tools like Entity Framework and DTOs, I somewhat alleviate this problem by abstracting the database from the results given by the controllers. If I change the name of a bunch of fields in the database, I might have to change my database access logic. Still, hopefully, the DTOs that I map to my entities using AutoMapper can remain the same, so I don't break frontend functionality that relies on a given DTO structure.

Should I replicate this structure with Go's structs? Something about such an approach seems wrong given that structs are just DTOs, and I will have quite a few DTO structs that are identical to the entity structs. I also have to setup logic to map entities to DTOs. This all just feels very unidiomatic somehow, and many of the examples I see on the web just serialize the database structs.

In short, how do people avoid excessive coupling between their API and the database in Go, and how would they broadly go about separating out the different parts of the application?

If it makes any difference, I am planning to use sqlx to marshal database results into structs which will mean more tags in addition to the JSON ones if I don't separate entities from DTOs.

In the case of a REST API, you'll typically deal with at least three different implementation layers:

• HTTP handler
• some sort of business logic/use case
• persistent storage/database interface

You can treat and build each of these separately which does not only decouple it but makes it just a lot more testable, too. These parts then are put together by injecting the necessary bits since they conform to interfaces you define. Usually this ends up leaving the main or a separate configuration mechanism the only place that's aware of what is combined and injected how.

The article Applying The Clean Architecture to Go applications illustrates very well how the various parts can be separated. How strictly you should follow this approach depends a little on the complexity of your project.

Below is a very basic breakdown, separating the handler from logic and database layer.

HTTP handler

The handler does nothing else than mapping the request values into local variables or possibly custom data structures if needed. In addition to that it just runs the use case logic and maps the result before writing it to the response. This is also a good place to map different errors to different response objects.

type Interactor interface {
    Bar(foo string) ([]usecases.Bar, error)
}

type MyHandler struct {
    Interactor Interactor
}

func (handler MyHandler) Bar(w http.ResponseWriter, r *http.Request) {
    foo := r.FormValue("foo")
    res, _ := handler.Interactor.Bar(foo)

    // you may want to map/cast res to a different type that is encoded
    // according to your spec
    json.NewEncoder(w).Encode(res)
}

Unit tests are a great way to test that the HTTP response contains the correct data for different results and errors.

Use case / business logic

Since the repository is just specified as an interface it's very easy to create unit tests for the business logic with different results returned by a mock repository implementation that also conforms to DataRepository.

type DataRepository interface {
    Find(f string) (Bar, error)
}

type Bar struct {
    Identifier string
    FooBar     int
}

type Interactor struct {
    DataRepository DataRepository
}

func (interactor *Interactor) Bar(f string) (Bar, error) {
    b := interactor.DataRepository.Find(f)

    // ... custom logic

    return b
}	

Database interface

The part talking to the database implements the DataRepository interface but is otherwise totally independent on how it turns the data into the expected types.

type Repo {
    db sql.DB
}

func NewDatabaseRepo(db sql.DB) *Repo {
    // config if necessary...

    return &Repo{db: db}
}

func (r Repo)Find(f string) (usecases.Bar, error) {
    rows, err := db.Query("SELECT id, foo_bar FROM bar WHERE foo=?", f)
    if err != nil {
        log.Fatal(err)
    }
    defer rows.Close()

    for rows.Next() {
        var id string, fooBar int
        if err := rows.Scan(&id, &fooBar); err != nil {
            log.Fatal(err)
        }
        // map row value to desired structure
        return usecases.Bar{Identifier: id, FooBar: fooBar}
    }

    return errors.New("not found")
}

Again, this allows testing the database operations separately without the need of any mock SQL statements.

> Note: The code above is very much pseudo code and incomplete.

I had .NET MVC experience prior to developing my own application in Go. I did miss that the mapper between BLL and DTO in .NET, but as I write more code in Go, I am getting used to the fact that there is not much free lunch in Go.

The fact that there is almost no framework, NHibernate, Entity, and automatic mapping between URI and views in Go indicates that you probably have to do a lot of work that is taken care of by other frameworks. While this may be inefficient in some people's opinion, it is certainly a great opportunity for learning as well as building a highly customizable application in a less-inefficient manner. Sure, you have to write your own SQL to perform CRUD actions, read the rows returned by the query, map sql.Rows to model, apply some business logic, save changes, map models to DTOs, and send a response back to the client.

As for the difference between DTO and model: DTO is a representation of model for view and has no behaviors (methods). Model is the abstraction of your business logic and has a lot of complex behaviours. I would never implement methods like "checkProvisioning()" or "applyCouponCode()" on a DTO object because they are business logic.

As for mapping database with your model, I won't take any shortcuts because most of the ORM available at this point is pretty primitive. I won't even try to use foreign keys if I have to build my database with ORM. It is better to build your database from a SQL script since you can also configure indexes, triggers, and other bells and whistles. Yes, if the schema changes, you will have to update quite a bit of code to reflect that change, but most of the time it only affects data access layer code.

I have a project that utilizes C#'s MVC design but is completely written in Go, and I bet it's more complex than any toy projects that you can find from published books. If reading the code help you understand the structure better, then go for it: https://github.com/yubing24/das