When to use embed?

Starting Go 1.16, we have the embed directive. It helps us embed an extra file (say, a .txt file) into the executable without having to supply that file additionally. (Reference, here).

I don't quite follow what construes an 'extra file'. Are all files not ending in .go construed as extra files? Any exceptions?

I want to ship a binary protobuf definition file as a part of the main binary so that my code can read it. Would this file be an extra file? Or would it be a part of the main binary itself?

It is often the case that a running Go program does not have access to the file system directories from which the the program was compiled.

The embed feature slurps up files in the package source code tree at build time and makes those files available at runtime.

Packages should embed files from the package source code tree that the package needs to access at runtime.

Here's an example: A package foo has has the source file foo.go. The package needs to access the file abc.txt in the same directory as foo.go.

If the current working directory is the source code code directory, then the package code can access the file using os.Open or anything that calls os.Open:

  f, err := os.Open("abc.txt")
  if err != nil {
      // handle error
  }
  defer f.Close()
  // do something with f

This code can fail. The current working directory at runtime is not necessarily the same as the package source code directory. It usually is not the same directory. The package source directory may not even be available to the running application.

In this case, the package should embed the file to ensure that the package can access the contents of the file:

//go:embed abc.txt
var abc []byte

Embed allows you to treat non-Go files as a true part of the source tree, rather than as a run-time dependency. You can see embedded files as an extension of your source code, and you can treat them similarly (mainly that the contents of the file are assured by your usual source control measures).

With normal files, even if they are in theory a static element of the application, they still generally require error checking and validation every time the program is run, or else inexplicable errors may ensue. With an embedded file, you don't have to worry about this once all tests are passed and you deploy the application.

The main issues of an embedded file are that you can't change the contents without rebuilding the program, and the (potentially large) file size gets added to the binary. If that doesn't pose a problem, go ahead.

Any time you can reasonably embed a file instead of loading it at run time, it is probably desirable to do so as it makes program deployment simpler.

To add to the other answers, one shortcoming of the embed directive is that you cannot use it to embed OS-level resources, such as Windows application manifests or an icon for display next to the binary in Windows Explorer; at least not in a way that the OS will pick up on it.

On Windows, you would use something like goversioninfo to embed those.