To compile protobuf into the code of some programming language, you need the runtime for that language. The compiler itself doesn’t know how to compile protobuf into every language—it needs a language-specific runtime to do so.
Go has two runtimes to compile protobuf into Go code. The Go team and the protobuf team at Google developed the original runtime.[6] Then a team of folks who wanted more features forked the original runtime and developed it into gogoprotobuf, with more code-generation features and faster marshaling and unmarshaling. Projects like Etcd, Mesos, Kubernetes, Docker, CockroachDB, and NATS as well as companies like Dropbox and Sendgrid used gogoprotobuf. I used gogoprotobuf for my projects to integrate with Kubernetes’ protocol buffers and for gogoprotobuf’s features.
In March 2020, the Go team released a major revision of the Go API (APIv2)[7] for protocol buffers with improved performance[8] and a reflection API that enables adding features like those provided by gogoprotobuf. Projects[9] that used gogoprotobuf have begun switching to APIv2[10] because of APIv2’s improved performance, its new reflection API, its incompatibility with gogoprotobuf, and the gogoprotobuf project needing new ownership.[11] I recommend using APIv2, too.
To compile our protobuf into Go, we need to install the protobuf runtime by running the following command:
| | $ go get google.golang.org/protobuf/...@v1.25.0 |
You can now compile your protobuf by running the following command at the root of your project:
| | $ protoc api/v1/*.proto \ |
| | --go_out=. \ |
| | --go_opt=paths=source_relative \ |
| | --proto_path=. |
Look in the api/v1 directory and you’ll see a new file called log.pb.go. Open it up to see the Go code that the compiler generated from your protobuf code. Your protobuf message has been turned into a Go struct, along with some methods on the struct for marshaling to the protobuf binary wire format, and getters for your fields.
Since you’ll compile your protobuf every time you change them, it’s worth adding a Makefile file with a compile target that you can quickly run again and again. We’ll include a test target for testing our code too. So create a Makefile file at the root of your repo with the following code:
| | compile: |
| | protoc api/v1/*.proto \ |
| | --go_out=. \ |
| | --go_opt=paths=source_relative \ |
| | --proto_path=. |
| | |
| | test: |
| | go test -race ./... |
That’s all there is to compiling your protobuf code into Go code. Now let’s talk about how to work with the generated code and extend the compiler to generate your own code.