This package provides both a code generator that generates Go data structures from Avro schemas and a mapping between native Go data types and Avro schemas.
The API is modelled after that of Go's standard library encoding/json package.
The documentation can be found here.
It also provides support for encoding and decoding messages using an Avro schema registry - see github.com/heetch/avro/avroregistry.
When the avrogo
command generates Go datatypes from Avro schemas, it uses the following rules:
"int"
is represented asint
"long"
is represented asint64
"float"
is represented asfloat32
"double"
is represented asfloat64
"string"
is represented asstring
"boolean"
is represented asbool
"bytes"
is represented as[]byte
"null"
is represented as the Go valuenil
{"type": "array", "items": T}
is represented as[]T
{"type": "map", "values": T}
is represented asmap[string]T
{"type": "enum", "name": "E", "symbols": ["red", "green", "blue"]}
is represented a Go int type withString
,MarshalText
andUnmarshalText
methods so it will encode as a string when used in JSON.{"type": "fixed", "size": 123, "name": "F"}
will encode as a Go[123]byte
type namedF
["null", T]
encodes as*T
[T, "null"]
encodes as*T
[T₁, T₂, ...]
(a union) encodes asinterface{}
that should hold only the types forT₁
,T₂
, etc.{"type": "record", "name": "R", "fields": [....]}
encodes as a Go struct type namedR
with corresponding fields.
If a definition has a go.package
annotation the type from that package will be used instead of generating a Go type. The type must be compatible with the Avro schema (it may contain extra fields, but all fields in common must be compatible).
If a definition has a go.name
annotation the associated string will be used for the generated Go type name.
github.com/linkedin/goavro/v2, is oriented towards dynamic processing of Avro data. It does not provide an idiomatic way to marshal/unmarshal Avro data into Go struct values. It does, however, provide good support for encoding and decoding with the standard Avro JSON format, which this package does not.
github.com/actgardner/gogen-avro was the original inspiration for this package. It generates Go code for Avro schemas. It uses a neat VM-based schema for encoding and decoding (and is also used by this package under the hood), but the generated Go data structures are awkward to use and don't reflect the data structures that people would idiomatically define in Go.
For example, in gogen-avro
the Avro type ["null", "int"]
(either null
or an integer) is represented as a
struct containing three members, and an associated enum type:
type UnionNullIntTypeEnum int
const (
UnionNullIntTypeEnumNull UnionNullIntTypeEnum = 0
UnionNullIntTypeEnumInt UnionNullIntTypeEnum = 1
)
type UnionNullInt struct {
Null *types.NullVal
Int int32
UnionType UnionNullIntTypeEnum
}
With heetch/avro
, the above type is simply represented as a *int
, a representation
likely to be familiar to most Go users.
A github.com/heetch/avro/avroregistrytest
package is provided to run
integration test against a real schema registry.
import "github.com/heetch/avro/avroregistrytest"
type X struct {
A int
}
avroregistrytest.Register(context.Background(), t, A{}, "test-topic")
This code snippet register an avro type for X
struct for
test-topic
in the schema registry defined by KAFKA_REGISTRY_ADDR
environment variable that must set to host:port
form.