-
-
Notifications
You must be signed in to change notification settings - Fork 33
Home
This Elixir library supports convenient encoding and decoding of Avro messages.
It can read the Avro schema from local files or the Confluent® Schema Registry, caching data in memory for performance.
It supports reading and writing data Kafka wire format prefix and from Object Container Files formats. Along with Confluent® Schema References it has Inter-Schema references feature for older Schema Registry versions.
Many thanks to the AvroTurf Ruby gem for the initial inspiration 💙
Add Avrora to mix.exs
as a dependency:
def deps do
[
{:avrora, "~> 0.16"}
]
end
Configure the library in config/config.exs
:
config :avrora,
registry_url: "https://localhost:8081",
registry_auth: {:basic, ["username", "password"]}, # optional
schemas_path: Path.expand("./priv/schemas"),
registry_schemas_autoreg: false, # optional: if you want manually register schemas
convert_null_values: false, # optional: if you want to keep decoded `:null` values as is
convert_map_to_proplist: false # optional: if you want to restore the old behavior for decoding map-type
names_cache_ttl: :timer.minutes(5) # optional: if you want periodic disk reads
-
registry_url
- URL for the Schema Registry, defaultnil
-
registry_auth
– Credentials to authenticate in the Schema Registry, defaultnil
-
schemas_path
- Base path for locally stored schema files, default./priv/schemas
-
registry_schemas_autoreg
v0.13 - Flag for automatic schemas registration in the Schema Registry, defaulttrue
-
convert_null_values
v0.14 - Flag for automatic conversion of decoded:null
values intonil
, defaulttrue
-
convert_map_to_proplist
v0.15 restore old behaviour and confiugre decoding map-type to proplist, defaultfalse
-
names_cache_ttl
v0.10 - Time in ms to cache schemas by name in memory, default:infinity
Set names_cache_ttl
to :infinity
will cache forever (no more disk reads will
happen). This is safe when schemas are resolved in the Schema Registry by
numeric id or versioned name, as it is unique. If you need to reload schema
from the disk periodically, TTL different from :infinity
ensures that.
If the schema is resolved by name it will be always overwritten with the latest schema received from Schema Registry.v0.10
💡 Disable schemas auto-registration if you want to avoid storing schemas and manually control registration process. Also it is recommended to turn off auto-registration when schemas containing Confluent Schema References.v0.14
Avrora uses an in-memory cache to speed up schema lookup.
Add it to your supervision tree:
children = [
Avrora
]
Supervisor.start_link(children, strategy: :one_for_one)
Or start the cache process manually:
{:ok, pid} = Avrora.start_link()
If you like the project and want to support me on my sleepless nights, you can