C++ code generator based on xsdata to generate code base off a schema (YML or XSD).
- Generate Protobuf converter classes capable of converting a schema defined C++ message class to a protobuf message.
- Generate C++ Message classes with built in serialization capability based off a schema.
- Generate .proto files for the Protobuf compiler based off a yaml schema.
Generate Protobuf converters
poetry run python -m metatemplate -t protobuf_converters ./schemas/yaml/sample.yaml
Generate C++ message classes
poetry run python -m metatemplate -t api ./schemas/yaml/sample.yaml
Generate .proto files for the protobuf compiler
poetry run python -m metatemplate -t protobuf ./schemas/yaml/sample.yaml
If you already have an environment set up, or are running inside a built container:
python -m metatemplate -t [TEMPLATE_TYPE] ./schemas/yaml/sample.yaml
NOTE: if running in a poetry env, prefix commands with poetry run
NOTE: template generation is ADDITIVE meaning existing files in src/
and test/
are not deleted, you should generally delete those directories before generating.
Given an input message/object spec, generate one or more template based sets of code to use or convert data following that spec. Primary support is xsd and yml, see section below about all supported formats.
Templates can live inside the application, or referenced from a location on disk outside the repo. Unlikely you'll have an external set of templates, but you could. See filters.py
for the filters or tests that can be used on the variables in template scope (jinja2 speak, docs here).
Each folder under metatemplate/templates
can be considered an output "type" which can be enabled using the -t
The file structure under the first directory or "type" specifies the structure to render the files to. This allows the templates to be stored
exactly how they will be rendered (in subdirs). The output directory for non-test classes is src/[expanded namespace]/[template structure]/[filename]
The exception is test, which is rendered based on the structure after test
after test/unit
You will notice the filenames are strange, they map to a set of features to define how and when the files are rendered.
These are files that are rendered at most once per package and follow the naming convention :[filter]<[filename_pattern]>.[ext]
where the []
are not include, just representing a variable in the string, e.g.: :utils<Clock>.h
For globals, only one filter exists: utils
. If set on a file, it will not be rendered in the case were a --utils-ns
is specified, meaning this is a required for a utils class, which should be shared for derived api packages.
ignoring the filter, the file <Clock>.h
is rendered to Clock.h
. the filename_pattern
could include python f-string with variables but those variables would have to be passed into tpl.to_path
in generator.py
to support any variables.
Full template spec, some parts option, see below:
_[class_type]:[filter_name]<[filename_pattern]>.[ext]
The _[class_type]
prefix is required (again, without "[]", just representing a variable). where class_type
is one of: [all, enum, alias, variant, struct]
. Except in the case of all
, files are specific to the type of class, therefore "struct" class objects only render templates starting with _struct
.
If a filter is included (:[filter_name]
, :
not included in the case of no filter), it means that the template has a secondary condition that defines whether it is rendered. filter_name
must associate to a method on FilterMethods
in overrides/helpers.py
, and the response is None if the class should not be rendered, or a dict containing zero or more extra template context params if it should be rendered. e.g. :is_abstract
filtered template files are only rendered for abstract base classes and in the context of their template, a new variable derived
is available with the list of classes that extend the abstract type.
If a filename_pattern
is included (if not, it implies <{type_name}>
) then the file generated by the template for a given class is the value within <>
supporting f-strings, currently only type_name
is available, which is the name of the class.
The ext
is required, the shortest filename is _[class_type].[ext]
.
the api templates have examples of all these features, render the sample Message schema xsd and see the resulting files for an example of it working.
NOTE: This application is intended to be run using poetry, but can be used with a python env or virtual env, use requirements.txt
to setup up dependencies, and remove poetry run
from the front of all the example commands.
- Run the installation command
curl -sSL https://install.python-poetry.org | python3 -
- Add Poetry to your PATH
export PATH="$HOME/.local/bin:$PATH"
- Reload your shell configuration
- Verify the installation
poetry --version
Once Poetry is installed, you can install dependencies and create the required virtual environment with a single command:
poetry install
NOTE: If you update pyproject.toml directly and want to update the env, just run poetry update
, make sure to keep requirements.txt in sync.
Using poetry add [package]
will update pyproject.toml and install the package into your env.
using the poetry env is just like python, except you prefix your commands with poetry run
, to run the metatemplate application:
poetry run python -m metatemplate --help
poetry run python -m metatemplate -t api -ns api,common::api::cpp ./schemas/yaml/sample.yaml
-t [tpl_dir] is used to specify which specific template directory you would like to use
-ns is a namespace override for autogenerated types. format for usage is [tpl_dir],[namespace]
-nsm allows you to map specific files to specific namespaces and headers with a yaml file. Usage: -nsm [path_to_yaml_file]
-tpl argument for mapping path to template directory. Usage: -tpl [path_to_directory]
The most feature complete input/schema format, fully supported and parsed directly by xsdata before rendering into the output template format.
yaml_mapper.py
defines the supported schema transformations that are used for a yaml based format. The trick is to massage a simple yaml format into the structure xsdata uses. Currently the format is loosely defined as follows.
- The input yaml should be a list of class definitions, each class definition is a yaml dictionary/obj.
- Each class must have a type, one of: struct, enum, variant, alias, which is stored in the key
type
. - Each class must have a name, assigned to the key
name
. - Each class or field may have a description/documentation embedded in a field, either named
help
ordoc
(both are supported) - any field with two supported names will clobber each other if you use both, you can see how in
yaml_mapper.py
This gets us up to type specific content, ignoring that, we have:
- type: struct
name: Class1
doc: A custom class made of fields to use for stuff
[... struct specific]
- type: enum
name: Enum1
[... enum specific]
- type: struct
name: BaseClass
fields:
- type: str
name: private_key
repeat: True
- type: struct
name: SampleStruct
extends: BaseClass
fields:
- type: str
name: public_key
required: False
- type: int
name: a_number
optional: True
doc: A number that might be there
fields
: a list of field defs, defined belowextends|parent
: the name of a class this class extends
type
: the class/type (think class in code speak) of the fieldname
: the name of the field, will define the getter/setter/member namerequired
(optional): if explicitly False, the field is marked optional, * default=Trueoptional
(optional): if explicitly True, the field is marked optional, * default=Falserepeat|repeats
(optional): if explicitly True, fieldrestrictions
(optional): dict (key/values) mapping the restrictions fields like min_occurs/max_occurs, see class Restrictions in xsdata.codegen.models for all fieldshelp|doc
(optional): docs assigned to field
- type: enum
name: SampleEnum
base: uint32
values:
- SingleStringVal
- key: AsDict
value: 10
doc: Special value when it's a dict?
base|base_type
: type of the enum values, only numeric currently supportedvalues
: list of Enum values
Either a string name --or--
key|name
: enum name stringvalue
(optional): currently unused, eventually will support custom valueshelp|doc
(optional): docs assigned to enum option
- type: alias
name: SampleAlias
base: str
restrictions:
min_length: 1
max_length: 10
base|base_type
: type to extend from, usually a native typerestrictions
(optional): dict (key/values) mapping the restrictions fields like min_occurs/max_occurs, see class Restrictions in xsdata.codegen.models for all fields. In the case of Alias, generally used for min/max value constraints or min/max length (strings)
- type: variant
name: SampleVariant
types:
- BaseClass
- str
- type: str
key: SpecialString
help: This is when the value is a string but it's marked as a different choice with a different name...Quite confusing, but required for OMS :)
- UUID
- Datetime
types
: list of variant types
Either a string type name --or--
type
: name of the type associated with the variant choicekey
(optional): a custom key to assign to the type, assigned to the typed accessors, held type checkers, and the Choice enum generated. default =type
help|doc
(optional): variant specific docs
xsdata supports other formats, see their documentation for supported inputs.
- name: [class_name]
namespace: [some_namespace]
includes:
- [some_header_file]
- [some_other_header_file]
- There's a bit of code that finds your git structure root and would install into the "destination" repo, but didn't get finished, so it's not activated. Requires
git
on PATH, so doesn't work inside the current container. - libclang in python would be an interesting way to auto-detect and generate the conversion code between one of the outputs (api/proto/etc) and an externally defined library. Currently the templates depend on assumptions about naming patterns in external libraries, and are not re-usable for others.