Kubling Data Source Virtualization (or DBVirt) is a virtual-schemafull database engine that delegates the data fetch and feeding to custom Javascript modules, meaning better flexibility by removing dependencies on vendor-provided connectors.
DBVirt follows CI/CD and DevOps principles, that is, the engine loads definition and configurations dynamically from a released bundle file which, in turn, can define other dependencies as JavaScript modules.
While working on a platform for containerized workloads, we realized that the proliferation of tools would bring a lot of complexity to our platform's model
if we wanted to support them all.
One of the first challenges we encountered is that those tools name things differently, even though most of them are based originally in Kubernetes.
That lead us to start designing an abstraction layer which also involves a minimal definition of certain common concepts, like DEPLOYMENT or STORAGE VOLUME.
Conclusion was interesting: we found out that the intersection of definitions from multiple tools is a set a common properties that, fortunately, represent
the dynamic part of the workload scheduling process and resource creation, whereas the tool-specific parts were mostly associated to properties defined in provisioning scripts
which are occasionally changed.
That was good news because minimizing the definition of DEPLOYMENT would imply the loss of flexibility and could compromise the adaptability when providing services
to teams that require capacity for running workloads.
Kubling DBVirt then acts as a fully-consistent CRUD translation layer, allowing to define "simplified" entities whose meaning should not be associated to any specific implementation, with the
ability to deal with any tool's complexity using custom JavaScript code.
Besides, those minimized entities could be the result of a combination of other entities, that is, when querying DEPLOYMENTs the result we see is an aggregation of deployments
fetched from, for example, Kubernetes on-prem, AKS and Docker containers running in VMs.
| Directory | Goal |
|---|---|
| minimal | A really simple example in which we create a Virtual Database with a single virtual entity. |
| RBAC | Similar to the minimal example but with a really small custom authentication/authorization script. |
| initializer/scheduler | Explores how to use initialization script and scheduler |
| documents | Learn how to deal with complex nested documents, and how to decide whether DBVirt can help. |
| endpoints | In this sample we will expose some endpoints to chain operations (actions) and to query data. |
| azure | Interacts with Azure API using a generated Client based on OpenApi Spec. |
| appmodel | This is the most advanced example in which we build a Data Model for a company that has multiple applications and wants to create provisioning workflows and abstract developers from the underlying system's complexity. |
| hibernate | A really simple example that shows how to start with DBVirt and JPA/Hibernate in a SpringBoot context. |
| Directory/File | |
|---|---|
| descriptor | Contains the main bundle that will be loaded during engine initialization. |
| modules | Contains the JavaScript modules the descriptor specifies |
| app-config.yaml | The only non-dynamic configuration file that the Engine reads during initialization which specifies some Engine configurations and, more important, VDB descriptor bundle location |
Yaml files are always parsed as templates, in this sample project we will use basic expressions delimited by {{ ··· }} that will be mostly replaced by environment variables and loaded variables injected in the template context.
More advanced use cases will be added as this project grows.
Once your instance is running and configured, the easiest way to interact with it is by connecting using PostgreSQL
client since, by default, DBVirt listens to connections using this protocol on port 35432.
Just run psql -h <IP/HOSTNAME> -p 35432 -U <USERNAME> -d <VDB>
By default, DBVirt does not print out any internal log message to the console, leaving it clean for scripts.
In case you need to add debug messages, just print("My debug message"); from any script and get container's logs printed in the stdout.
kdv is released as a native executable (still in progress) and as a OCI image published in DockerHub.
Usage:
docker run --rm -v [path/of/your/project]:[path/in/container] kubling/dbvirt-cli:latest [command] [subcommand] [params] [options]
Example:
docker run --rm -v /root/dbvirt-samples/:/dbvirt-samples/ kubling/dbvirt-cli:latest bundle genmod /dbvirt-samples/azure/modules/delegate -o /dbvirt-samples/azure/modules/delegate/azure-module-bundle.zip
For more information about available commands please see this doc.
DBeaver is our first option when testing things locally. Please follow this steps to configure it in your environment.