| Status | |
| ------------- |-----------|
| Stability | [development]: metrics, traces, logs |
| Distributions | [] |
| Issues | [](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aexporter%2Fkinetica) [](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aexporter%2Fkinetica) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@am-kinetica](https://www.github.com/am-kinetica), [@TylerHelmuth](https://www.github.com/TylerHelmuth) |
[development]: https://github.com/open-telemetry/opentelemetry-collector#development
# Kinetica OpenTelemetry Collector Exporter Plug-In
- [Overview](#overview)
- [Installation](#installation)
- [Documentation](#documentation)
- [Support](#support)
- [Contact Us](#contact-us)
## Overview
## Installation
# Creating the tables
The schema name `otel` is an example. The user may change this to another name but the same name has to be used
in the exporter configuration. The `SQL` scripts follow.
```SQL
----- Logs
CREATE TABLE otel.log
(
log_id VARCHAR (uuid), -- generated
trace_id VARCHAR(32),
span_id VARCHAR(16),
time_unix_nano TIMESTAMP,
observed_time_unix_nano TIMESTAMP,
severity_id TINYINT,
severity_text VARCHAR(8),
body VARCHAR,
flags INT,
PRIMARY KEY (log_id)
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.log_attribute
(
log_id VARCHAR (uuid),
key VARCHAR(256, dict),
string_value VARCHAR(256),
bool_value BOOLEAN,
int_value INT,
double_value DOUBLE,
bytes_value BYTES,
PRIMARY KEY (log_id, key),
SHARD KEY (log_id),
FOREIGN KEY (log_id) REFERENCES otel.log(log_id) AS fk_log
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.log_resource_attribute
(
log_id VARCHAR (uuid), -- generated
key VARCHAR(256, dict),
string_value VARCHAR,
bool_value BOOLEAN,
int_value INT,
double_value DOUBLE,
bytes_value BYTES,
PRIMARY KEY (log_id, key),
SHARD KEY (log_id),
FOREIGN KEY (log_id) REFERENCES otel.log(log_id) AS fk_log_resource
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.log_scope_attribute
(
log_id VARCHAR (uuid), -- generated
scope_name VARCHAR(64, dict),
scope_ver VARCHAR(16, dict),
key VARCHAR(256, dict),
string_value VARCHAR,
bool_value BOOLEAN,
int_value INT,
double_value DOUBLE,
bytes_value BYTES,
PRIMARY KEY (log_id, key),
SHARD KEY (log_id),
FOREIGN KEY (log_id) REFERENCES otel.log(log_id) AS fk_log_scope
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
----- Traces
CREATE TABLE "otel"."trace_span"
(
"id" UUID (primary_key) NOT NULL,
"trace_id" VARCHAR (32) NOT NULL,
"span_id" VARCHAR (16) NOT NULL,
"parent_span_id" VARCHAR (16),
"trace_state" VARCHAR (256),
"name" VARCHAR (256, dict) NOT NULL,
"span_kind" TINYINT (dict),
"start_time_unix_nano" TIMESTAMP NOT NULL,
"end_time_unix_nano" TIMESTAMP NOT NULL,
"dropped_attributes_count" INTEGER,
"dropped_events_count" INTEGER,
"dropped_links_count" INTEGER,
"message" VARCHAR(256),
"status_code" TINYINT (dict)
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."trace_span_attribute"
(
"span_id" UUID (primary_key, shard_key) NOT NULL,
"key" VARCHAR (primary_key, 256, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (span_id) references otel.trace_span(id) as fk_span
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."trace_resource_attribute"
(
span_id VARCHAR (UUID) NOT NULL,
"key" VARCHAR (256, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
PRIMARY KEY (span_id, key),
SHARD KEY (span_id),
FOREIGN KEY (span_id) references otel.trace_span(id) as fk_span_resource
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."trace_scope_attribute"
(
"span_id" UUID (primary_key) NOT NULL,
"name" VARCHAR (256, dict),
"version" VARCHAR (256, dict),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (span_id),
FOREIGN KEY (span_id) references otel.trace_span(id) as fk_span_scope
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."trace_event_attribute"
(
"span_id" UUID (primary_key) NOT NULL,
"event_name" VARCHAR (128, dict) NOT NULL,
"time_unix_nano" TIMESTAMP,
"key" VARCHAR (primary_key, 128) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (span_id),
FOREIGN KEY (span_id) references otel.trace_span(id) as fk_span_event
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."trace_link_attribute"
(
"link_span_id" UUID (primary_key) NOT NULL,
"trace_id" VARCHAR (32),
"span_id" VARCHAR (16),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" TINYINT,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (link_span_id),
FOREIGN KEY (link_span_id) references otel.trace_span(id) as fk_span_link
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
------ METRICS
------ GAUGE
CREATE TABLE otel.metric_gauge
(
gauge_id UUID (primary_key, shard_key) not null,
metric_name varchar(256) not null,
metric_description varchar (256),
metric_unit varchar (256)
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_gauge_datapoint
(
gauge_id UUID (primary_key, shard_key) not null,
id UUID (primary_key) not null,
start_time_unix TIMESTAMP NOT NULL,
time_unix TIMESTAMP NOT NULL,
gauge_value DOUBLE,
flags INT,
FOREIGN KEY (gauge_id) references otel.metric_gauge(gauge_id) as fk_gauge_datapoint
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_gauge_datapoint_attribute"
(
"gauge_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (gauge_id) references otel.metric_gauge(gauge_id) as fk_gauge_datapoint_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_gauge_datapoint_exemplar
(
"gauge_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
time_unix TIMESTAMP NOT NULL,
gauge_value DOUBLE,
"trace_id" VARCHAR (32),
"span_id" VARCHAR (16),
FOREIGN KEY (gauge_id) references otel.metric_gauge(gauge_id) as fk_gauge_datapoint_exemplar
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_gauge_datapoint_exemplar_attribute
(
"gauge_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (gauge_id) references otel.metric_gauge(gauge_id) as fk_gauge_datapoint_exemplar_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_gauge_resource_attribute"
(
"gauge_id" UUID (primary_key) NOT NULL,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (gauge_id),
FOREIGN KEY (gauge_id) references otel.metric_gauge(gauge_id) as fk_gauge_resource_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_gauge_scope_attribute"
(
"gauge_id" UUID (primary_key) NOT NULL,
"name" VARCHAR (256),
"version" VARCHAR (256),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (gauge_id),
FOREIGN KEY (gauge_id) references otel.metric_gauge(gauge_id) as fk_gauge_scope_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
------- SUM
CREATE TABLE otel.metric_sum
(
sum_id UUID (primary_key, shard_key) not null,
metric_name varchar (256) not null,
metric_description varchar (256),
metric_unit varchar (256),
aggregation_temporality INTEGER,
is_monotonic BOOLEAN
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_sum_datapoint
(
sum_id UUID (primary_key, shard_key) not null,
id UUID (primary_key) not null,
start_time_unix TIMESTAMP NOT NULL,
time_unix TIMESTAMP NOT NULL,
sum_value DOUBLE,
flags INT,
FOREIGN KEY (sum_id) references otel.metric_sum(sum_id) as fk_sum_datapoint
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_sum_datapoint_attribute"
(
"sum_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (sum_id) references otel.metric_sum(sum_id) as fk_sum_datapoint_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_sum_datapoint_exemplar
(
"sum_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
time_unix TIMESTAMP NOT NULL,
sum_value DOUBLE,
"trace_id" VARCHAR (32),
"span_id" VARCHAR (16),
FOREIGN KEY (sum_id) references otel.metric_sum(sum_id) as fk_sum_datapoint_exemplar
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_sum_datapoint_exemplar_attribute
(
"sum_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (sum_id) references otel.metric_sum(sum_id) as fk_sum_datapoint_exemplar_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_sum_resource_attribute"
(
"sum_id" UUID (primary_key) NOT NULL,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (sum_id),
FOREIGN KEY (sum_id) references otel.metric_sum(sum_id) as fk_sum_resource_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_sum_scope_attribute"
(
"sum_id" UUID (primary_key) NOT NULL,
"name" VARCHAR (256),
"version" VARCHAR (256),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (sum_id),
FOREIGN KEY (sum_id) references otel.metric_sum(sum_id) as fk_sum_scope_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
------ HISTOGRAM
CREATE TABLE otel.metric_histogram
(
histogram_id UUID (primary_key, shard_key) not null,
metric_name varchar (256) not null,
metric_description varchar (256),
metric_unit varchar (256),
aggregation_temporality int8
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_histogram_datapoint
(
histogram_id UUID (primary_key, shard_key) not null,
id UUID (primary_key) not null,
start_time_unix TIMESTAMP,
time_unix TIMESTAMP NOT NULL,
count LONG,
data_sum DOUBLE,
data_min DOUBLE,
data_max DOUBLE,
flags INT,
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_datapoint
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_histogram_datapoint_bucket_count
(
histogram_id UUID (primary_key, shard_key) not null,
datapoint_id UUID (primary_key) not null,
count_id UUID (primary_key) not null,
count LONG,
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_datapoint_bucket_count
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_histogram_datapoint_explicit_bound
(
histogram_id UUID (primary_key, shard_key) not null,
datapoint_id UUID (primary_key) not null,
bound_id UUID (primary_key) not null,
explicit_bound DOUBLE,
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_datapoint_explicit_bound
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_histogram_datapoint_attribute"
(
"histogram_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_datapoint_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_histogram_datapoint_exemplar
(
"histogram_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
time_unix TIMESTAMP NOT NULL,
histogram_value DOUBLE,
"trace_id" VARCHAR (32),
"span_id" VARCHAR (16),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_datapoint_exemplar
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_histogram_datapoint_exemplar_attribute
(
"histogram_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_datapoint_exemplar_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_histogram_resource_attribute"
(
"histogram_id" UUID (primary_key) NOT NULL,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (histogram_id),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_resource_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_histogram_scope_attribute"
(
"histogram_id" UUID (primary_key) NOT NULL,
"name" VARCHAR (256),
"version" VARCHAR (256),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (histogram_id),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_scope_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
------- EXPONENTIAL HISTOGRAM
CREATE TABLE otel.metric_exp_histogram
(
histogram_id UUID (primary_key, shard_key) not null,
metric_name varchar (256) not null,
metric_description varchar (256),
metric_unit varchar (256),
aggregation_temporality int8
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_exp_histogram_datapoint
(
histogram_id UUID (primary_key, shard_key) not null,
id UUID (primary_key) not null,
start_time_unix TIMESTAMP,
time_unix TIMESTAMP NOT NULL,
count LONG,
data_sum DOUBLE,
scale INTEGER,
zero_count LONG,
buckets_positive_offset INTEGER,
buckets_negative_offset INTEGER,
data_min DOUBLE,
data_max DOUBLE,
flags INT,
FOREIGN KEY (histogram_id) references otel.metric_exp_histogram(histogram_id) as fk_exp_histogram_datapoint
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_exp_histogram_datapoint_bucket_positive_count
(
histogram_id UUID (primary_key, shard_key) not null,
datapoint_id UUID (primary_key) not null,
count_id UUID (primary_key) not null,
count LONG,
FOREIGN KEY (histogram_id) references otel.metric_exp_histogram(histogram_id) as fk_exp_histogram_datapoint_bucket_count
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_exp_histogram_datapoint_bucket_negative_count
(
histogram_id UUID (primary_key, shard_key) not null,
datapoint_id UUID (primary_key) not null,
count_id UUID (primary_key) not null,
count LONG,
FOREIGN KEY (histogram_id) references otel.metric_exp_histogram(histogram_id) as fk_exp_histogram_datapoint_bucket_count
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_exp_histogram_datapoint_attribute"
(
"histogram_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (histogram_id) references otel.metric_exp_histogram(histogram_id) as fk_exp_histogram_datapoint_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_exp_histogram_datapoint_exemplar
(
"histogram_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
time_unix TIMESTAMP NOT NULL,
sum_value DOUBLE,
"trace_id" VARCHAR (32),
"span_id" VARCHAR (16),
FOREIGN KEY (histogram_id) references otel.metric_exp_histogram(histogram_id) as fk_exp_histogram_datapoint_exemplar
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_exp_histogram_datapoint_exemplar_attribute
(
"histogram_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
exemplar_id UUID (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (histogram_id) references otel.metric_exp_histogram(histogram_id) as fk_exp_histogram_datapoint_exemplar_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_exp_histogram_resource_attribute"
(
"histogram_id" UUID (primary_key) NOT NULL,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (histogram_id),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_resource_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_exp_histogram_scope_attribute"
(
"histogram_id" UUID (primary_key) NOT NULL,
"name" VARCHAR (256),
"version" VARCHAR (256),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (histogram_id),
FOREIGN KEY (histogram_id) references otel.metric_histogram(histogram_id) as fk_histogram_scope_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
------- SUMMARY
CREATE TABLE otel.metric_summary
(
summary_id UUID (primary_key, shard_key) not null,
metric_name varchar (256) not null,
metric_description varchar (256),
metric_unit varchar (256)
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_summary_datapoint
(
summary_id UUID (primary_key, shard_key) not null,
id UUID (primary_key) not null,
start_time_unix TIMESTAMP,
time_unix TIMESTAMP NOT NULL,
count LONG,
data_sum DOUBLE,
flags INT,
FOREIGN KEY (summary_id) references otel.metric_summary(summary_id) as fk_summary_datapoint
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_summary_datapoint_attribute"
(
"summary_id" UUID (primary_key, shard_key) NOT NULL,
datapoint_id uuid (primary_key) not null,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
FOREIGN KEY (summary_id) references otel.metric_summary(summary_id) as fk_summary_datapoint_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE otel.metric_summary_datapoint_quantile_values
(
summary_id UUID (primary_key, shard_key) not null,
datapoint_id UUID (primary_key) not null,
quantile_id UUID (primary_key) not null,
quantile DOUBLE,
value DOUBLE,
FOREIGN KEY (summary_id) references otel.metric_summary(summary_id) as fk_summary_datapoint_quantile
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_summary_resource_attribute"
(
"summary_id" UUID (primary_key) NOT NULL,
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (summary_id),
FOREIGN KEY (summary_id) references otel.metric_summary(summary_id) as fk_summary_resource_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
CREATE TABLE "otel"."metric_summary_scope_attribute"
(
"summary_id" UUID (primary_key) NOT NULL,
"name" VARCHAR (256),
"version" VARCHAR (256),
"key" VARCHAR (primary_key, 128, dict) NOT NULL,
"string_value" VARCHAR (256),
"bool_value" BOOLEAN,
"int_value" INTEGER,
"double_value" DOUBLE,
"bytes_value" BLOB (store_only),
SHARD KEY (summary_id),
FOREIGN KEY (summary_id) references otel.metric_summary(summary_id) as fk_summary_scope_attribute
) USING TABLE PROPERTIES (NO_ERROR_IF_EXISTS = TRUE);
```
# Kinetica Opentelemetry Exporter
This exporter could be used to as part of an `Opentelemetry` collector to persist data related to
`logs`, `traces` and `metrics` to the `Kinetica` database.
This component is under `development` status.
## Steps to build the collector binary
- Please refer to for instructions on how to get the latest binary and install locally. This link mentions - "Download the binary for your respective platform under the "Releases" page. If install an official release build, the binary is named ocb, but if you installed by using go install, it will be called builder".
- As this link describes install the Collector Builder as a package; it will be available as `ocb` after installation if installed
from an official release.
- An example config file for building the collector binary is as follows:
```yaml
dist:
name: otelcol-kinetica
description: Otel collector with Kinetica exporter
output_path: /home/kinetica/otelexporter_utils/collector-binary
otelcol_version: 0.78.2
exporters:
- gomod:
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/fileexporter v0.78.0
processors:
- gomod:
go.opentelemetry.io/collector/processor/batchprocessor v0.78.2
receivers:
- gomod:
go.opentelemetry.io/collector/receiver/otlpreceiver v0.78.2
- gomod:
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver v0.78.0
```
- Save the config file as e.g., `config_file.yaml`.
- Once the builder config file has been created the binary can be build using the command `ocb --config config_file.yaml`.
## Configuring the collector
The OTEL collector has a number of components at play.
1. The Receiver - this will receive the telemetry data from one of the sources which has been instrumented to produce logs, traces and metrics.
2. The Processor - the metric data received from the `Receiver`s will be passed on to this component.
3. The Exporter - this component is fed by the processor and is used to persist the metrics data to a data store (kike Kinetica) or some other component for further processing.
A sample configuration for the collector binary is as follows:
```yaml
receivers:
otlp:
protocols:
grpc:
endpoint: localhost:4317
prometheus:
config:
scrape_configs:
- job_name: 'ki_stats'
honor_labels: true
static_configs:
- targets: ['172.31.32.21:9010', '172.31.32.15:9010', '172.31.32.16:9010', '172.31.32.18:9010', '172.31.33.29:9010', '172.31.32.19:9010', '172.31.32.26:9010', '172.31.32.20:9010', '172.31.32.17:9010']
processors:
batch:
exporters:
kinetica:
host: http://localhost:9191/
schema: otel
username: admin
password: password
bypasssslcertcheck: true
logconfigfile: log_config.yaml
service:
pipelines:
traces:
receivers:
- otlp
processors:
- batch
exporters:
- kinetica
metrics:
receivers:
- otlp
- prometheus
processors:
- batch
exporters:
- kinetica
logs:
receivers:
- otlp
processors:
- batch
exporters:
- kinetica
```
The first section of the config file describes the components (receivers,
exporters etc.) with their configurations. The second section describes the
service details which defines the pipeline stages for logs, traces, and metrics.
Save the config file as `collector-config.yaml`.
The command to run the collector binary produced by the steps described above is:
`collector-binary --config collector-config.yaml`
### Configuring the Kinetica Exporter
The following parameters will be necessary to define the connection for the
*Kinetica OpenTelemetry Collector Exporter*.
#### Configuration parameters for the Kinetica Exporter
| Name | Description |
|--|--|
| host | Kinetica host URL |
| schema | Kinetica OpenTelemetry schema, if omitted the default schema will be used |
| username | Kinetica User |
| password | Kinetica Password |
| bypasssslcertcheck | Whether to skip SSL certificate check or not (true or false)|
| logconfigfile | Name of the log config file |
### Configuring Logging
The logging is done using Uber zap package and `lumberjack` for rotating files based on size. Time based rotation is not supported yet. The configuration for `lumberjack` can be found here -
#### Default Log Config file
This is included and will be used in case a user defined config file is not found. The name of the file is `config_log_zap.yaml`.
```yaml
level: 'info'
development: true
disableCaller: false
disableStacktrace: false
encoding: 'console'
encoderConfig:
messageKey: 'msg'
levelKey: 'level'
timeKey: 'ts'
nameKey: 'logger'
callerKey: 'caller'
functionKey: 'function'
stacktraceKey: 'stacktrace'
skipLineEnding: false
lineEnding: "\n"
levelEncoder: 'capital'
timeEncoder: 'iso8601'
durationEncoder: 'string'
callerEncoder: 'full'
nameEncoder: 'full'
consoleSeparator: ' | '
outputPaths:
# Implements loggin to the console
- 'stdout'
# Implements rolling logs using lumberjack logger; config parameters are supplied as
# query params. Here maxSize is 10MB after which the logger rolls over; maximum
# number of backups (maxBackups) kept is 5 and maxAge is 10 days.
# The name of the log file in this case is "logs/kinetica-exporter.log" where the
# "logs" directory is under the current directory on the local machine.
- 'lumberjack://localhost/logs/kinetica-exporter.log?maxSize=10&maxBackups=5&maxAge=10'
errorOutputPaths:
- 'stderr'
- './logs/error_logs'
initialFields:
app: 'kinetica-exporter'
```
## Documentation
- [Full Documentation](https://docs.kinetica.com/7.1/)
- [OpenTelemetry Docs](https://opentelemetry.io/docs/)
- [OpenTelemetry Exporter Registry](https://opentelemetry.io/ecosystem/registry/?component=exporter)
## Support
For support, you can post on
[stackoverflow](https://stackoverflow.com/questions/tagged/kinetica) under the
``kinetica`` tag or
[Slack](https://join.slack.com/t/kinetica-community/shared_invite/zt-1bt9x3mvr-uMKrXlSDXfy3oU~sKi84qg).
## Contact Us
- Ask a question on Slack:
[Slack](https://join.slack.com/t/kinetica-community/shared_invite/zt-1bt9x3mvr-uMKrXlSDXfy3oU~sKi84qg)
- Follow on GitHub:
[Follow @kineticadb](https://github.com/kineticadb)
- Email us:
- Visit: