Skip to content

Commit

Permalink
Clarifies storage versions supported vs tested (openzipkin#2623)
Browse files Browse the repository at this point in the history
  • Loading branch information
adriancole authored Jun 4, 2019
1 parent dfe5856 commit e88be8d
Show file tree
Hide file tree
Showing 5 changed files with 44 additions and 26 deletions.
50 changes: 33 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,17 +80,23 @@ is for testing, for example starting a server on your laptop without any
database needed.

### Cassandra
The [Cassandra](zipkin-storage/cassandra) component is tested against
Cassandra 3.11.3+. It stores spans using UDTs, such that they appear like
the v2 Zipkin model in cqlsh. It is designed for scale. For example, it
uses a combination of SASI and manually implemented indexes to make
querying larger data more performant.
The [Cassandra](zipkin-storage/cassandra) component uses Cassandra
3.11.3+ features, but is tested against the latest patch of Cassandra 3.11.

Note: This store requires a [spark job](https://github.com/apache/incubator-zipkin-dependencies) to aggregate dependency links.
This is the second generation of our Cassandra schema. It stores spans
using UDTs, such that they appear like Zipkin v2 json in cqlsh. It is
designed for scale, and uses a combination of SASI and manually
implemented indexes to make querying larger data more performant.

Note: This store requires a [job to aggregate](https://github.com/apache/incubator-zipkin-dependencies) dependency links.

### Elasticsearch
The [Elasticsearch](zipkin-storage/elasticsearch) component is tested against Elasticsearch 2-6.x.
It stores spans as json and has been designed for larger scale.
The [Elasticsearch](zipkin-storage/elasticsearch) component uses
Elasticsearch 5+ features, but is tested against Elasticsearch 6-7.x.

It stores spans as Zipkin v2 json so that integration with other tools is
straightforward. To help with scale, this uses a combination of custom
and manually implemented indexing.

Note: This store requires a [spark job](https://github.com/apache/incubator-zipkin-dependencies) to aggregate dependency links.

Expand Down Expand Up @@ -120,20 +126,30 @@ data layouts based on Zipkin's V1 Thrift model, as opposed to the
simpler v2 data model currently used.

#### MySQL
The [MySQL v1](zipkin-storage/mysql-v1) component currently is only
tested with MySQL 5.6-7. It is designed to be easy to understand, and
get started with. For example, it deconstructs spans into columns, so
The [MySQL v1](zipkin-storage/mysql-v1) component uses MySQL 5.6+
features, but is tested against MariaDB 10.3.

The schema was designed to be easy to understand and get started with;
it was not designed for performance. Ex spans fields are columns, so
you can perform ad-hoc queries using SQL. However, this component has
[known performance issues](https://github.com/apache/incubator-zipkin/issues/1233): queries will eventually take seconds to return
if you put a lot of data into it.

This store does not require a [job to aggregate](https://github.com/apache/incubator-zipkin-dependencies) dependency links.
However, running the job will improve performance of dependencies
queries.

#### Cassandra
The [Cassandra v1](zipkin-storage/cassandra-v1) component is tested
against Cassandra 2.2+. It stores spans as opaque thrifts which means
you can't read them in cqlsh. However, it is designed for scale. For
example, it has manually implemented indexes to make querying larger
data more performant. This store requires a [spark job](https://github.com/apache/incubator-zipkin-dependencies) to aggregate
dependency links.
The [Cassandra v1](zipkin-storage/cassandra-v1) component uses Cassandra
2.2+ features, but is tested against the latest patch of Cassandra 3.11.

The CQL was written in 2015, based on the original Cassandra schema from
Twitter, and since been extended. Spans are stored as opaque thrifts,
which means you cannot query fields in cqlsh. The schema was designed
for scale, including manually implemented indexes to make querying
larger data more performant.

Note: This store requires a [job to aggregate](https://github.com/apache/incubator-zipkin-dependencies) dependency links.

## Running the server from source
The [Zipkin server](zipkin-server) receives spans via HTTP POST and respond to queries
Expand Down
7 changes: 4 additions & 3 deletions zipkin-storage/cassandra-v1/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# storage-cassandra

This is a CQL-based Cassandra 2.2+ storage component, built upon the [Zipkin v1 thrift model](https://github.com/apache/incubator-zipkin-api/tree/master/thrift).
This is a CQL-based Cassandra storage component, built upon the [Zipkin v1 thrift model](https://github.com/apache/incubator-zipkin-api/tree/master/thrift).
This uses Cassandra 2.2+ features, but is tested against the latest patch of Cassandra 3.11.

`CassandraSpanStore.getDependencies()` returns pre-aggregated dependency links (ex via [zipkin-dependencies](https://github.com/apache/incubator-zipkin-dependencies)).

The implementation uses the [Datastax Java Driver 3.1.x](https://github.com/datastax/java-driver).
The implementation uses the [Datastax Java Driver 3.x](https://github.com/datastax/java-driver).

`zipkin2.storage.cassandra.v1.CassandraStorage.Builder` includes defaults that will
operate against a local Cassandra installation.
Expand All @@ -19,7 +20,7 @@ See [Logging Query Latencies](https://docs.datastax.com/en/developer/java-driver/
This module conditionally runs integration tests against a local Cassandra instance.

Tests are configured to automatically access Cassandra started with its defaults.
To ensure tests execute, download a Cassandra 2.2-3.4 distribution, extract it, and run `bin/cassandra`.
To ensure tests execute, download a Cassandra 2.2+ distribution, extract it, and run `bin/cassandra`.

If you run tests via Maven or otherwise when Cassandra is not running,
you'll notice tests are silently skipped.
Expand Down
5 changes: 3 additions & 2 deletions zipkin-storage/cassandra/README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# storage-cassandra

This is a CQL-based Cassandra 3.9+ storage component, built upon the [Zipkin v2 api and model](https://zipkin.apache.org/zipkin-api/#/default/post_spans).
This is a CQL-based Cassandra storage component, built upon the [Zipkin v2 api and model](https://zipkin.apache.org/zipkin-api/#/default/post_spans).
This uses Cassandra 3.11.3+ features, but is tested against the latest patch of Cassandra 3.11.

`CassandraSpanStore.getDependencies()` returns pre-aggregated dependency links (ex via [zipkin-dependencies](https://github.com/apache/incubator-zipkin-dependencies)).

The implementation uses the [Datastax Java Driver 3.1.x](https://github.com/datastax/java-driver).
The implementation uses the [Datastax Java Driver 3.x](https://github.com/datastax/java-driver).

`zipkin2.storage.cassandra.CassandraStorage.Builder` includes defaults that will operate against a local Cassandra installation.

Expand Down
4 changes: 2 additions & 2 deletions zipkin-storage/elasticsearch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

This is is a plugin to the Elasticsearch storage component, which uses
HTTP by way of [OkHttp 3](https://github.com/square/okttp) and
[Moshi](https://github.com/square/moshi). This currently supports 5.x,
6.x and 7.x version families.
[Moshi](https://github.com/square/moshi). This uses Elasticsearch 5+
features, but is tested against Elasticsearch 6-7.x.

## Multiple hosts
Most users will supply a DNS name that's mapped to multiple A or AAAA
Expand Down
4 changes: 2 additions & 2 deletions zipkin-storage/mysql-v1/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# storage-mysql-v1

This MySQL (Legacy) storage component includes a blocking `SpanStore` and span consumer function.
`SpanStore.getDependencies()` aggregates dependency links on-demand.

The implementation uses JOOQ to generate MySQL SQL commands. It is only tested on MySQL 5.6-5.7.
The implementation uses JOOQ to generate MySQL SQL commands. MySQL 5.6+
features are used, but tests run against MariaDB 10.3.

See the [schema DDL](src/main/resources/mysql.sql).

Expand Down

0 comments on commit e88be8d

Please sign in to comment.