Skip to content

Latest commit

 

History

History
81 lines (62 loc) · 2.15 KB

README.md

File metadata and controls

81 lines (62 loc) · 2.15 KB

DataBrew - Blink

OpenSource data streaming & processing engine to build event-driven systems

Project social preview


Build on push to main Latest version (latest semver)

Blink gives your ability to create event driven systems by adopting CDC or enabling integration with third-party systems like AirTable or stripe to stream your data directly to your systems.

Table of Contents

  1. Installation
  2. Running Blink locally
  3. Docs

Getting started

Let's give Blink a try. Check out different options to start Blink on your local machine

Run with Docker on Linux

Currently, we offer only Linux-based arm64 build for docker Create a config with the name blink.yaml and run the docker image

docker run -v ./blink.yaml:/app/blink.yaml usedatabrew/blink start

Install Golang binary

Latest version (latest semver)

go install github.com/usedatabrew/[email protected]

Running Blink locally

To run Blink locally - you have to create a config file that will define streams

service:
  pipeline_id: 1
source:
  driver: playground
  config:
    data_type: market
    publish_interval: 1
    historical_batch: false
  stream_schema:
    - stream: market
      columns:
        - name: company
          nativeConnectorType: String
          databrewType: String
          nullable: false
          pk: false
        - name: currency
          nativeConnectorType: String
          databrewType: String
          nullable: false
          pk: false
processors:
  - driver: sql
    config:
      query: "select * from streams.market where currency = 'USD'"
sink:
  driver: stdout
  config: {}
blink start -c blink-config.yaml