forked from usdot-jpo-ode/jpo-ode
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
3 changed files
with
123 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
|
||
# Getting Started with Docker | ||
The following sections summarize installation and setup of Docker on your machine. | ||
## Windows | ||
- Enable virtualization in your BIOS | ||
- Install [Docker for Windows](https://store.docker.com/editions/community/docker-ce-desktop-windows) | ||
## Mac OSX | ||
- Enable virtualization in your BIOS | ||
- Install [Docker for Mac](## Windows | ||
- Enable visualization in your BIOS | ||
- Install [Docker for Windows](https://store.docker.com/editions/community/docker-ce-desktop-windows) | ||
) | ||
## Linux | ||
- Install [Docker Engine](https://store.docker.com/search?type=edition&offering=community&operating_system=linux) on Docker host machine | ||
- Install [docker-compose](https://docs.docker.com/compose/install/) on docker host and/or local machine | ||
- Create a ```.env``` file in the same directory as ```docker-compose.yml```. | ||
|
||
## Step 1 - Build and deploy the application | ||
Run ```docker-compose up --build -d``` | ||
|
||
This starts the configured containers. | ||
|
||
## Helpful hints | ||
|
||
### Obtaining DOCKER_HOST_IP | ||
1. **Windows**: Run `ifconfig` | ||
2. **OSX**: Look for "inet addr:" under the "en0" network interface | ||
3. **Linux**: Look for "inet addr:" under the "eth0" network interface | ||
|
||
### Resolve Git SSL issues by disabling SSL verification | ||
- `export GIT_SSL_NO_VERIFY=1` | ||
### Docker commands | ||
- Bring UP (deploy and start) all containers | ||
- `docker-compose up -d` | ||
- Bring DOWN all containers | ||
- `docker-compose down` | ||
- Lists the running containers under docker-compose | ||
- `docker-compose ps` | ||
- Start a container | ||
- `docker-compose start <container>` | ||
- Restart a container | ||
- `docker-compose restart <container>` | ||
- Stop a container | ||
- `docker-compose stop <container>` | ||
- Stop all containers | ||
- `docker-compose stop` | ||
|
||
## Troubleshooting | ||
|
||
- Docker builds fail with a `wget: bad address` error | ||
- [Docker fix for SSL issues due to corporate network](https://github.com/usdot-jpo-ode/jpo-ode/wiki/Docker-fix-for-SSL-issues-due-to-corporate-network) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
|
||
[![Docker Pulls](https://img.shields.io/docker/pulls/wurstmeister/kafka.svg)](https://hub.docker.com/r/wurstmeister/kafka/) | ||
[![Docker Stars](https://img.shields.io/docker/stars/wurstmeister/kafka.svg)](https://hub.docker.com/r/wurstmeister/kafka/) | ||
[![](https://badge.imagelayers.io/wurstmeister/kafka:latest.svg)](https://imagelayers.io/?images=wurstmeister/kafka:latest) | ||
|
||
# kafka-docker | ||
|
||
Dockerfile for [Apache Kafka](http:https://kafka.apache.org/) | ||
|
||
The image is available directly from https://registry.hub.docker.com/ | ||
|
||
## Tutorial | ||
|
||
[http:https://wurstmeister.github.io/kafka-docker/](http:https://wurstmeister.github.io/kafka-docker/) | ||
|
||
## Helpful Hints | ||
### Pre-Requisites | ||
|
||
- install [docker](docker.md) | ||
- modify the ```KAFKA_ADVERTISED_HOST_NAME``` in ```docker-compose.yml``` to match your docker host IP (Note: Do not use localhost or 127.0.0.1 as the host ip if you want to run multiple brokers.) | ||
- if you want to customize any Kafka parameters, simply add them as environment variables in ```docker-compose.yml```, e.g. in order to increase the ```message.max.bytes``` parameter set the environment to ```KAFKA_MESSAGE_MAX_BYTES: 2000000```. To turn off automatic topic creation set ```KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'``` | ||
|
||
### Usage | ||
|
||
Start a cluster: | ||
|
||
- ```docker-compose up -d ``` | ||
|
||
### Add more brokers: | ||
|
||
The default [docker-compose.yml](https://github.com/wurstmeister/kafka-docker/blob/master/docker-compose.yml) can launch multiple kafka brokers giving each broker a unique port number. Depending on your use case this might not be desirable. If you need to use specific ports and broker ids, modify the `docker-compose` configuration accordingly. To add more brokers, run: | ||
- ```docker-compose scale kafka=3``` | ||
|
||
The alternative [docker-compose-single-broker.yml](https://github.com/wurstmeister/kafka-docker/blob/master/docker-compose-single-broker.yml) can be used to deploy a single Kafka broker: | ||
- ```docker-compose -f docker-compose-single-broker.yml up``` | ||
|
||
### Broker IDs | ||
|
||
If you don't specify a broker id in your docker-compose file, it will automatically be generated (see [https://issues.apache.org/jira/browse/KAFKA-1070](https://issues.apache.org/jira/browse/KAFKA-1070). This allows scaling up and down. In this case it is recommended to use the ```--no-recreate``` option of docker-compose to ensure that containers are not re-created and thus keep their names and ids. | ||
|
||
|
||
### Automatically create topics | ||
|
||
If you want to have kafka-docker automatically create topics in Kafka during | ||
creation, a ```KAFKA_CREATE_TOPICS``` environment variable can be | ||
added in ```docker-compose.yml```. | ||
|
||
Here is an example snippet from ```docker-compose.yml```: | ||
|
||
environment: | ||
KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1" | ||
|
||
```Topic 1``` will have 1 partition and 3 replicas, ```Topic 2``` will have 1 partition and 1 replica. | ||
|
||
### Advertised hostname | ||
|
||
You can configure the advertised hostname in different ways | ||
|
||
1. explicitly, using ```KAFKA_ADVERTISED_HOST_NAME``` | ||
2. via a command, using ```HOSTNAME_COMMAND```, e.g. ```HOSTNAME_COMMAND: "route -n | awk '/UG[ \t]/{print $$2}'"``` | ||
|
||
When using commands, make sure you review the "Variable Substitution" section in [https://docs.docker.com/compose/compose-file/](https://docs.docker.com/compose/compose-file/) | ||
|
||
If ```KAFKA_ADVERTISED_HOST_NAME``` is specified, it takes presendence over ```HOSTNAME_COMMAND``` | ||
|
||
For AWS deployment, you can use the Metadata service to get the container host's IP: | ||
``` | ||
HOSTNAME_COMMAND=wget -t3 -T2 -qO- http:https://169.254.169.254/latest/meta-data/local-ipv4 | ||
``` | ||
Reference: http:https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html |