- Protocol Buffers
protoc: required for compilationprotobuf-devel(or equivalent): required to import common message types
- Protocol Buffers
protoc \ --go_out=. \ --go_opt=paths=source_relative \ --go-grpc_out=. \ --go-grpc_opt=paths=source_relative \ proto/*.proto - Server (Node)
go build -ldflags="-s -w" -o build/server cmd/server/main.go - Client
go build -ldflags="-s -w" -o build/client cmd/client/main.go
Local:
- Configure
config.jsonand run
python deploy/local.py- Start by following the instructions
Docker Compose:
- Configure
config.jsonand run
python deploy/compose.py- Start using
docker compose up --buildRequirements:
terraform: installation instructions- AWS CLI: installation instructions
python: run automated script
Getting started:
- Configure variables in
config.jsonas desired - Configure AWS CLI in
$HOME/.aws/credentials - Download
labsuser.pemand change the key's permission:chmod 400 labsuser.pem
- Make
aws/mq.sh,aws/client.shandaws/node.shexecutable:chmod +x aws/mq.shchmod +x aws/client.shchmod +x aws/node.sh
Run:
python aws/deploy.pyTo delete from AWS run terraform destroy in aws/ folder
- To get the web client IP, run:
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' `docker ps -f "ancestor=distributed-pagerank-client" | awk 'FNR==2{ print $1 }'`├── aws - AWS Deploy Configuration files
│ ├── client.sh - Deploy Web client script
│ ├── mq.sh - Deploy RabbitMQ script
│ ├── node.sh - Deploy node script
│ ├── rabbitmq.repo - RabbitMQ repo (used by mq.sh)
│ ├── dp.tf - Terraform specs
│ └── variables.tf - Terraform variables
├── config.json - Configuration for local and docker compose deploy
├── deploy - Deploy Scripts for Docker Compose, Local and AWS
│ ├── aws.py - AWS deploy script (based on variables.tf)
│ ├── compose.py - Generate compose.yaml (based on config.json)
│ ├── local.py - Local deploy instructions (based on config.json)
│ ├── Dockerfile.client - Dockerfile for the web client
│ └── Dockerfile.server - Dockerfile for the node
├── go.mod - Go dependencies
├── go.sum - Go dependencies
├── cmd - Entrypoints
│ ├── client
│ │ └── main.go - Web Client entrypoint
│ └── server
│ └── main.go - Node entrypoint
├── pkg - Code logic
│ ├── graph - Graph logic
│ │ ├── graph.go - Graph loading and random generation
│ │ └── pagerank.go - PageRank implementation (single node)
│ ├── node - gRPC and node logic
│ │ ├── api.go - gRPC interaction between client and master
│ │ ├── server.go - gRPC interaction between nodes
│ │ ├── models.go - Node models
│ │ ├── master.go - Master node logic
│ │ └── worker.go - Worker node logic
│ └── utils - Utility functions
│ ├── env.go - Environment variables loading
│ ├── logs.go - Custom Logging
│ ├── queue.go - Useful queue functions
│ └── utils.go - General useful functions
├── proto - Protocol Buffers
│ ├── common.proto - Common messages
│ ├── api.proto - API services (web client - master)
│ ├── node.proto - Node services (node - node)
│ └── jobs.proto - Queue Messages
├── public - Files used by web client
│ ├── index.html - Main web page
│ └── tmpl.html - Templates page
└── README.md - This file