A python CDK project to deploy IPFS Cluster on ECS Fargate. Pleare refer to the AWS Blog for more details.

For the region contains AZ that does not support One Zone EFS.

Set ONE_ZONE_EFS=False in ipfscluster.env

Getting started with CDK

npm install -g aws-cdk

If it's your first time using CDK follow the instruction in the documentation to setup your environment.

Make sure pythons3 and pip are installed on your computer.

Run pip install -r requirements.txt to install all the current dependencies.

The cdk.json file tells the CDK Toolkit how to execute your app.

This project is set up like a standard Python project. The initialization process also creates a virtualenv within this project, stored under the .venv directory. To create the virtualenv it assumes that there is a python3 (or python for Windows) executable in your path with access to the venv package. If for any reason the automatic creation of the virtualenv fails, you can create the virtualenv manually.

To manually create a virtualenv on MacOS and Linux:

$ python3 -m venv .venv

After the init process completes and the virtualenv is created, you can use the following step to activate your virtualenv.

$ source .venv/bin/activate

If you are a Windows platform, you would activate the virtualenv like this:

% .venv\Scripts\activate.bat

Once the virtualenv is activated, you can install the required dependencies.

$ pip install -r requirements.txt

At this point you can now synthesize the CloudFormation template for this code.

$ cdk synth

To add additional dependencies, for example other CDK libraries, just add them to your requirements.txt file and rerun the pip install -r requirements.txt command.

• cdk ls list all stacks in the app
• cdk synth emits the synthesized CloudFormation template
• cdk deploy deploy this stack to your default AWS account/region
• cdk diff compare deployed stack with current state
• cdk docs open CDK documentation

The app will retrive stack name from context variable. You can reference doc for more details. The default stack name is IpfsClusterFargateStack. Following is a sample command for specifing stack name while deploy stack via CDK CLI.

cdk deploy -c stack_name=$YOUR_IPFS_CLUSTER_NAME

Or you can specify stack name in cdk.json

{
  "context":{
    ...
    "stack_name": "$YOUR_STACK_NAME",
    ...
  }
}

Set ONE_ZONE_EFS to True in ipfscluster.env will deploy EFS one zone file system and access poinbt on each AZ

Set ONE_ZONE_EFS to False in ipfscluster.env will deploy EFS file system cross AZs. EFS access point will be created on each AZ.

Set EFS_REMOVE_ON_DELETE to True in ipfscluster.env will DELETE the EFS file system while destroying the CDK stack. The defulat behavior is RETAIN the EFS file system.

Set ECS_EXEC to True in ipfscluster.env will ENABLE ECS EXEC Command for debugging purpose.

DUE TO SECRET CONCERN, parameter will be stored in Secret Manager and will not showed up on Cloudformation console and only default value will show up in template. However, you need those paramters to inovke IPFS Cluster API. Plase taking a note on those parameter and keep it safe.

1. Download ipfs-cluster-service
2. Execute following command and find id and private_key in identity.json. Find secret in service.json

./ipfs-cluster-service -c /tmp/ipfs init 

Export CDK_DEPLOY_REGION variable can specify region other than default CDK region for the deployment.

export CDK_DEPLOY_REGION=ap-northeast-3; cdk deploy \
--parameters ClusterId=12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC \
--parameters ClusterSecret=221813dc706c2d1baaf0a15a8710e3c5e2072783c49d214243540044f21a7315 \
--parameters ClusterPrivateKey=CAESQGM5s/BsNF06WZ6Kzn4uAnPGsgXo6Ir3hmcW981bo8v57Jj0nPJGTB86m95dygxailhiWgVEB0qf+N8Nd7ozaxM= \
--parameters ClusterCredential=admin:p@ssw0rd

Output

Outputs:
IpfsClusterFargateStack.IpfsClusterEndpoint = dg2xxxxxxxxxx.cloudfront.net
IpfsClusterFargateStack.IpfsGatewayEndpoint = d2xxxxxxxxxxx.cloudfront.net

Download ipfs-cluster-ctl Get $IpfsClusterEndpoint from CDK output.

export REST_API_DNS_ENDPOINT=$IpfsClusterEndpoint
export CLUSTER_SECRET=221813dc706c2d1baaf0a15a8710e3c5e2072783c49d214243540044f21a7315
export CLUSTER_RESTAPI_BASICAUTHCREDENTIALS=admin:p@ssw0rd
./ipfs-cluster-ctl -l /dns/${REST_API_DNS_ENDPOINT}/tcp/443 \
--secret ${CLUSTER_SECRET} \
--basic-auth ${CLUSTER_RESTAPI_BASICAUTHCREDENTIALS} peers ls

Example Output:

12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC | IpfsCluster0 | Sees 2 other peers
  > Addresses:
    - /ip4/10.0.0.71/tcp/9096/p2p/12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC
    - /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWRjwfEtpPmnnjkUbwv4mWtWoUdmKemcqwfLNFGPV3P8PC
  > IPFS: 12D3KooWRpU1e9Q7hgowMiZDjpK4xvh2jXEXRuRBud9XYGetTniK
  ...
12D3KooWH7w2NDcZWxnS885rZgaPz65NAXhv45JkMLdK1x5aJi2C | IpfsCluster1 | Sees 2 other peers
  > Addresses:
    - /ip4/10.0.2.66/tcp/9096/p2p/12D3KooWH7w2NDcZWxnS885rZgaPz65NAXhv45JkMLdK1x5aJi2C
    - /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWH7w2NDcZWxnS885rZgaPz65NAXhv45JkMLdK1x5aJi2C
  > IPFS: 12D3KooWGyUcYasEGvbvPHgoGcfgxnTqT9Syg7dkW9BWWxkcbcpT
  ...
12D3KooWBhEzKxgj5CMSriTXPXtjdpkoXxuHZAJvZ77NAQqNijXW | IpfsCluster2 | Sees 2 other peers
  > Addresses:
    - /ip4/10.0.1.104/tcp/9096/p2p/12D3KooWBhEzKxgj5CMSriTXPXtjdpkoXxuHZAJvZ77NAQqNijXW
    - /ip4/127.0.0.1/tcp/9096/p2p/12D3KooWBhEzKxgj5CMSriTXPXtjdpkoXxuHZAJvZ77NAQqNijXW
  > IPFS: 12D3KooWFFRMoufVPwMoX6JiNu1wJ9yYm971kRYK16af2QLZkdsN
  ...

./ipfs-cluster-ctl -l /dns/${REST_API_DNS_ENDPOINT}/tcp/443 \
--secret ${CLUSTER_SECRET} \
--basic-auth ${CLUSTER_RESTAPI_BASICAUTHCREDENTIALS} add $PATH_TO_FILE

Output

added Qmc9CzkoBMoPGXt78mGcE9SAXcTnvttR8UNNXXXXXXXXXX $FILENAME

curl https://ipfs.io/ipfs/$CID --output $PATH_TO_SAVE

Get $IpfsGatewayEndpoint from CDK output

curl https://$IpfsGatewayEndpoint/ipfs/$CID --output $PATH_TO_SAVE

See CONTRIBUTING for more information.

This library is licensed under the MIT-0 License. See the LICENSE file.