Skip to content

port9labs/s3-cli

 
 

Repository files navigation

s3-cli -- Go version of s3cmd

Command line utility frontend to the AWS Go SDK for S3. Inspired by s3cmd and attempts to be a drop-in replacement.

Features

  • Compatible with s3cmd's config file
  • Supports a subset of s3cmd's commands and parameters
    • including put, get, del, ls, sync, cp
    • commands are much smarter (get, put, cp - can move to and from S3)
  • When syncing directories, instead of uploading one file at a time, it uploads many files in parallel resulting in more bandwidth.
  • Uses multipart uploads for large files and uploads each part in parallel. This is accomplished using the s3manager that comes with the SDK
  • More efficent at using CPU and resources on your local machine

Install

go get github.com/koblas/s3-cli

Configuration

s3-cli is compatible with s3cmd's config file, so if you already have that configured, you're all set. Otherwise you can put this in ~/.s3cfg:

[default]
access_key = foo
secret_key = bar

You can also point it to another config file with e.g. $ s3-cli --config /path/to/s3cmd.conf ....

Documentation

In general the commands follow rsync as a guide for command options or the unix command line commands.

cp

Copy files to and from S3

Example:

s3-cli cp /path/to/file s3:https://bucket/key/on/s3
s3-cli cp s3:https://bucket/key/on/s3 /path/to/file
s3-cli cp s3:https://bucket/key/on/s3 s3:https://another-bucket/some/thing

get

Download a file from S3 -- really an alias for cp

put

Upload a file to S3 -- really an alias for cp

del

Deletes an object or a directory on S3.

Example:

s3-cli del [--recursive] s3:https://bucket/key/on/s3/

rm

Alias for del

s3-cli rm [--recursive] s3:https://bucket/key/on/s3/

sync

Sync a local directory to S3

s3-cli sync [--delete-removed] /path/to/folder/ s3:https://bucket/key/on/s3/

mv

Move an object which is already on S3.

Example:

s3-cli mv s3:https://sourcebucket/source/key s3:https://destbucket/dest/key

General Notes about s3cmd commpatability

DONE -

  • s3cmd mb s3:https://BUCKET
  • s3cmd rb s3:https://BUCKET
  • s3cmd ls [s3:https://BUCKET[/PREFIX]]
  • s3cmd la
  • s3cmd put FILE [FILE...] s3:https://BUCKET[/PREFIX]
  • s3cmd get s3:https://BUCKET/OBJECT LOCAL_FILE
  • s3cmd del s3:https://BUCKET/OBJECT
  • s3cmd rm s3:https://BUCKET/OBJECT
  • s3cmd du [s3:https://BUCKET[/PREFIX]]
  • s3cmd cp s3:https://BUCKET1/OBJECT1 s3:https://BUCKET2[/OBJECT2]
  • s3cmd modify s3:https://BUCKET1/OBJECT
  • s3cmd sync LOCAL_DIR s3:https://BUCKET[/PREFIX] or s3:https://BUCKET[/PREFIX] LOCAL_DIR
  • s3cmd info s3:https://BUCKET[/OBJECT]

TODO - for full compatibility (with s3cmd)

  • s3cmd restore s3:https://BUCKET/OBJECT

  • s3cmd mv s3:https://BUCKET1/OBJECT1 s3:https://BUCKET2[/OBJECT2]

  • s3cmd setacl s3:https://BUCKET[/OBJECT]

  • s3cmd setpolicy FILE s3:https://BUCKET

  • s3cmd delpolicy s3:https://BUCKET

  • s3cmd setcors FILE s3:https://BUCKET

  • s3cmd delcors s3:https://BUCKET

  • s3cmd payer s3:https://BUCKET

  • s3cmd multipart s3:https://BUCKET [Id]

  • s3cmd abortmp s3:https://BUCKET/OBJECT Id

  • s3cmd listmp s3:https://BUCKET/OBJECT Id

  • s3cmd accesslog s3:https://BUCKET

  • s3cmd sign STRING-TO-SIGN

  • s3cmd signurl s3:https://BUCKET/OBJECT <expiry_epoch|+expiry_offset>

  • s3cmd fixbucket s3:https://BUCKET[/PREFIX]

  • s3cmd ws-create s3:https://BUCKET

  • s3cmd ws-delete s3:https://BUCKET

  • s3cmd ws-info s3:https://BUCKET

  • s3cmd expire s3:https://BUCKET

  • s3cmd setlifecycle FILE s3:https://BUCKET

  • s3cmd dellifecycle s3:https://BUCKET

  • s3cmd cflist

  • s3cmd cfinfo [cf:https://DIST_ID]

  • s3cmd cfcreate s3:https://BUCKET

  • s3cmd cfdelete cf:https://DIST_ID

  • s3cmd cfmodify cf:https://DIST_ID

  • s3cmd cfinvalinfo cf:https://DIST_ID[/INVAL_ID]

Releases

No releases published

Packages

No packages published

Languages

  • Go 99.8%
  • Makefile 0.2%