- Archiving artifacts - storing final deliverables
- Promoting artifacts though pipeline and workflow
- Debugging - Store and inspect logs, screenshots and other debug data through UI or CLI
- Logging is disabled for testing. If some tests are failing and you can't find the reason why, you can add something like
func init() {
if strings.HasSuffix(os.Args[0], ".test") { // testing
Init(true)
}
}
- Before committing anything, symlink pre-commit script with
ln -s ../../script/pre-commit .git/hooks/
Artifacts can be stored and accessed on three different layers. This concept makes it easy to use artifacts for different purposes with very simple commands.
Artifact stores:
- Project - one per project
- Workflow - nasted under current workflow
- Job - nasted under current job
You can add verbose logging by the --verbose
or -v
flag.
Use-case - Storing final deliverables
It's an artifact store that is per project and it's possible to simply upload/download files and directories from this level from every job from any workflow and pipeline. Here is an example how you can interect with project level store from any job in your pipeline or from your local development environemnt.
From any jobs running on Semaphore:
artifact push project myapp-v3.tar
artifact pull project myapp-v3.tar
From your development environment: TODO: this is not done yet
sem artifact push project --project payment-api myapp-v3.tar
sem artifact pull project -p payment-api myapp-v3.tar
You can also view and download your artifacts from project page in the UI.
Use-case - Promoting build artifacts accross pipelines and block levels. e.g. promoting from Build and test pipeline into Production deployment pipeline.
New store is created for each new workflow. Here are examples for interacting with this store from any pipeline and any job within workflow.
From any jobs running on Semaphore:
artifact push workflow myapp-v3.tar
artifact pull workflow myapp-v3.tar
From your development environment: TODO: this is not done yet
sem artifact push workflow --workflow <WORKFLOW_ID> myapp-v3.tar
sem artifact pull workflow -w <WORKFLOW_ID> myapp-v3.tar
You can also view and download artifacts from workflow page in the UI.
Use-case - Debugging jobs with easy access to artifacts that job created. e.g. Storing logs, screenshots, core dumps and inspecting them them on the job page.
New store is created for each new job. Here are examples for interacting with this store from job.
From job running on Semaphore:
artifact push job logs/build.log
artifact pull job logs/build.log
From your development environment: TODO: this is not done yet
sem artifact push job --job <JOB_ID> logs/build.log
sem artifact pull job -p <JOB_ID> logs/build.log
You can also view and download artifacts on the job page in the UI.
$HOME/.artifact.yaml or similar, for more, look at Viper.
Project level artifacts default expire time in the same format as Alternative forms and flags #3.
Workflow level artifacts default expire time in the same format as Alternative forms and flags #3.
Job level artifacts default expire time in the same format as Alternative forms and flags #3.
Uploads file or path into /artifacts/jobs/<SEMAPHORE_JOB_ID>/x.zip
.
artifact push job logs/webserver/access.log
pushes file into /artifacts/jobs/<SEMAPHORE_JOB_ID>/log/webserver/access.log
artifact push job myapp/v1/logs/webserver
pushes directory with all sub directories and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/webserver
artifact push job /var/semaphore/webserver/logs/testxyz
pushes directory with all sub directories and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/testxyz
--destination
or-d
sets destination directory or file path
artifact push job x.zip -d y.zip
pushes file into /artifacts/jobs/<SEMAPHORE_JOB_ID>/y.zip
.
Example for directory: artifact push job logs/webserver --destination debuglogs
pushes all sub-dirs and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/debuglogs
.
Example for deeply nested directory as destination: artifact push job logs/webserver --destination path/to/debuglogs
pushes all sub-dirs and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/path/to/debuglogs
.
--job <job-id>
or-j <job-id>
By default command is looking for SEMAPHORE_JOB_ID
env var. If it's not available it fails. If flag --job
is specified it takes precedence over SEMAPHORE_JOB_ID
.
--expire-in 10d
or-e 10d
Expires - deletes the files or directories after amount of time specified.
Supported options are:
never
orNever
means it never expiresNd
for N daysNw
for N weeksNm
for N monthsNy
for N years
If expires flag is not set artifact never expires.
--force
or-f
artifact push job x.zip
if x.zip
exists in the bucket this command should fail. To overwrite file or directory user would need to specify "force" flag.
TODO
- SEMAPHORE_JOB_ID (not required if
--job
flag is specified) - Linux, macOS:
~/.artifact/credentials
- Windows:
dir "%UserProfile%\.artifact\credentials"
Other supported levels include workflow
and project
level. These are variations of the command depending on the level:
File is stored into /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/x.zip
File is stored into /artifacts/projects/<SEMAPHORE_PROJECT_ID>/x.zip
Artifact stored at /artifacts/jobs/<SEMAPHORE_JOB_ID>/x.zip
will be downloaded in current directory as x.zip
.
Example with directory: artifact pull job logs
, if logs is directory logs
it will be created locally in current directory and whole content of logs
from bucket will be downloaded into logs
directory locally.
--destination
or-d
sets destination directory or file path
artifact pull job x.zip -d z.zip
pushes file into z.zip
.
Example for directory: artifact pull job logs --destination debuglogs
pulls all sub-dirs and files into debuglogs
locally.
Example for deeply nested directory as destination: artifact pull job logs --destination path/to/debuglogs
pulls all sub-dirs and files into path/to/debuglogs
in current local directory.
--job <job-id>
or-j <job-id>
By default command is looking for SEMAPHORE_JOB_ID
env var. If it's not available it fails. If flag --job
is specified it takes precedence over SEMAPHORE_JOB_ID
.
- SEMAPHORE_JOB_ID (not required if
--job
flag is specified) - Linux, macOS:
~/.artifact/credentials
- Windows:
dir "%UserProfile%\.artifact\credentials"
Other supported levels include workflow
and project
level. These are variations of the command depending on the level:
File is stored into /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/x.zip
would be restored at current directory as x.zip
.
File is stored into /artifacts/projects/<SEMAPHORE_PROJECT_ID>/x.zip
would be restored at current directory as x.zip
.
Deletes artifact.
artifact yank job x.zip
deletes /artifacts/jobs/<SEMAPHORE_JOB_ID>/x.zip
Example for directory: artifact yank job logs
deletes /artifacts/jobs/<SEMAPHORE_JOB_ID>/logs
directory and all recursively all the content that is in the logs
directory in the bucket.
artifact yank workflow x.zip
deletes /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/x.zip
artifact yank project x.zip
deletes /artifacts/projects/<SEMAPHORE_PROJECT_ID>/x.zip
TODO: this is not done yet
artifact list job
lists root of the job directory /artifacts/jobs/<SEMAPHORE_JOB_ID>/
artifact list workflow
lists root of the job directory /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/
artifact list project
lists root of the job directory /artifacts/projects/<SEMAPHORE_PROJECT_ID>/