Travis CI Integration with Docker Build Image

[akshmakov]

Travis Docker integration with monolithic docker image for build pipeline

This is intentionally a very verbose and arguably "heavy" approach, there are alternatives we can persue to reduce build time and image size. The main benefit is that build steps and requirements can be updated in one place and the docker image can be used to publish to dockerhub out of the box once you are ready to do so.

1. Use an external build-dependency image Example project: https://hub.docker.com/r/nc5ng/gmt-builder-docker/

• Requires external images hosted on dockerhub containing pre-configured build environment
• Pro: Faster build, less to download and more re-use
• Con: Splits build dependencies into multiple projects, more confusing to use and debug,

2. All commands are written as one-line bash commands in the Dockerfile, it is also possible to split the file into external scripts stored in the repository (Less verbose Dockerfile)

• Pro: Simpler to add more build steps in a bash script
• Con: Less transparent what build steps are done in image, cache misses more likely

3. The alternate development image is not strictly necessary

Documentation was updated to give some guidance on use, build has been tested with travis ci.
Sample Build (if PR doesn't trigger build with new CI file): https://travis-ci.org/nc5ng/gmt/builds/417081935

Fixes #13

[welcome]

💖 Thanks for opening this pull request! 💖

A few things to keep in mind:

• Make sure that your code follows our style. Use the other functions/files as a basis.
• If you need help writing tests, take a look at the existing ones for inspiration. If you don't know where to start, let us know and we'll walk you through it.
• All new features should be documented. It helps to write the comments below for your functions describing the it, all arguments, and return types before writing the code. This will help you think about your code design and results in better code.
• We are grateful that you put in the effort to do this! 🎉

[leouieda]

Hi @akshmakov thanks for putting this together!

From what I understand, Travis will create a new docker container every time that runs the tests. Is that right?

What I had in mind was having a container that had the dependencies and coastlines/borders. Then we could pull it and run our tests inside it. This way, we avoid running apt-get and wget on every build to save time. It seems that running docker build does the same amount of work as our bash scripts were doing.

Is this possible to do?

[leouieda]

By the way, kudos on writing up a whole documentation file for this! Awesome 🥇

[akshmakov]

@leouieda Indeed, in the current configuration it will download packages every time when built on a clean system. Travis CI does not persist the environment between builds so the docker cache is lost every build. On other build systems or your local machine the cache can be preserved between builds. On the other hand even without the cache the base images are much more stable than Travis CI build environment, so while you waste time and bandwidth, at least the image is consistent and repeatable. There are some other, technical benefits that may become more obvious when/if you decide to publish docker based releases, but basically, yes, its a heavy and verbose approach.

In a nutshell, the issue is that somewhere, someone, needs to build and push that build image to dockerhub (or a private registry, whatever) for it to be available globally. Now that someone can replace or change that image without you really having any control. Additionally that external image is now decoupled from your project, so if dependencies change you would need to go update something else t before you can build, it also means maintaining version maps between the build environment and the project version. This is done all the time but I wanted to make sure you understand what this would imply.

For me to proceed with the pre-built image approach I would need you to weigh in on the following choices from the GMT organizations perspective. There are two basic ways to proceed

1. Create A GMT account on dockerhub that gmt organization controls, I can handle getting the build images up on there.

2. Give your OK to use my image from the time being (built from github repo but published on Dockerhub )

Basically its an exercise in infrastructure management. We take what I have done and break it up into separate pieces that are pre-built and fetched at build time.

[akshmakov]

Hopefully some of that was understandable. In any case, I have updated the image to use a pre-built one instead of downloading dependencies every build.

Do you think it would be useful to upload every build to docker repository (dockerhub) so that people can try whatever gmt version they want?

[seisman]

Any updates on this PR?

[leouieda]

@akshmakov sorry for leaving this hanging for so long. I recently realised that using the image kind of defeats the purpose of the CI. It won't save time downloading the packages and data because we have to download the docker image (with these things in there) anyway. And using this on Mac defeats the purpose of testing on Mac since we'd be running things on the Linux image.

That said, having the docker images for recent GMT might be useful for other users. We could add instructions for using your images on the GMT docs or in the install instructions. What do you think?

[seisman]

Closed since we no longer need dockers for travis CI.