I’ve been toying around with the idea of using containers for running ci tests, primarily to have a quicker feedback loop; instead of setups that involve VMs etc. They are ideal for getting an environment up and running quickly and cheap to throw away too.
A project which I’m spending some time lately go-ceph, which provides Go bindings for ceph/rados, kind of ideally fit the bill for using this, since testing this project locally, usually needed something like a VM running a ceph cluster, or a locally running ceph. Though both of the above aren’t that hard, and there are projects around which kind of ease the process (Vagrant ceph for eg.), it still requires a somewhat longer setup time if you’re looking for a bringup teardown sort of environment. Writing dockerfiles & scripts for this helped me appreciate the best practices for writing dockerfiles a bit better.
Ideally the workflow expected would be that you would have
a docker container with the necessary setup & dependencies already
installed, and the docker container could ultimately be run with
something as simple as a
go test to test the latest code. Also since
building a docker container for every run may not be what want; the
idea would be to volume mount the current code tree as a volume, so
that a simple docker run would do the job of a ci tester/builder etc
(something like a local travis)
Dockerfiles & ENTRYPOINT hack
Docker containers are well suited for single processes, which can be
set as default by the
CMD or the
ENTRYPOINT directives. (There are
differences between the two, which I’ll not be getting into for
now). However this presented a problem since even the most basic ceph cluster required atleast 4 daemon processes to be run. Ways of solving this include:
- Creating multiple docker containers for each process and linking them Cleanest method. This is what ceph-docker actually uses and the recommended way if you want to run a ceph cluster in containers.
- Manage multiple processes in a docker using a process manager such as supervisor, official docker docs
A third alternative is to run a shell script as the entrypoint; finishing off the script with an
exec, note that this method will fail when the entrypoint is overridden. However since all I wanted was a minimally working ceph cluster; this hack was used to run a shell script that basically started all the necessary processes and finished off with an
exec call running
go test -v for the package. This is how the dockerfile looked ultimately
FROM golang:1.3-wheezy MAINTAINER Abhishek Lekshmanan "email@example.com" ENV CEPH_VERSION giant RUN echo deb http://ceph.com/debian-$CEPH_VERSION/ wheezy main | tee /etc/apt/sources.list.d/ceph-$CEPH_VERSION.list # Running wget with no certificate checks, alternatively ssl-cert package should be installed RUN wget --no-check-certificate -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | apt-key add - &&\ apt-get update && \ apt-get install -y ceph \ librados-dev librbd-dev libcephfs-dev VOLUME /go/src/github.com/noahdesu/go-ceph COPY ./ci/entrypoint.sh /tmp/entrypoint.sh ENTRYPOINT ["/tmp/entrypoint.sh", "/tmp/micro-ceph"]
For those interested in ceph, the entrypoint script was a modification
of Loic’s micro-osd script, with the only addition being getting the go dependencies and finishing of with an
exec call of
go test. For the gory details refer to the actuall pull request submitted to the upstream project.
Though this particular case needed a little bit of a tweak to run tests in containers, in a general case it is far easier to run local ci like tests even covering multiple Go versions with other dependencies etc easily in a docker.