This repository contains an OCurrent pipeline for deploying the
various other pipelines we use. When a new commit is pushed to the
live branch of a source repository, it builds a new Docker image
for the project and upgrades the service to that version.
The main configuration is in pipeline.ml. For example, one entry is:
ocurrent, "docker-base-images", [ docker "Dockerfile" ["live", "ocurrent/base-images:live", [`Toxis, "base-images_builder"]]; ];
This says that for the https://github.com/ocurrent/docker-base-images repository:
- We should use Docker to build the project’s
Dockerfile(and report the status on GitHub for each branch and PR).
- For the
livebranch, we should also publish the image on Docker Hub as
ocurrent/base-images:liveand deploy it as the image for the
base-images_builderDocker service on
The pipeline also deploys some MirageOS unikernels, e.g.
mirage, "mirage-www", [ unikernel "Dockerfile" ~target:"hvt" ["EXTRA_FLAGS=--tls=true"] ["master", "www"]; unikernel "Dockerfile" ~target:"xen" ["EXTRA_FLAGS=--tls=true"] ; (* (no deployments) *) ];
See VM-host.md for instructions about setting up a host for unikernels.
There are 3 different flavours of pipelines:
- Tarides - existing Tarides/OCamlLabs pipelines on deploy.ci3.ocamlabs.io.
- OCaml - pipelines for deploying ocaml.org services.
- Toxis - existing Mirage piplines on deploy.ocamllabs.io.
Each pipeline flavour is connected to a different GitHub Application:
- deploy.ci.ocaml.org @ https://github.com/apps/deploy-ci-ocaml-org on branch live-ocaml-org
- deploy.ci3.ocamllabs.io @ https://github.com/apps/deploy-ci3-ocamllabs-io on branch live-ci3
- ci.ocamllabs.io (toxis deployer) @ https://github.com/apps/deploy-ocamllabs-io on branch live-toxis
To test changes to the pipeline, use:
dune exec -- ocurrent-deployer-local --confirm=harmless --submission-service submission.cap \ --github-webhook-secret-file github-secret-file \ --flavour tarides -v \ ocurrent/ocaml-ci
You will need a
submission.cap to access an OCluster build cluster
(you can run one locally fairly easily if needed), along with a
github-secret-file containing a valid GitHub
secret for securing webhooks.
ocurrent/ocaml-ci with the GitHub repository you want to check, or omit it to check all of them.
Unlike the full pipeline, this:
- Only tries to build the deployment branches (not all PRs).
- Doesn’t post the result to Slack.
- Uses anonymous access to get the branch heads.
You can supply
--github-app-id and related options if you want to access GitHub via an app
(this gives a higher rate limit for queries, allows setting the result status and handling GitHub webhooks).
To update a deployment that is managed by ocurrent-deployer (which could be ocurrent-deployer itself):
- Make a PR on that project’s repository targetting its master branch as usual.
- Once it has passed CI/review, a project admin will
git push origin HEAD:liveto deploy it.
- If it works, the PR can be merged to master.
Add a new service
- Deploy the service(s) manually using
docker stack deployfirst.
- Once that’s working, make a PR against the ocurrent-deployer repository adding a rule to keep the services up-to-date.
For the PR:
- Drop the id_rsa.pub key in the ~/.ssh/authorized_keys file on the machine where you want the deployer to deploy the container.
- Add the machine where you want to have the deployments to the
context/metafolder. eg to add
docker --config config/docker context create \ --docker host=ssh://awesome.ocaml.org \ --description="awesome.ocaml.org" \ awesome-ocaml-org
- The hash for the folder inside
context/metais generated with
docker context create <machine_name>.
- Add to
known_hostswith ssh-keyscan of the host where you are deploying the service. eg
ssh-keyscan -H awesome.ocaml.org >> config/ssh/known_hosts