Build a website deployer with OCurrent
This is a tutorial on how to use existing OCurrent plugins to build a continuous deployment service for your website. It will be relatively simple, but the concepts should transfer nicely.
We will learn to:
- Monitor and pull in our site content from Github
- Configure an environment in the pipeline to build the site
- Build and deploy the site using Docker (in reality it will be more of a development type server)
To show the flexibility of this approach the site won’t even be in OCaml, it will be a Gatsby website which uses Javascript and Node. But for good measure, we’ll use cohttp-server-lwt as the web-server to run our static site just to show more flexibility.
The first part of our pipeline is to pull content from Github. OCurrent is in now way limited to Github, but it is the only backend currently supported. So that’s what we’ll use.
Authentication
In order to pull things from Github we’ll have to authenticate in order to use the Github API. You will want to go to your profile Settings > Developer Settings > Personal Access Tokens
. Next you will want to generate a new token with support for access to your public repositories.
Fetching your repository
We’re almost done believe it or not! The next step is to actually use the OCurrent Github plugin (Current_github
) to fetch the head commit of the main branch (ref
).
module Gh = Current_github
module Git = Current_git
module Docker = Current_docker.Default
let fetch ~github ~repo () =
let head = Gh.Api.head_commit github repo in
let t = map Gh.Api.Commit.id head in
Git.fetch t
The ~github
parameter is our Api value, we won’t have to generate this as you will see later. And ~repo
is our repository which will eventually be in the form username/repo
which the user can specify from the command-line.
We first get the head of the repository using the Api
’s exposed functionality and pull out the unique id using the Current.map
function. Now we use the Git
plugin to actually go get the specific head commit.
If we look at the plugin code we can see that it defines a new component. This is what we see in the generated dependency graph. Not only that, but it is being monitored which allows OCurrent to propagate new values when the head commit changes (like if a new commit is pushed).
A Note on Caching
One of the big benefits of OCurrent pipelines is a smart use of caching. Luckily a lot of the plugins give you this out of the box. For example, the Git.fetch
function is backed by the fetch cache. In the next tutorial on writing plugins we’ll look at this more deeply, but you can see that the functor to a build a cache is relatively simple.
# #require "current.cache"
# #show Current_cache.S.BUILDER
module type BUILDER =
sig
type t
val id : string
module Key : Current_cache.S.WITH_DIGEST
module Value : Current_cache.S.WITH_MARSHAL
val build : t -> Current.Job.t -> Key.t -> Value.t Current.or_error Lwt.t
val pp : Key.t Fmt.t
val auto_cancel : bool
end
Building with Docker
In order to build and run are Gatsby site we’ll need two docker images:
- A build image in order to run
gatsby build
– this will need NodeJS and yarn installed. We will usenode:12-buster
for this. - A server image with OCaml and opam installed. We will use
ocaml/opam:alpine-ocaml-4.11
for this, which is pushed to docker hub using an OCurrent pipeline!
The build process is quite simple:
- Pull the docker images
- Write a dockerfile in OCaml
- Build the Dockerfile using the git repository we got earlier
- Run the image we built
Thankfully, we can use the Docker plugin for all of this.
let build ~src () =
let open Dockerfile in
let dockerfile =
let+ builder = Docker.pull ~schedule gatsby_builder
and+ server = Docker.pull ~schedule gatsby_server in
from ~alias:"build" (Docker.Image.hash builder)
@@ workdir "/app"
@@ run "yarn global add gatsby-cli"
@@ copy ~src:[ "package.json" ] ~dst:"." ()
@@ run "yarn"
@@ copy ~src:[ "." ] ~dst:"." ()
@@ run "gatsby build"
@@ from (Docker.Image.hash server)
@@ run "sudo apk update && sudo apk add m4"
@@ run "opam install cohttp-lwt-unix"
@@ copy ~from:"build" ~src:[ "/app/public" ] ~dst:"/pub" ()
@@ workdir "/pub" @@ expose_port 8000
@@ cmd "opam exec -- cohttp-server-lwt -p 8000 -s 0.0.0.0"
|> fun d -> `Contents d
in
Docker.build ~pull:false ~dockerfile (`Git src)
We’re using the Dockerfile eDSL for writing Dockerfiles in OCaml which almost reads like a normal file. Using the Docker plugin’s pull
function we get both image and “expose” their values using the let+
syntax. With Docker.Image.hash
we use the hash version of the docker image which will be more reproducible than simply the name of the image.
After that it should be mostly straightforward:
- In the build image we copy in the
package.json
to install the write dependencies, then we copy in the main site and rungatsby build
which outputs the website in/app/public
. - In the server image we install
cohttp-lwt-unix
to get the server and the copy the site from the build into/pub
. We expose port8000
and run the server on that port with host0.0.0.0
. This will allows us to see the server running atlocalhost:8000
.
One aspect to make note of is the schedule
parameter. We build this using the Current_cache
module.
let schedule = Current_cache.Schedule.v ~valid_for:(Duration.of_day 7) ()
let gatsby_builder = "node:12-buster"
let gatsby_server = "ocaml/opam:alpine-ocaml-4.11"
This tells our pipeline how often we should re-pull our Docker images.
The Final Pipeline
let pipeline ~github ~repo () =
let src = fetch ~github ~repo () in
let image = build ~src () in
Docker.run image ~run_args:[ "-p=8000:8000" ] ~args:[]
Our pipeline reuses the fetch
and build
we have already looked at. We pull that altogether in our pipeline
function and run the Docker image we produced (with the additional port mapping parameter).
The last step is to wrap this pipeline up in the Current.Engine
which we can run in a thread alongside a website for viewing our pipeline in action. The website just needs a route list
which can be automatically generated from our engine
value.
We then run this inside the main lwt
thread. You could also use this logging function if you want more information printed to the console.
let main config mode github repo =
let engine = Current.Engine.create ~config (pipeline ~github ~repo) in
let routes = Current_web.routes engine in
let site =
Current_web.Site.(v ~has_role:allow_all) ~name:"gatsby_deployer" routes
in
Lwt_main.run
(Lwt.choose [ Current.Engine.thread engine; Current_web.run ~mode site ])
Wrapping it up as a CLI tool
The final stage is to bring our pipeline together and use Cmdliner to produce a useful CLI tool.
(* Command-line parsing *)
open Cmdliner
let repo =
Arg.required
@@ Arg.pos 0 (Arg.some Gh.Repo_id.cmdliner) None
@@ Arg.info ~doc:"The GitHub repository (owner/name) to monitor." ~docv:"REPO"
[]
let cmd =
let doc = "Monitor a GitHub repository containing a Gatsby site." in
( Term.(
const main $ Current.Config.cmdliner $ Current_web.cmdliner
$ Gh.Api.cmdliner $ repo),
Term.info "gatsby_deployer" ~doc )
let () = Term.(exit @@ eval cmd)
We can use the predefined arguments that the various Current
module provide which add flags like port
mapping. The only one we do provide is the repo
parameter to specify which repository on Github we want to pull. We will also need to provide the authentication token from the beginning by pasting it into a file (here .token
).
deployer --github-token-file=./.token patricoferris/gatsby-starter-default
And that’s it! After running your pipeline go to localhost:8080
to see your pipeline in action. Once it get’s to the run
component you should be able to go to localhost:8000
and see your Gatsby site built and deployed on a Cohttp server!