Host your own CI/CD server with Drone

Discover Drone, the lightest CI/CD open-source software available. This post will help you get started and covers its basic features.

Continuous Integration and Continuous Delivery are very trendy topics in the DevOps world right now.

There are quite a lot of services and software to build, test and deploy your code, but actually, a few are free and open-source and self-hostable.

The most well-know softwares corresponding to these characteristics are Jenkins and GitLab CI. However, Jenkins has a huge memory footprint since it runs on Tomcat (Java).

As for GitLab CI, it’s very good but requires you to run your own GitLab (which is huge) or to be on gitlab.com. You can run your own runner independently though.

Introducing Drone 🔥

The best alternative right now is Drone. It’s an open-source software written in Go available trough a SaaS in beta, but you can actually selft-host your own very easily.

Drone is made of 2 components available trough docker images: a server and one or more agent(s). All the sources are available on GitHub. The server runs the web service and manages repos, users, secrets, etc. The agents will receive building instructions and will spawn docker containers to run the CI/CD.

You can configure your agents to run multiple builds at the same time. I only have one for my single agent and I think this is enough for light CI/CD.

Drone is incredibly light. It requires less than 10 Mo of RAM to run on my server! Jenkins and GitLab need dozens of time the amount of RAM to run… Just saying.

I guess running Drone with Gitea is the ultimate combo, since both are very light Go softwares. 🚀

Setting up Drone 🛠

See the documentation for more details on the options available.

Let’s look at my setup as an example. I’m using it with GitHub and behind a Nginx reverse-proxy.

You can also use GitLab, Gitea, Gogs, Bitbucket…

Setting up the Docker containers

Here is my docker-compose.yml:

version: "2"

services:
  drone-server:
    container_name: drone_server
    image: drone/drone:0.8
    ports:
      - 127.0.0.1:8080:8000
      - 9000
    volumes:
      - ./data:/var/lib/drone/
    restart: always
    environment:
      - DRONE_OPEN=false
      - DRONE_HOST=https://drone.angristan.xyz
      - DRONE_ADMIN=angristan
      - DRONE_GITHUB=true
      - DRONE_GITHUB_CLIENT=
      - DRONE_GITHUB_SECRET=
      - DRONE_SECRET=

  drone-agent:
    container_name: drone_agent
    image: drone/agent:0.8
    command: agent
    restart: always
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_SERVER=drone-server:9000
      - DRONE_SECRET=

The ./data:/var/lib/drone/ volume is needed to host a SQLite database. Yes, Drone really is light!

You can get DRONE_GITHUB_CLIENT and DRONE_GITHUB_SECRET by creating an OAuth App. DRONE_SECRET is just a long random string that you have to generate on your side.

You should change DRONE_HOST to your Drone URL and DRONE_ADMIN to your GitHub username.

Setting up the Nginx reverse-proxy

My Nginx server block looks like this:

server {
        listen 80;
        listen [::]:80;
        server_name drone.angristan.xyz;
        return 301 https://drone.angristan.xyz$request_uri;

        access_log /dev/null;
        error_log /dev/null;
}
server {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;
        server_name drone.angristan.xyz;

        access_log /var/log/nginx/drone-access.log;
        error_log /var/log/nginx/drone-error.log;

        ssl_protocols TLSv1.2;
        ssl_ecdh_curve X25519:P-521:P-384:P-256;
        ssl_ciphers EECDH+CHACHA20:EECDH+AESGCM:EECDH+AES;
        ssl_prefer_server_ciphers on;

        ssl_certificate /etc/nginx/certs/angristan.xyz.fullchain.pem;
        ssl_certificate_key /etc/nginx/certs/angristan.xyz.key.pem;

        location / {
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_pass http://localhost:8080;
                proxy_redirect off;
                proxy_http_version 1.1;
                proxy_buffering off;
                chunked_transfer_encoding off;
        }
}

I recommend acme.sh to get a Let’s Encrypt certificate.

Setting up Drone

When your Docker containers and Nginx are ready, you can now get to your Drone URL. Right away, Drone will ask for your GitHub account’s access. You will have to use it later in order to login.

Setting up drone for a git repository

You will then be able to enable Drone for your repos.

At first, Drone won’t find a .drone.yml in your repo, so it will do nothing.

Here’s how a basic Drone pipeline works:

Github triggers a hook on the Drone server (upon a push, tag, etc)
The server create a build and sends it to the agent
The agent creates a Docker container based on the image defined in .drone.yml
It clone the GitHub repository
It runs the commands defined in .drone.yml

But you can do much more with the plugins, as we’ll see below.

The output ans status of the agent are streamed and sent to the server so that you can view it on the web UI.

There are quite a lot of examples in the docs to create pipelines.

Here’s one:

pipeline:
  build:
    image: node:latest
    commands:
      - npm install
      - npm run test

Here, we will clone our repo into a container running the node:latest image, and then run npm install and npm run test. Easy!

I’ve also added a Telegram bot to notify me of my builds… But I will keep this for my next post!

You can also check out my GitHub profile and search for some examples!

Using secrets

I have multiple pipelines where I build and push images to the Docker hub. For this I need to store my Docker Hub credentials somewhere.

Using secrets, this:

pipeline:
  docker-build:
    image: plugins/docker
    dockerfile: Dockerfile
    repo: angristan/myimage
    tags: [latest, 1.0]
    username: angristan
    password: my$uperP@$$word

Becomes:

pipeline:
  docker-build:
    image: plugins/docker
    dockerfile: Dockerfile
    repo: angristan/myimage
    tags: [latest, 1.0]
    secrets: [docker_username, docker_password]

And then you can add two secrets through the website:

And that’s all!

Using drone-cli

drone-cli is a Go utility to manage your drone server trough the API.

You can manage everything: users, repos, build, secrets…

Talking of secrets, here’s how I could have added the two secrets above, using docker-cli:

drone secret add -repository angristan/myimage -name docker_password -value my$uperP@$$word

Very convenient.

The plugins

There are quite a lot of plugins to use in pipelines.

I’m currently using the docker plugin to build and publish images to the docker hub. I’m also using the telegram plugin to get build notifications… But I will cover that soon!

You can also get notified via mail, discord, hipchat, slack, matrix, etc; publish to the Docker Hub, NPM, GitHub pages, now, PyPi… There’s some choice!

About the project 👨🏻‍💻

Overall, Drone is a great project and it suits my needs for now. It’s stable and has a great set of features, but a lot of work can still be done.

The web UI is light and clean, but quite… minimalistic.

Some parts of the docs are good, but overall there is a real lack of content. A lot of pages are here but… empty.

I will try to help with that and make a couple of pull requests to the docs. 👍

What really bothers me is that even though the project is well-known and has over 15k stars on GitHub, nearly all the work was made by a single contributor.

Brad has been less active for the past year, so that got me a bit worried. But looking at the issues, I see he is still active and version 0.9 should be out soon.

As of now, there is no alternative that is as light and easy to use. Drone suits my needs and is an amazing software. I look forward to see how it will evolve!

Thanks to @Sir_Boops for their help!