CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

Setting this up is relatively straightforward, but there are a lot of steps. This isn’t really a primer on GitHub Actions or CI; it is a direct setup guild for getting continuous integration and deployment running with the Hello application, which is a very simple Phoenix 1.6 application that uses PostgreSQL.

If you don’t have Phoenix running in Docker on a Droplet, you may want to start here.

CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean
Contents
Setup up CI
Create the workflow on GitHub
Code
Set up CD
Publish Docker Container to Docker Hub
Create an Access Token on Docker Hub
Create Encrypted Secrets on GitHub
Publish a new image with CD
Deploy new releases to Digital Ocean
Add an SSH Key for the job
Add secrets to GitHub
Make a small change to Hello
VS Code Extension
Verify CI/CD pipeline
Code
Conclusion
References

Setup up CI

Create the workflow on GitHub

First, log in to GitHub, navigate to the repository, click the Actions Tab, and then click the New workflow button. You can then search for a workflow. You should see the “Elixir” workflow. Click the Configure button and you will be presented with the following screen.

The Elixir Workflow Action

Click the Configure button and you will be presented with the following screen.

The CD workflow

All this screen does is present you with a YAML file in the ./github/workflows/ folder of the project. This file will be committed to the repository when you click the Start commit button. You don’t have to edit it here; you are free to make it yourself in the repository and push it up if you wish.

We want to add a PostgreSQL database and change the versions of Elixir and OTP to match what we are using on Digital Ocean. Here’s the completed file.

name: Elixir CI

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:

    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

Take note of the following.

The Postgres image was added as a service. Now we have a database available for our tests.
The Postgres image matches what we are using in the docker-compose.yml and deploying on Digital Ocean.
The elixir-version and otp-version were also changed to match.

That’s pretty much it. Now, every time something is committed to the main branch and action will run our project’s tests.

Code

The current state of the Hello project is here.

GitHub - code-lever/phx_hello at cont-integration

A Phoenix 1.6.12 Hello Application for using in testing Digital Ocean deployments - GitHub - code-lever/phx_hello at cont-integration

github.com

GitHub - code-lever/phx_hello at cont-integration

Set up CD

Continuous Deployment is configured as a two-step process that will only run if the CI step passes, i.e., the project builds and the tests pass. This way, we don’t deploy failing builds. The first step is to publish the new container to Docker Hub. The second is to deploy it on our droplet on Digital Ocean.

Publish Docker Container to Docker Hub

To accomplish this, we will use the community-created Publish Docker action. You can find it here:

Publish Docker - GitHub Marketplace

This Action for Docker uses the Git branch as the Docker tag for building and pushing the container. Hereby the master-branch is published as the latest-tag. name is the name of the image you would like to push username the login username for the registry password the authentication token [preferred] or login password for the registry.

github.com

Update the workflow and add a new job. The workflow is in .github/workflows/elixir.yml. It should now look like this.

name: Elixir CI/CD

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

  publish:
    name: Publish to Docker Hub
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/checkout@v3
    - name: Publish to DockerHub
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: codelever/hello_app:${{ github.sha }}
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

Two things to note:

needs: build ensures that this action only runs if the CI build step we set up previously succeeds. This prevents us from deploying a build that fails CI.
github.sha is used for the image tag instead of a version number. This way we don’t have to update the project version number every time we commit and push to main.

Create an Access Token on Docker Hub

Follow the instructions to create an Access Token for the CD action to log in to Docker Hub. This is much better than using your user password. It can be revoked at any time and the access the CD service has to the Docker Hub repositories is tracked separately from your user account access.

Manage access tokens

Estimated reading time: 3 minutes Docker Hub lets you create personal access tokens as alternatives to your password. You can use tokens to access Hub images from the Docker CLI. Using personal access tokens provides some advantages over a password: You can investigate the last usage of the access token and disable or delete it if you find any suspicious activity.

docs.docker.com

Create Encrypted Secrets on GitHub

Follow the instructions for creating and storing encrypted secrets for the DOCKER_USERNAME and DOCKER_PASSWORD. These are the credentials for the CD action to access Docker Hub. Make sure to use the Access Token created in the previous step instead of your actual Docker Hub password.

Encrypted secrets - GitHub Docs

Secrets are encrypted environment variables that you create in an organization, repository, or repository environment. The secrets that you create are available to use in GitHub Actions workflows. GitHub uses a libsodium sealed box to help ensure that secrets are encrypted before they reach GitHub and remain encrypted until you use them in a workflow.

docs.github.com

You should end up with something like this in the GitHub UI for your project.

Docker secrets added to GitHub

Publish a new image with CD

Commit the updated workflow with the publish job, and push to main.

Now log in to GitHub and look at the actions. You should see that the workflow has completed both jobs.

Elixir CI/CD workflow completed successfully

We should also have a new image tagged with the SHA of the commit on Docker Hub.

New Hello image with commit SHA as the tag

Deploy new releases to Digital Ocean

The last step is to deploy the image pushed to Docker Hub on the Digital Ocean Droplet. To do this, we will use another action from the marketplace. This one is SSH Remote Commands

SSH Remote Commands - GitHub Marketplace

GitHub Action for executing remote ssh commands. Important: Only support Linux docker container. See action.yml for more detailed information.

github.com

SSH Remote Commands - GitHub Marketplace

Add a deploy job to the workflow at .github/workflows/elixir.yml. It should now look like this.

name: Elixir CI/CD

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

  publish:
    name: Publish to Docker Hub
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/checkout@v3
    - name: Publish to DockerHub
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: codelever/hello_app:${{ github.sha }}
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

  deploy:
    name: Deploy to Digital Ocean
    runs-on: ubuntu-latest
    needs: [build, publish]
    steps:
    - uses: actions/checkout@v3
    - name: Executing remote  command
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        USERNAME: ${{ secrets.USERNAME }}
        KEY: ${{ secrets.SSHKEY }}
        script: docker service update --image codelever/hello_app:${{ github.sha }} hello_app_web

Things to note.

The new deploy job will only run if both the build and publish jobs succeed.
We are updating the hello_app_web service in the swarm using the same command we used previously from the Droplet’s shell. This means the swarm must be running. If it is not, this will fail. Also, note how we again use the commit SHA from GitHub to identify the image to deploy.

Add an SSH Key for the job

Personally, I use 1Password to create and manage my SSH keys. So, I won’t detail the steps of creating them here as I won’t be testing them.

The following guide will give you some guidance on how to create an SSH key pair. Take a look at step 1. This guide is for Linux, but it should also work on a Mac. Be careful not to overwrite your existing key, if you have one!

How to Set Up SSH Keys on Ubuntu 22.04 | DigitalOcean

SSH, or secure shell, is an encrypted protocol used to administer and communicate with servers. When working with an Ubuntu server, chances are you will spend most of your time in a terminal session connected to your server through SSH. In this guide, we'll focus on setting up SSH keys for an Ubuntu 22.04 installation.

www.digitalocean.com

How to Set Up SSH Keys on Ubuntu 22.04 | DigitalOcean

This guide also has good instructions on how to create and load SSH keys onto your droplet. It was very helpful to me in this whole process.

Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

Once you Deploy Phoenix and Postgres to Digital Ocean with Docker, the next step is to eliminate the manual deployment steps and establish a continuous integration / continuous deployment (CI/CD) pipeline. With a continuous deployment pipeline, every change that passes all stages of your production pipeline is released to your users.

axelclark.com

Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

It is worth noting that I am using an ed25519 key.

Once you have an SSH key pair generated, you will need to add it to the Droplet and add the new public key to the authorized_keys file. This guide explains how to do this. Since we created the Droplet without password access, we will have to use the third method it discusses.

How to Upload an SSH Public Key to an Existing Droplet | DigitalOcean Documentation

DigitalOcean Droplets are Linux-based virtual machines (VMs) that run on top of virtualized hardware. Each Droplet you create is a new server you can use, either standalone or as part of a larger, cloud-based infrastructure.

docs.digitalocean.com

How to Upload an SSH Public Key to an Existing Droplet | DigitalOcean Documentation

This is also a good resource. It has instructions for generating ed25519 keys from the command line.

GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

GitHub Action for executing remote ssh commands. Important: Only support Linux docker container. See action.yml for more detailed information.

github.com

GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

Here’s what worked for me to get the key installed on the Droplet. Log into the Droplet with SSH as root.

ssh root@67.205.159.134

The ~/.ssh directory already exists, so we don’t need to create it.

Open the authorized_keys file.

nano ~/.ssh/authorized_keys

Paste the new Public key at the bottom of the file. Make sure there are no blank lines in the file. Then save the file. From the above guide:

Paste the contents of your SSH key into the file by right-clicking in your terminal and choosing Paste or by using a keyboard shortcut like CTRL+SHIFT+V. Then, save and close the file. In nano, save by pressing CTRL+Oand then ENTER, and exit by pressing CTRL+X.

Once the authorized_keys file contains the public key, you need to update permissions on some of the files. The ~/.ssh directory and authorized_keys file must have specific restricted permissions (700 for ~/.ssh and 600 for authorized_keys). If they don’t, you won’t be able to log in.

Check the permissions and ownership of the files.

This is important!

chmod -R go= ~/.ssh
chown -R $USER:$USER ~/.ssh

This permission issue hung me up. The job didn’t work until I ensured the permissions were correct, as shown above.

Add secrets to GitHub

We need to add GitHub secrets for the HOST, USERNAME, and SSHKEY we want to use to access the Droplet. As discussed above, It’s a good idea to make a new SSH key just for this GitHub action. That way, it can easily be revoked. Don’t just use the same key you used when you created the Droplet.

Encrypted secrets - GitHub Docs

docs.github.com

For HOST, use the IP address of your Droplet.
For USERNAME, use root.
For SSHKEY, use the private portion of the SSH key we just generated.

When you add the SSHKEY secret, ensure you do not modify the file. Leave the BEGIN PRIVATE KEY and END PRIVATE KEY headers. I removed them and the job failed.

-----BEGIN PRIVATE KEY-----
M ..snip.....
Q==
-----END PRIVATE KEY-----

When you are done, GitHub should look something like this.

Secrets for Deploy job added

Make a small change to Hello

Open the file /templates/page/index.html.heex and make this change. This way we can see right away if the updates we committed made it to the Droplet. This small change doesn’t break any tests.

.. snip
<article class="column">
    <h2>Phoenix Resources</h2>
    <ul>
      <li>
        <a href="https://hexdocs.pm/phoenix/overview.html">Guides &amp; Docs</a>
      </li>
      <li>
        <a href="https://github.com/phoenixframework/phoenix">Source</a>
      </li>
      <li>
        <a href="https://github.com/phoenixframework/phoenix/blob/v1.6/CHANGELOG.md">v1.6 Changelog</a>
      </li>
.. snip

VS Code Extension

I ran into formatting issues with the workflow, so I installed this extension for Visual Studio Code. You may find it useful as well.

GitHub Actions - Visual Studio Marketplace

Provides Github Actions YAML support via yaml-language-server.

marketplace.visualstudio.com

GitHub Actions - Visual Studio Marketplace

Verify CI/CD pipeline

Commit the change to the template and the workflow (elixir.yml), and push them to main on GitHub. This will trigger the workflow.

Check to make sure the Workflow ran successfully on GitHub.

Finally, success!

Then check to make sure the Hello application has our small change.

Code

The current state of the Hello project is here.

GitHub - code-lever/phx_hello at ci-cd

A Phoenix 1.6.12 Hello Application for using in testing Digital Ocean deployments - GitHub - code-lever/phx_hello at ci-cd

github.com

Conclusion

Enjoy CI/CD for your Phoenix project!

References

These posts were extremely helpful. All of the articles in this series are worth reviewing.

Set Up Continuous Deployment for Elixir with GitHub Actions - StakNine

Once you Deploy Elixir to Digital Ocean with Docker, the next step is to eliminate the manual Elixir deployment steps and establish a continuous integration / continuous deployment (CI/CD) pipeline. There are many CI/CD services available, but we are going to set up continuous deployment for Elixir with GitHub Actions.

staknine.com

Set Up Continuous Deployment for Elixir with GitHub Actions - StakNine

Same with this series.

Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

axelclark.com

Deploying with Releases - Phoenix v1.6.12

The only thing we'll need for this guide is a working Phoenix application. For those of us who need a simple application to deploy, please follow the Up and Running guide. Our main goal for this guide is to package your Phoenix application into a self-contained directory that includes the Erlang VM, Elixir, all of your code and dependencies.

hexdocs.pm

Publish Docker - GitHub Marketplace

github.com

GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

GitHub Action for executing remote ssh commands. Important: Only support Linux docker container. See action.yml for more detailed information.

github.com

CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

Contents

Setup up CI

Create the workflow on GitHub

Code

GitHub - code-lever/phx_hello at cont-integration

Set up CD

Publish Docker Container to Docker Hub

Publish Docker - GitHub Marketplace

Create an Access Token on Docker Hub

Manage access tokens

Create Encrypted Secrets on GitHub

Encrypted secrets - GitHub Docs

Publish a new image with CD

Deploy new releases to Digital Ocean

SSH Remote Commands - GitHub Marketplace

Add an SSH Key for the job

How to Set Up SSH Keys on Ubuntu 22.04 | DigitalOcean

Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

How to Upload an SSH Public Key to an Existing Droplet | DigitalOcean Documentation

GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

Add secrets to GitHub

Encrypted secrets - GitHub Docs

Make a small change to Hello

VS Code Extension

GitHub Actions - Visual Studio Marketplace

Verify CI/CD pipeline

Code

GitHub - code-lever/phx_hello at ci-cd

Conclusion

References

Set Up Continuous Deployment for Elixir with GitHub Actions - StakNine

Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

Deploying with Releases - Phoenix v1.6.12

Publish Docker - GitHub Marketplace

GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.