CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

Setting this up is relatively straightforward, but there are a lot of steps. This isn’t really a primer on GitHub Actions or CI; it is a direct setup guild for getting continuous integration and deployment running with the Hello application, which is a very simple Phoenix 1.6 application that uses PostgreSQL.

If you don’t have Phoenix running in Docker on a Droplet, you may want to start here.

Contents

Setup up CI

Create the workflow on GitHub

First, log in to GitHub, navigate to the repository, click the Actions Tab, and then click the New workflow button. You can then search for a workflow. You should see the “Elixir” workflow. Click the Configure button and you will be presented with the following screen.

The Elixir Workflow Action
The Elixir Workflow Action

Click the Configure button and you will be presented with the following screen.

The CD workflow
The CD workflow

All this screen does is present you with a YAML file in the ./github/workflows/ folder of the project. This file will be committed to the repository when you click the Start commit button. You don’t have to edit it here; you are free to make it yourself in the repository and push it up if you wish.

We want to add a PostgreSQL database and change the versions of Elixir and OTP to match what we are using on Digital Ocean. Here’s the completed file.

name: Elixir CI

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:

    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

Take note of the following.

  • The Postgres image was added as a service. Now we have a database available for our tests.
  • The Postgres image matches what we are using in the docker-compose.yml and deploying on Digital Ocean.
  • The elixir-version and otp-version were also changed to match.

That’s pretty much it. Now, every time something is committed to the main branch and action will run our project’s tests.

Code

The current state of the Hello project is here.

Set up CD

Continuous Deployment is configured as a two-step process that will only run if the CI step passes, i.e., the project builds and the tests pass. This way, we don’t deploy failing builds. The first step is to publish the new container to Docker Hub. The second is to deploy it on our droplet on Digital Ocean.

Publish Docker Container to Docker Hub

To accomplish this, we will use the community-created Publish Docker action. You can find it here:

Update the workflow and add a new job. The workflow is in .github/workflows/elixir.yml. It should now look like this. 

name: Elixir CI/CD

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

  publish:
    name: Publish to Docker Hub
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/checkout@v3
    - name: Publish to DockerHub
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: codelever/hello_app:${{ github.sha }}
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

Two things to note:

  • needs: build ensures that this action only runs if the CI build step we set up previously succeeds. This prevents us from deploying a build that fails CI.
  • github.sha is used for the image tag instead of a version number. This way we don’t have to update the project version number every time we commit and push to main.

Create an Access Token on Docker Hub

Follow the instructions to create an Access Token for the CD action to log in to Docker Hub. This is much better than using your user password. It can be revoked at any time and the access the CD service has to the Docker Hub repositories is tracked separately from your user account access.

Create Encrypted Secrets on GitHub

Follow the instructions for creating and storing encrypted secrets for the DOCKER_USERNAME and DOCKER_PASSWORD. These are the credentials for the CD action to access Docker Hub. Make sure to use the Access Token created in the previous step instead of your actual Docker Hub password.

You should end up with something like this in the GitHub UI for your project.

Docker secrets added to GitHub
Docker secrets added to GitHub

Publish a new image with CD

Commit the updated workflow with the publish job, and push to main.

Now log in to GitHub and look at the actions. You should see that the workflow has completed both jobs.

Elixir CI/CD workflow completed successfully
Elixir CI/CD workflow completed successfully

We should also have a new image tagged with the SHA of the commit on Docker Hub.

New Hello image with commit SHA as the tag
New Hello image with commit SHA as the tag

Deploy new releases to Digital Ocean

The last step is to deploy the image pushed to Docker Hub on the Digital Ocean Droplet. To do this, we will use another action from the marketplace. This one is SSH Remote Commands

Add a deploy job to the workflow at .github/workflows/elixir.yml. It should now look like this. 

name: Elixir CI/CD

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

  publish:
    name: Publish to Docker Hub
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/checkout@v3
    - name: Publish to DockerHub
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: codelever/hello_app:${{ github.sha }}
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

  deploy:
    name: Deploy to Digital Ocean
    runs-on: ubuntu-latest
    needs: [build, publish]
    steps:
    - uses: actions/checkout@v3
    - name: Executing remote  command
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        USERNAME: ${{ secrets.USERNAME }}
        KEY: ${{ secrets.SSHKEY }}
        script: docker service update --image codelever/hello_app:${{ github.sha }} hello_app_web

Things to note.

  • The new deploy job will only run if both the build and publish jobs succeed.
  • We are updating the hello_app_web service in the swarm using the same command we used previously from the Droplet’s shell. This means the swarm must be running. If it is not, this will fail. Also, note how we again use the commit SHA from GitHub to identify the image to deploy.

Add an SSH Key for the job

Personally, I use 1Password to create and manage my SSH keys. So, I won’t detail the steps of creating them here as I won’t be testing them.

The following guide will give you some guidance on how to create an SSH key pair. Take a look at step 1. This guide is for Linux, but it should also work on a Mac. Be careful not to overwrite your existing key, if you have one!

This guide also has good instructions on how to create and load SSH keys onto your droplet. It was very helpful to me in this whole process.

It is worth noting that I am using an ed25519 key.

Once you have an SSH key pair generated, you will need to add it to the Droplet and add the new public key to the authorized_keys file. This guide explains how to do this. Since we created the Droplet without password access, we will have to use the third method it discusses.

This is also a good resource. It has instructions for generating ed25519 keys from the command line.

Here’s what worked for me to get the key installed on the Droplet. Log into the Droplet with SSH as root.

ssh root@67.205.159.134

The ~/.ssh directory already exists, so we don’t need to create it.

Open the authorized_keys file.

nano ~/.ssh/authorized_keys

Paste the new Public key at the bottom of the file. Make sure there are no blank lines in the file. Then save the file. From the above guide:

Paste the contents of your SSH key into the file by right-clicking in your terminal and choosing Paste or by using a keyboard shortcut like CTRL+SHIFT+V. Then, save and close the file. In nano, save by pressing CTRL+Oand then ENTER, and exit by pressing CTRL+X.
Once the authorized_keys file contains the public key, you need to update permissions on some of the files. The ~/.ssh directory and authorized_keys file must have specific restricted permissions (700 for ~/.ssh and 600 for authorized_keys). If they don’t, you won’t be able to log in.
Check the permissions and ownership of the files.

This is important!

chmod -R go= ~/.ssh
chown -R $USER:$USER ~/.ssh

This permission issue hung me up. The job didn’t work until I ensured the permissions were correct, as shown above.

Add secrets to GitHub

We need to add GitHub secrets for the HOST, USERNAME, and SSHKEY we want to use to access the Droplet. As discussed above, It’s a good idea to make a new SSH key just for this GitHub action. That way, it can easily be revoked. Don’t just use the same key you used when you created the Droplet.

  • For HOST, use the IP address of your Droplet.
  • For USERNAME, use root.
  • For SSHKEY, use the private portion of the SSH key we just generated.

When you add the SSHKEY secret, ensure you do not modify the file. Leave the BEGIN PRIVATE KEY and END PRIVATE KEY headers. I removed them and the job failed.

-----BEGIN PRIVATE KEY-----
M ..snip.....
Q==
-----END PRIVATE KEY-----

When you are done, GitHub should look something like this.

Secrets for Deploy job added
Secrets for Deploy job added

Make a small change to Hello

Open the file /templates/page/index.html.heex and make this change. This way we can see right away if the updates we committed made it to the Droplet. This small change doesn’t break any tests.

.. snip
<article class="column">
    <h2>Phoenix Resources</h2>
    <ul>
      <li>
        <a href="https://hexdocs.pm/phoenix/overview.html">Guides &amp; Docs</a>
      </li>
      <li>
        <a href="https://github.com/phoenixframework/phoenix">Source</a>
      </li>
      <li>
        <a href="https://github.com/phoenixframework/phoenix/blob/v1.6/CHANGELOG.md">v1.6 Changelog</a>
      </li>
.. snip

VS Code Extension

I ran into formatting issues with the workflow, so I installed this extension for Visual Studio Code. You may find it useful as well.

Verify CI/CD pipeline

Commit the change to the template and the workflow (elixir.yml), and push them to main on GitHub. This will trigger the workflow.

Check to make sure the Workflow ran successfully on GitHub.

Finally, success!
Finally, success!

Then check to make sure the Hello application has our small change.

image

Code

The current state of the Hello project is here.

Conclusion

Enjoy CI/CD for your Phoenix project!

References

These posts were extremely helpful. All of the articles in this series are worth reviewing.

Same with this series.