Logo
    CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

    CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

    CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean

    Setting this up is relatively straightforward, but there are a lot of steps. This isn’t really a primer on GitHub Actions or CI; it is a direct setup guild for getting continuous integration and deployment running with the Hello application, which is a very simple Phoenix 1.6 application that uses PostgreSQL.

    If you don’t have Phoenix running in Docker on a Droplet, you may want to start here.

    Contents

    • CI/CD for Phoenix 1.6 with GitHub and Docker on Digital Ocean
    • Contents
    • Setup up CI
    • Create the workflow on GitHub
    • Code
    • Set up CD
    • Publish Docker Container to Docker Hub
    • Create an Access Token on Docker Hub
    • Create Encrypted Secrets on GitHub
    • Publish a new image with CD
    • Deploy new releases to Digital Ocean
    • Add an SSH Key for the job
    • Add secrets to GitHub
    • Make a small change to Hello
    • VS Code Extension
    • Verify CI/CD pipeline
    • Code
    • Conclusion
    • References

    Setup up CI

    Create the workflow on GitHub

    First, log in to GitHub, navigate to the repository, click the Actions Tab, and then click the New workflow button. You can then search for a workflow. You should see the “Elixir” workflow. Click the Configure button and you will be presented with the following screen.

    The Elixir Workflow Action
    The Elixir Workflow Action

    Click the Configure button and you will be presented with the following screen.

    The CD workflow
    The CD workflow

    All this screen does is present you with a YAML file in the ./github/workflows/ folder of the project. This file will be committed to the repository when you click the Start commit button. You don’t have to edit it here; you are free to make it yourself in the repository and push it up if you wish.

    We want to add a PostgreSQL database and change the versions of Elixir and OTP to match what we are using on Digital Ocean. Here’s the completed file.

    Take note of the following.

    • The Postgres image was added as a service. Now we have a database available for our tests.
    • The Postgres image matches what we are using in the docker-compose.yml and deploying on Digital Ocean.
    • The elixir-version and otp-version were also changed to match.

    That’s pretty much it. Now, every time something is committed to the main branch and action will run our project’s tests.

    Code

    The current state of the Hello project is here.

    GitHub - code-lever/phx_hello at cont-integration

    A Phoenix 1.6.12 Hello Application for using in testing Digital Ocean deployments - GitHub - code-lever/phx_hello at cont-integration

    github.com

    GitHub - code-lever/phx_hello at cont-integration

    Set up CD

    Continuous Deployment is configured as a two-step process that will only run if the CI step passes, i.e., the project builds and the tests pass. This way, we don’t deploy failing builds. The first step is to publish the new container to Docker Hub. The second is to deploy it on our droplet on Digital Ocean.

    Publish Docker Container to Docker Hub

    To accomplish this, we will use the community-created Publish Docker action. You can find it here:

    Publish Docker - GitHub Marketplace

    This Action for Docker uses the Git branch as the Docker tag for building and pushing the container. Hereby the master-branch is published as the latest-tag. name is the name of the image you would like to push username the login username for the registry password the authentication token [preferred] or login password for the registry.

    github.com

    Publish Docker - GitHub Marketplace

    Update the workflow and add a new job. The workflow is in .github/workflows/elixir.yml. It should now look like this.

    Two things to note:

    • needs: build ensures that this action only runs if the CI build step we set up previously succeeds. This prevents us from deploying a build that fails CI.
    • github.sha is used for the image tag instead of a version number. This way we don’t have to update the project version number every time we commit and push to main.

    Create an Access Token on Docker Hub

    Follow the instructions to create an Access Token for the CD action to log in to Docker Hub. This is much better than using your user password. It can be revoked at any time and the access the CD service has to the Docker Hub repositories is tracked separately from your user account access.

    Manage access tokens

    Estimated reading time: 3 minutes Docker Hub lets you create personal access tokens as alternatives to your password. You can use tokens to access Hub images from the Docker CLI. Using personal access tokens provides some advantages over a password: You can investigate the last usage of the access token and disable or delete it if you find any suspicious activity.

    docs.docker.com

    Manage access tokens

    Create Encrypted Secrets on GitHub

    Follow the instructions for creating and storing encrypted secrets for the DOCKER_USERNAME and DOCKER_PASSWORD. These are the credentials for the CD action to access Docker Hub. Make sure to use the Access Token created in the previous step instead of your actual Docker Hub password.

    Encrypted secrets - GitHub Docs

    Secrets are encrypted environment variables that you create in an organization, repository, or repository environment. The secrets that you create are available to use in GitHub Actions workflows. GitHub uses a libsodium sealed box to help ensure that secrets are encrypted before they reach GitHub and remain encrypted until you use them in a workflow.

    docs.github.com

    Encrypted secrets - GitHub Docs

    You should end up with something like this in the GitHub UI for your project.

    Docker secrets added to GitHub
    Docker secrets added to GitHub

    Publish a new image with CD

    Commit the updated workflow with the publish job, and push to main.

    Now log in to GitHub and look at the actions. You should see that the workflow has completed both jobs.

    Elixir CI/CD workflow completed successfully
    Elixir CI/CD workflow completed successfully

    We should also have a new image tagged with the SHA of the commit on Docker Hub.

    New Hello image with commit SHA as the tag
    New Hello image with commit SHA as the tag

    Deploy new releases to Digital Ocean

    The last step is to deploy the image pushed to Docker Hub on the Digital Ocean Droplet. To do this, we will use another action from the marketplace. This one is SSH Remote Commands

    SSH Remote Commands - GitHub Marketplace

    GitHub Action for executing remote ssh commands. Important: Only support Linux docker container. See action.yml for more detailed information.

    github.com

    SSH Remote Commands - GitHub Marketplace

    Add a deploy job to the workflow at .github/workflows/elixir.yml. It should now look like this.

    Things to note.

    • The new deploy job will only run if both the build and publish jobs succeed.
    • We are updating the hello_app_web service in the swarm using the same command we used previously from the Droplet’s shell. This means the swarm must be running. If it is not, this will fail. Also, note how we again use the commit SHA from GitHub to identify the image to deploy.

    Add an SSH Key for the job

    Personally, I use 1Password to create and manage my SSH keys. So, I won’t detail the steps of creating them here as I won’t be testing them.

    The following guide will give you some guidance on how to create an SSH key pair. Take a look at step 1. This guide is for Linux, but it should also work on a Mac. Be careful not to overwrite your existing key, if you have one!

    How to Set Up SSH Keys on Ubuntu 22.04 | DigitalOcean

    SSH, or secure shell, is an encrypted protocol used to administer and communicate with servers. When working with an Ubuntu server, chances are you will spend most of your time in a terminal session connected to your server through SSH. In this guide, we'll focus on setting up SSH keys for an Ubuntu 22.04 installation.

    www.digitalocean.com

    How to Set Up SSH Keys on Ubuntu 22.04 | DigitalOcean

    This guide also has good instructions on how to create and load SSH keys onto your droplet. It was very helpful to me in this whole process.

    Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

    Once you Deploy Phoenix and Postgres to Digital Ocean with Docker, the next step is to eliminate the manual deployment steps and establish a continuous integration / continuous deployment (CI/CD) pipeline. With a continuous deployment pipeline, every change that passes all stages of your production pipeline is released to your users.

    axelclark.com

    Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

    It is worth noting that I am using an ed25519 key.

    Once you have an SSH key pair generated, you will need to add it to the Droplet and add the new public key to the authorized_keys file. This guide explains how to do this. Since we created the Droplet without password access, we will have to use the third method it discusses.

    How to Upload an SSH Public Key to an Existing Droplet | DigitalOcean Documentation

    DigitalOcean Droplets are Linux-based virtual machines (VMs) that run on top of virtualized hardware. Each Droplet you create is a new server you can use, either standalone or as part of a larger, cloud-based infrastructure.

    docs.digitalocean.com

    How to Upload an SSH Public Key to an Existing Droplet | DigitalOcean Documentation

    This is also a good resource. It has instructions for generating ed25519 keys from the command line.

    GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

    GitHub Action for executing remote ssh commands. Important: Only support Linux docker container. See action.yml for more detailed information.

    github.com

    GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

    Here’s what worked for me to get the key installed on the Droplet. Log into the Droplet with SSH as root.

    ssh root@67.205.159.134

    The ~/.ssh directory already exists, so we don’t need to create it.

    Open the authorized_keys file.

    nano ~/.ssh/authorized_keys

    Paste the new Public key at the bottom of the file. Make sure there are no blank lines in the file. Then save the file. From the above guide:

    Paste the contents of your SSH key into the file by right-clicking in your terminal and choosing Paste or by using a keyboard shortcut like CTRL+SHIFT+V. Then, save and close the file. In nano, save by pressing CTRL+Oand then ENTER, and exit by pressing CTRL+X.
    Once the authorized_keys file contains the public key, you need to update permissions on some of the files. The ~/.ssh directory and authorized_keys file must have specific restricted permissions (700 for ~/.ssh and 600 for authorized_keys). If they don’t, you won’t be able to log in.
    Check the permissions and ownership of the files.

    This is important!

    chmod -R go= ~/.ssh
chown -R $USER:$USER ~/.ssh

    This permission issue hung me up. The job didn’t work until I ensured the permissions were correct, as shown above.

    Add secrets to GitHub

    We need to add GitHub secrets for the HOST, USERNAME, and SSHKEY we want to use to access the Droplet. As discussed above, It’s a good idea to make a new SSH key just for this GitHub action. That way, it can easily be revoked. Don’t just use the same key you used when you created the Droplet.

    Encrypted secrets - GitHub Docs

    Secrets are encrypted environment variables that you create in an organization, repository, or repository environment. The secrets that you create are available to use in GitHub Actions workflows. GitHub uses a libsodium sealed box to help ensure that secrets are encrypted before they reach GitHub and remain encrypted until you use them in a workflow.

    docs.github.com

    Encrypted secrets - GitHub Docs
    • For HOST, use the IP address of your Droplet.
    • For USERNAME, use root.
    • For SSHKEY, use the private portion of the SSH key we just generated.

    When you add the SSHKEY secret, ensure you do not modify the file. Leave the BEGIN PRIVATE KEY and END PRIVATE KEY headers. I removed them and the job failed.

    -----BEGIN PRIVATE KEY-----
M ..snip.....
Q==
-----END PRIVATE KEY-----

    When you are done, GitHub should look something like this.

    Secrets for Deploy job added
    Secrets for Deploy job added

    Make a small change to Hello

    Open the file /templates/page/index.html.heex and make this change. This way we can see right away if the updates we committed made it to the Droplet. This small change doesn’t break any tests.

    VS Code Extension

    I ran into formatting issues with the workflow, so I installed this extension for Visual Studio Code. You may find it useful as well.

    GitHub Actions - Visual Studio Marketplace

    Provides Github Actions YAML support via yaml-language-server.

    marketplace.visualstudio.com

    GitHub Actions - Visual Studio Marketplace

    Verify CI/CD pipeline

    Commit the change to the template and the workflow (elixir.yml), and push them to main on GitHub. This will trigger the workflow.

    Check to make sure the Workflow ran successfully on GitHub.

    Finally, success!
    Finally, success!

    Then check to make sure the Hello application has our small change.

    image

    Code

    The current state of the Hello project is here.

    GitHub - code-lever/phx_hello at ci-cd

    A Phoenix 1.6.12 Hello Application for using in testing Digital Ocean deployments - GitHub - code-lever/phx_hello at ci-cd

    github.com

    GitHub - code-lever/phx_hello at ci-cd

    Conclusion

    Enjoy CI/CD for your Phoenix project!

    References

    These posts were extremely helpful. All of the articles in this series are worth reviewing.

    Set Up Continuous Deployment for Elixir with GitHub Actions - StakNine

    Once you Deploy Elixir to Digital Ocean with Docker, the next step is to eliminate the manual Elixir deployment steps and establish a continuous integration / continuous deployment (CI/CD) pipeline. There are many CI/CD services available, but we are going to set up continuous deployment for Elixir with GitHub Actions.

    staknine.com

    Set Up Continuous Deployment for Elixir with GitHub Actions - StakNine

    Same with this series.

    Continuously Deploy Phoenix to Digital Ocean with GitHub Actions

    Once you Deploy Phoenix and Postgres to Digital Ocean with Docker, the next step is to eliminate the manual deployment steps and establish a continuous integration / continuous deployment (CI/CD) pipeline. With a continuous deployment pipeline, every change that passes all stages of your production pipeline is released to your users.

    axelclark.com

    Continuously Deploy Phoenix to Digital Ocean with GitHub Actions
    Deploying with Releases - Phoenix v1.6.12

    The only thing we'll need for this guide is a working Phoenix application. For those of us who need a simple application to deploy, please follow the Up and Running guide. Our main goal for this guide is to package your Phoenix application into a self-contained directory that includes the Erlang VM, Elixir, all of your code and dependencies.

    hexdocs.pm

    Deploying with Releases - Phoenix v1.6.12
    Publish Docker - GitHub Marketplace

    This Action for Docker uses the Git branch as the Docker tag for building and pushing the container. Hereby the master-branch is published as the latest-tag. name is the name of the image you would like to push username the login username for the registry password the authentication token [preferred] or login password for the registry.

    github.com

    Publish Docker - GitHub Marketplace
    GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.

    GitHub Action for executing remote ssh commands. Important: Only support Linux docker container. See action.yml for more detailed information.

    github.com

    GitHub - appleboy/ssh-action: GitHub Actions for executing remote ssh commands.
    name: Elixir CI

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:

    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test
    name: Elixir CI/CD

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

  publish:
    name: Publish to Docker Hub
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/checkout@v3
    - name: Publish to DockerHub
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: codelever/hello_app:${{ github.sha }}
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}
    name: Elixir CI/CD

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:
    name: Build and test
    runs-on: ubuntu-latest
    services:
      db:
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        image: postgres:14.5
        ports: ['5432:5432']
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
          
    steps:
    - uses: actions/checkout@v3
    - name: Set up Elixir
      uses: erlef/setup-beam@988e02bfe678367a02564f65ca2e37726dc0268f
      with:
        elixir-version: '1.14.0' # Define the elixir version [required]
        otp-version: '25.0.1' # Define the OTP version [required]
    - name: Restore dependencies cache
      uses: actions/cache@v3
      with:
        path: deps
        key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}
        restore-keys: ${{ runner.os }}-mix-
    - name: Install dependencies
      run: mix deps.get
    - name: Run tests
      run: mix test

  publish:
    name: Publish to Docker Hub
    runs-on: ubuntu-latest
    needs: build
    steps:
    - uses: actions/checkout@v3
    - name: Publish to DockerHub
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: codelever/hello_app:${{ github.sha }}
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

  deploy:
    name: Deploy to Digital Ocean
    runs-on: ubuntu-latest
    needs: [build, publish]
    steps:
    - uses: actions/checkout@v3
    - name: Executing remote  command
      uses: appleboy/ssh-action@master
      with:
        host: ${{ secrets.HOST }}
        USERNAME: ${{ secrets.USERNAME }}
        KEY: ${{ secrets.SSHKEY }}
        script: docker service update --image codelever/hello_app:${{ github.sha }} hello_app_web
    .. snip
<article class="column">
    <h2>Phoenix Resources</h2>
    <ul>
      <li>
        <a href="https://hexdocs.pm/phoenix/overview.html">Guides &amp; Docs</a>
      </li>
      <li>
        <a href="https://github.com/phoenixframework/phoenix">Source</a>
      </li>
      <li>
        <a href="https://github.com/phoenixframework/phoenix/blob/v1.6/CHANGELOG.md">v1.6 Changelog</a>
      </li>
.. snip