Web Framework

The web framework recommended by this guide is the Phoenix Framework. The Phoenix framework is particularly well-suited to applications requiring real-time communication between the client and server. In our case this will be useful to automatically push AI-generated images from the server to the client as they become ready.

The Phoenix framework is both a frontend and backend framework, so you can write your entire web app using the Phoenix Framework. Other web frameworks such as Angular or React are frontend only frameworks. If you used them you would need to user and backend framework such as ExpressJS or Spring.

The Phoenix framework uses Elixir as the backend language. For the frontend we will use TypeScript.

Installation

To use the Phoenix framework, a few pre-requisites must be installed first.

Elixir

To use the Phoenix framework, you will need to have Elixir and the Erlang virtual machine (BEAM) installed. Elixir code runs on the BEAM virtual machine. To install these pre-requisites, run:

sudo dnf install elixir erlang

Phoenix Application Generator

To generate new Phoenix application, we need to install the Phoenix Application Generator:

mix archive.install hex phx_new

Mix is a build tool for Elixir. It is shipped together with Elixir, so you should already have it installed if you followed the Elixir step above.

Database

Technically the Phoenix framework can be used without a database, however for most web apps a database is required. The database recommended by this guide is PostgreSQL.

To install the database on your development machine, run:

sudo dnf install postgresql-server postgresql-contrib

Now enable your PostgreSQL server to run on boot:

sudo systemctl enable postgresql

Now your database server should start automatically on every boot of your operating system. To add some initial configuration to your database run the below command:

sudo postgresql-setup --initdb --unit postgresql

You may now manually start your database server:

sudo systemctl start postgresql

We will now create a database and user for your database. To start, enter the PostgreSQL console by running:

sudo -u postgres psql

Note that we run the above "psql" command using the "postgres" user. The postgres user was created automatically when you installed PostgreSQL earlier.

In the SQL console, run the below command to create a new user:

CREATE USER henrik;

Replace the username, e.g. "henrik" above, with your desired username.

Next set a password for your new user by running:

\password henrik

Naturally replace the username "henrik" with the username you chose above. You will be prompted to enter a password. Save the database credentials in a safe place, such as your password manager.

Give your new user permission to create new databases:

ALTER USER henrik CREATEDB;

Naturally replace the username "henrik" with the username you chose above.

Now enter "\q" into the console and press Enter to exit. You should now have PostgreSQL setup and running on your development machine. For further information about PostgreSQL on Fedora, you can refer to the PostgreSQL Fedora Documentation.

Live Reloading

To support live reloading on Linux, install the inotify-tools:

sudo dnf install inotify-tools

The inotify-tools will automatically reload the web app in your browser as you make code changes.

Generate Phoenix Project

We will store our Phoenix project in a directory named "web" in the root of our Git repository. In the root of your Git repository run the below command to create this new directory:

mkdir web

Now navigate inside the new directory:

cd web

To generate a new Phoenix project run the below command in your Git repository's root directory:

mix phx.new . --app epic_fantasy_forge --module EpicFantasyForge

On the prompt that the directory already exists choose "Y" and press Enter. On the prompt whether to install dependencies choose "Y" and press Enter.

Once Phoenix is done generating the new project and installing dependencies you should have a new Phoenix project in your web folder. Below you can find an example of what creating a new Phoenix project should look like:

Database Configuration

By default, Phoenix expects your database to have a user named "postgres" with a password "postgres". In our case we don't have such a user. For our database we earlier created a new user with a custom password in the Database section above.

Tip

Note that whilst during installation of PostgreSQL a user named "postgres" was automatically created on our operating system, this is not the same as a database user. Database users and operating system users are not the same thing.

To make Phoenix use our custom database credentials, edit the file development configuration file located at "web/config/dev.exs" relative to the root of your Git repository. Update the "username" and "password" fields as below:

dev.exs

username: System.get_env("POSTGRES_USER") || "postgres",
password: System.get_env("POSTGRES_PASSWORD") || "postgres",

To keep your actual username and password outside of the version control system they will be read in from environment variables. To set the POSTGRES_USER environment variable run the below command:

read -s -p "Enter PostgreSQL username:" POSTGRES_USER

On the prompt enter your PostgreSQL username. Then export this new variable:

export POSTGRES_USER

Do the same for your PostgreSQL password:

read -s -p "Enter PostgreSQL password:" POSTGRES_PASSWORD

export POSTGRES_PASSWORD

Database Creation

Now that our database credentials are setup, we can create the database for our development environment by running:

mix ecto.create

You should now have a development database on your development machine that your Phoenix web app will use.

Run Phoenix Project

Everything should be setup now to start your Phoenix web app. To run your Phoenix web app, run the below command:

mix phx.server

Your Phoenix web app should now be running:

The Phoenix web app should now be accessible from the link http://localhost:4000:

Linting

To lint the code in our Phoenix project, we will use the built-in format tool and the credo linter. Additionally we will use ESLint to lint our TypeScript code.

Built-in Formatter

To check for formatting issues in your code, run the below command in the root directory of your Phoenix project:

mix format --check-formatted

To automatically apply any formatting suggested by the formatter, run:

mix format

Credo

To use Credo, you need to add it as a dependency to your project. To do so, open the file "mix.exs" in the root directory of your Phoenix project and add the below line to your dependencies list:

mix.exs

{:credo, "~> 1.7", only: [:dev, :test], runtime: false}

Now run the below command to install your dependencies:

mix deps.get

Now you should be able to run the Credo linter against your project:

mix credo --strict

Unfortunately, the linting may fail as the automatically generated files by the Phoenix framework do not pass linting with the strict checks. We need to manually fix these issues. To do so, add the following line near the top of core_components.ex where another alias is already defined:

core_components.ex

alias Phoenix.HTML.Form

Then replace all usages of "Phoenix.HTML.Form" in core_components.ex with "Form".

We need to do a similar thing in data_case.ex. Add an alias for the Ecto Sandbox near the top of the file:

data_case.ex

alias Ecto.Adapters.SQL.Sandbox

Now replace all usages of "Ecto.Adapters.SQL.Sandbox" with "Sandbox" in data_case.ex.

ESLint

To use ESLint in our Phoenix project, we need to first install Node.js and npm. To install them run:

sudo dnf install nodejs npm

Next navigate to your Phoenix project's assets directory:

cd assets

Initialize npm:

npm init

When prompted, given your package a name, e.g. "epic-fantasy-forge". On all other prompts you can leave the default option by hitting the Enter key.

Install eslint and typescript-eslint:

npm install --save-dev eslint @eslint/js typescript typescript-eslint

Create a configuration file for ESLint:

eslint.config.mjs

import eslint from '@eslint/js';
import tseslint from 'typescript-eslint';

export default tseslint.config(
  eslint.configs.recommended,
  tseslint.configs.recommended
);

We will only lint our own TypeScript files under the directory "ts" since we will not add any new JavaScript files to the project. Unfortunately the JavaScript files that are auto-generated by the Phoenix framework, e.g. app.js, don't pass the linting on default settings. Therefore we exclude them from ESLint instead of wasting time fixing these auto-generated files.

Note that the directory "ts" does not exist yet, so create it now by running:

mkdir ts

Now create an empty file inside the new directory:

touch ts/app.ts

The purpose of this file is so that we can add the "ts" directory to git already. You cannot add empty directories to git repositories.

Now we will add an npm script for linting TypeScript code. To do so, open package.json and add the following line in the scripts section:

package.json

"lint": "eslint ts"

You can now lint your TypeScript code by running:

npm run lint

Building

To build your Phoenix project, run:

mix compile --warnings-as-errors

Unfortunately the build may fail as the automatically generated Phoenix framework code files already contain some warnings. To fix these warnings, replace the existing "Gettext" line in gettext.ex with:

gettext.ex

use Gettext.Backend, otp_app: :epic_fantasy_forge

Replace the Gettext import line in core_components.ex with:

core_components.ex

use Gettext, backend: MyApp.Gettext

Testing

Unit tests

To run all Elixir unit tests, run the below command:

mix test

To run TypeScript tests, we will first need to install the Jest testing framework. Navigate to the assets folder in your Phoenix project and run the below command:

npm install --save-dev jest ts-jest @types/jest

Now update the "test" script in package.json to run jest:

package.json

"test": "jest --passWithNoTests"

You can now run TypeScript unit tests by running:

npm run test

Naturally you don't have any tests yet. We will add them in a later section of this guide.

Integration tests

To run all Elixir integration tests, the same command as for the unit tests is used:

mix test

To run all TypeScript integration tests, the same command as for the unit tests is used. Navigate to the assets folder in your Phoenix project and run:

npm run test

E2E tests

For E2E testing our web app, we will use BrowserStack. For more information about E2E testing our web app, see the E2E Tests section on the Testing page in this guide.

Code Coverage

Elixir

To see how much of our Elixir production code is covered by tests we will use a code coverage tool called ExCoveralls. First we need to add ExCoveralls as a dependency to our Phoenix project. To do so, add the below line to the dependencies list in mix.exs:

mix.exs

{:excoveralls, "~> 0.18", only: :test}

Install the dependency by running:

mix deps.get

Add the below configuration to the project section of mix.exs in order to use ExCoveralls in your project:

mix.exs

test_coverage: [tool: ExCoveralls],
    preferred_cli_env: [
      coveralls: :test,
      "coveralls.detail": :test,
      "coveralls.post": :test,
      "coveralls.html": :test,
      "coveralls.cobertura": :test
    ]

You can now get a code coverage report for your Elixir code in your project by running:

mix coveralls

TypeScript

To see how much of our TypeScript production code is covered by tests we will use Jest. With the correct configuration, Jest not only runs the tests but also generates a code coverage report. Start by generating an initial configuration file for Jest. Run the below command in the "assets" directory of your Phoenix project:

npm init jest@latest

On the prompts, give answers as in the below screenshot:

Install some necessary dependencies:

npm install --save-dev jest-environment-jsdom ts-node

When you now run the below command, you should also get a code coverage report.

npm run test

Since we don't have any Jest tests yet, no code coverage report is generated. We will add Jest tests in a later section of this guide. Later we will also set coverage thresholds, so that "npm run test" will fail if the code coverage is too low.

Deployment

We don't yet have a database for our test and production environments. So we cannot yet deploy our Phoenix application to those environments. Later when we add a database for those environments we will deploy our Phoenix application. For now we will just be able to run our Phoenix application on our development machine.

CI

Whilst we cannot yet deploy the Phoenix application to the test and production environments, we can already add the lint, build and test stages to our CI.

Lint Stage

To add a lint stage to your CI, modify your .gitlab-ci.yml file as per the below:

.gitlab-ci.yml

stages:
    - lint

include:
    - local: "ci/lint.yml"

In the "ci" directory of your Git repository, add a file named "lint.yml" and add the below content:

lint.yml

lint-web:
  before_script:
    - apt update
    - apt install -y nodejs npm
  image: elixir:latest
  script:
    - cd web
    - mix local.hex --force
    - mix local.rebar --force
    - mix deps.get
    - cd assets
    - npm ci
    - npm run lint
    - cd ..
    - mix format --check-formatted
    - mix credo --strict
  stage: lint-web

Build Stage

To add a build stage to your CI, modify your .gitlab-ci.yml file as per the below:

.gitlab-ci.yml

stages:
    - build

include:
    - local: "ci/build.yml"

In the "ci" directory of your Git repository, add a file named "build.yml" and add the below content:

build.yml

build:
  image: elixir:latest
  script:
    - cd web
    - mix local.hex --force
    - mix local.rebar --force
    - mix deps.get
    - mix compile --warnings-as-errors
  stage: build

Test Stage

To add a test stage to your CI, modify your .gitlab-ci.yml file as per the below:

.gitlab-ci.yml

stages:
    - test

include:
    - local: "ci/test.yml"

In the "ci" directory of your Git repository, add a file named "test.yml" and add the below content:

test.yml

test:
  image: elixir:latest
  script:
    - cd web
    - mix local.hex --force
    - mix local.rebar --force
    - mix deps.get
    - cd assets
    - apt update
    - apt install -y nodejs npm
    - npm ci
    - npm run test
    - cd ..
    - mix ecto.setup
    - mix coveralls
  services:
    - postgres:latest
  stage: test
  variables:
    MIX_ENV: "test"
    POSTGRES_DB: epic_fantasy_forge_test
    POSTGRES_HOST: postgres
    POSTGRES_PASSWORD: "postgres"
    POSTGRES_USER: postgres

Update "test.exs" to read the database connection details from environment variables since we set them in the CI job:

text.exs

username: System.get_env("POSTGRES_USER") || "postgres",
password: System.get_env("POSTGRES_PASSWORD") || "postgres",
hostname: System.get_env("POSTGRES_HOST") || "localhost",
database: System.get_env("POSTGRES_DB") || "epic_fantasy_forge_test",

We run the coveralls command instead of test since coveralls also runs the tests. In addition to running to tests it tells us how much code the tests covered. Later in this guide we will configure a minimum code coverage percentage. Therefore the code coverage tools is part of the CI run so that we can enforce a minimum coverage percentage in the CI.

You should now have three additional stages added to your pipeline. Your .gitlab-ci.yml might now look something like this:

.gitlab-ci.yml

stages:
  - lint
  - build
  - test
  - provision
  - documentation

include:
  - local: "ci/build.yml"
  - local: "ci/documentation.yml"
  - local: "ci/lint.yml"
  - local: "ci/provision.yml"
  - local: "ci/test.yml"

Visual Studio Code Integration

ElixirLS

To add support for the Elixir language to Visual Studio Code, run the below command:

code --install-extension jakebecker.elixir-ls

As your Phoenix project is not located at the root of your Git repository, i.e. the directory which you open in Visual Studio Code, you need to manually set the Phoenix project location in Visual Studio Code settings. To do so, start by creating a .vscode directory in the root of your Git repository:

mkdir .vscode

Now create a file named "settings.json" inside this new directory and populate it with the below content:

settings.json

{
  "elixirLS.projectDir": "web"
}

Now ElixirLS should be able to find your Phoenix project in the "web" directory relative to your Git repository's root directory.

Phoenix Framework

To add support for the Phoenix Framework to Visual Studio Code, run the below command:

code --install-extension phoenixframework.phoenix

Elixir Credo

To add support for Credo to Visual Studio Code, run the below command:

code --install-extension pantajoe.vscode-elixir-credo

The Visual Studio Code Credo extension requires there to be a Credo configuration file in your project. You may need to create one if you don't have one already by running the below command in the root directory of your Phoenix project:

mix credo gen.config

ESLint

To add support for ESLint to Visual Studio Code, run the below command:

code --install-extension dbaeumer.vscode-eslint

Cleanup

Your auto-generated Phoenix project contains a lot of demo code to display the default web page of the Phoenix project. The test coverage is also currently only 27%, i.e. most of the demo code is not covered by tests.

Let's remove all of this demo code so you have an empty page instead of the demo page displayed when you run the Phoenix project. Start by replacing all content in "lib/epic_fantasy_forge_web/components/layouts/root.html.heex" with the below:

root.html.heex

<!DOCTYPE html>
<html lang="en" class="[scrollbar-gutter:stable]">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <meta name="csrf-token" content={get_csrf_token()} />
    <.live_title suffix="">
      {assigns[:page_title] || "Epic Fantasy Forge"}
    </.live_title>
    <link phx-track-static rel="stylesheet" href={~p"/assets/app.css"} />
    <script defer phx-track-static type="text/javascript" src={~p"/assets/app.js"}>
    </script>
  </head>
  <body>
    {@inner_content}
  </body>
</html>

Replace the content of "lib/epic_fantasy_forge_web/components/layouts/app.html/heex" with the below:

app.html.heex

<main>
  {@inner_content}
</main>

Completely empty the file "/lib/epic_fantasy_forge_web/controllers/page_html/home.html.heex:

home.html.heex

After applying all of the above changes the test "GET /" in page_controller_test.exs should now fail since the expected string is no longer found on the home page. This test file should be located at test/epic_fantasy_forge_web/controllers/ relative to the Phoenix project's root directory. Update the test to only check for the HTTP response code but not the content of the message:

page_controller_test.exs

test "GET /", %{conn: conn} do
    conn = get(conn, ~p"/")
    assert conn.status == 200
  end

After this cleanup our code coverage has dropped to 10.2%. Later as we add more of our own code and test that code, the coverage percentage should increase. Unfortunately the Phoenix framework generated project doesn't ship with tests that cover most of its own code.

Tip

It is not recommended to write tests for the sole purpose of covering the automatically generated Phoenix framework code. Generally for any code coming from third-parties, you should for the most part trust that code and not write automated tests for it. Automated tests should only be written for your own code. Naturally when performing integration and E2E tests you may also cover the code of some third-party libraries and this is a good thing.

Later in this guide when we add more of our own code and E2E tests, your code coverage should naturally increase.