Bootstrapping your next Elixir Phoenix project

At YND, we believe in microservices and containerization. While working on complex and scalable products for Fintech companies, we need to get things up and running quickly. For this we decided to create reusable containers that come with most of the tooling out of the box. This greatly decreases the time needed for bootstrapping. And as we all know, time is money.

One of our latest Swiss-army toolkits is YND Phoenix Bootstrap, a production-ready and maintainable template for Elixir Phoenix server. This template should help all developers to speed up the development of production-ready apps with integrations for: error tracker, log manager, CI server, database and container orchestration platform.

Why Elixir?

1. Dynamic language Elixir is a dynamic, functional language designed for building scalable and maintainable applications. It leverages the Erlang VM, known for running low-latency, distributed and fault-tolerant systems. Elixir is also successfully used in web development and the embedded software domain. An excellent macro system allows to write DSL’s which look and feel like parts of the language itself.

2. Vertical scaling Elixir code runs in isolated processes that exchange information via messages. Processes are lightweight and it’s not uncommon to have thousands of processes running simultaneously on the same machine. As all processes run in isolation, they can be garbage collected independently, reducing system-wide pauses, therefore using all machine resources as efficiently as possible.

3. Horizontal scaling Processes are able to communicate with other processes running on different machines in the same network. This provides solid foundation for distribution.

4. Fault Tolerance Elixir provides a built-in way to cope with failures and crashes, called supervisors. With this, a developer can describe how to restart parts of the system when things go wrong, going back to a known initial state that is guaranteed to work.

5. Powerful Tools A build tool, such as Mix, allows to easily create projects, manage tasks or run tests. It also integrates with the Hex package manager, which provides dependency resolution and the ability to remotely fetch packages. Another tool called IEx, which is Elixir’s interactive shell, is definitely one of the best REPL’s (Read–Eval–Print Loop) on the market. It provides features like auto-completion, debugging tools, code reloading and many more.

Enter the Phoenix

If you’re familiar with server-side MVC pattern, commonly used in popular web frameworks like Ruby on Rails or Django for Python, and you want to start using Elixir in your projects — Phoenix is the perfect choice for you. While it is a very high-performance solution, it allows developers to be extremely productive.

Since it provides an excellent set of tools, it makes writing robust web applications much easier. Not only for building web APIs, but also real-time applications using streaming technology called Channels. Considering the above, Phoenix is our weapon of choice while working with Elixir backends.

The Rest of the Gang

But in real life, even the best language paired with the best framework is not enough. Here’s a short overview of our Elixir Phoenix-related toolkit.

Jenkins

A modern development process is not possible without a server automation solution. We use Jenkins to build, test, and automate deployments of our projects. At the end of our pipeline, the binary is generated and pushed to marathon (which builds and starts the server automatically) whenever a pull request is merged to our main development branch. As a build requires some extra steps and we are heavily relying on docker containers, we had to use extra rel/Dockerfile for building the production docker image containing the binary file. We found no issues with integration.

JUnitFormatter

Our JUnit xml report exporter of choice for Elixir’s ExUnit is JUnitFormatter, which we use as a test output formatter in Jenkins. It solves output of test visualisation in Jenkins. The setup is pretty easy. In config/test.exs we define the formatter configuration, while in test/test_helper.exs we actually tell the ExUnit test runner to use JUnitFormatter.

DC/OS

The Datacenter Operating System (DC/OS) is a distributed operating system based on the Apache Mesos distributed systems kernel. It integrates several battle-tested open-source components to enable the management of multiple machines as if they were a single computer. The combination of scalability, availability, efficiency, flexibility, and usability makes DC/OS the best way to build, deploy, and manage modern applications.

Scalability

DC/OS dynamically distributes tasks and programmatically manages resource allocations, enabling your services to be both highly elastic and highly scalable.

Availability

DC/OS monitors the health of your applications and services, load balancing, restarting, or even rescheduling them when necessary to make them highly available.

Efficiency

DC/OS supports many custom schedulers, sharing resources between diverse collocated workloads including microservices, batch jobs, analytics pipelines, message queues, and big data storage.

Flexibility

DC/OS can be installed on-premise or in the public cloud, on bare metal or on virtual machines, to run containers or legacy binaries, freeing you from making infrastructure decisions based purely on platform lock-in.

Usability

DC/OS simplifies the installation, management, and monitoring of the cluster and its distributed services with a scriptable command-line interface (CLI), intuitive web interface, and easy-to-use installer.

Marathon

Marathon is a production-proven Apache Mesos framework for container orchestration, providing a REST API for starting, stopping, and scaling applications. Written in Scala, Marathon can run in a highly-available mode by running multiple copies. The state of running tasks gets stored in the Mesos state abstraction.

Graylog

Application introspection always requires some extra work. Most of the frameworks provide some basic logging and error handling utilities. In distributed systems a centralized log manager like Graylog is a must have.

Sentry

The same goes for error tracking. We think Sentry, with its official Elixir SDK, does this job very well. We use it for error tracking as it exposes a simple API to capture exceptions, automatically handle Plug Exceptions and provides a backend for the Elixir Logger. We installed it as a mix dependency, configured in config/prod.exs using env variables for hiding the sensitive data.

GelfLogger

As an Elixir Logger backend, we decided on GelfLogger which will generate Graylog Extended Log Format messages. We simply installed it as a mix dependency, and configured it using environment variables.

Credo

Credo is used for static code analysis. It makes sure code style is enforced so it’s consistent within the team. This allows programmers to focus more on solving real problems rather than manually checking the code style, which in return speeds up code review. We use the syntax check as part of the automated build process. The build fails if the developer doesn’t follow the coding guidelines enforcing fixes before merging a pull request. We use themix credo --strict command in our Jenkins script.

However, we did encounter some issues with it. After adding the library to mix.exs and installing the deps, the command works out of the box, but it complains about the code auto generated by Phoenix and those issues have to be fixed manually before integrating this tool with Jenkins pipeline. 80 characters in a line for us seems a bit too restrictive. Fortunately, it’s easy to customize the rules. All you need to do, is to generate a config file and update it to: {Credo.Check.Readability.MaxLineLength, priority: :low, max_length: 120}

Distillery

Next, we have Distillery, which allows a pure Elixir implementation of release packaging functionality for Erlang VM. Setup an easy to use single command line to manage the server process and it helps with performing migrations. The purpose of the distillery is to take something and break it down to its component parts, reassembling it into something better, more powerful. The service takes your Mix project and produces an Erlang/OTP release, a distilled form of your raw application’s components; a single package which can be deployed anywhere, independently of an Erlang/Elixir installation. No dependencies, no hassle.

PlugCheckup

For adding simple health checks to your app, we use PlugCheckup as it also easily monitors Plug based applications. We defined a /health endpoint and exposed it in our API. It gets consumed by marathon — while the configuration was provided in marathon.json

Bootstrap… Standing by

We packed all the above into a compact, ready-to-use template which is available here.

If you feel like learning more about these topics, here’s some good reads to keep you going:

What I learned migrating a Rails app to Elixir/Phoenix

State of the art in deploying Elixir/Phoenix apps

10 amazing open source Elixir/Phoenix apps

We hope this will be your kickstarter to Elixir and Phoenix. Have fun!

At YND we help companies successfully launch apps across various industries: from mobile payment, finance management, travel booking to E-commerce. In need of some brain power? Reach out to us via hello@ynd.co with questions about your projects.