Now Viewing

Blog

basic-freebsd-webserv.md

Setting up a basic web service on FreeBSD 15

Does paying for CI have you feeling down? Do you find yourself wanting to publish a static website or a simple application, but feeling overwhelmed by the sheer amount of options and tools running around these days?

In this post I will detail a simple continuous integration and deployment method that requires basically zero additional infrastructure. You don’t need a dedicated build server, a git hosting provider, or a fleet of job runners to get basic automated deployments.

This guide is aimed at deploying an Elixir based webserver to a FreeBSD host. It covers the basics of host configuration, hosting an Elixir application as a service with rc.d, and a basic git-hook based build and deployment script.

This is the same mechanism I use to deploy this blog, and it should be suitable for a lot of different hobby applications however I would recommand something more reproducable - especially for the target machine setup - for anything professional. The main advantage here is that there are essentially 0 required external services (excluding the various ACME cert providers that Caddy uses).

Note: The git hook based deployment method will work fine for linux targets too, but the specifics will differ. I like to host my personal projects on FreeBSD, since I enjoy tinkering with FreeBSD servers.

Host Configuration

First, provision a FreeBSD 15 host somewhere. I have had decent experiences with Vultr, but there are many other reputable providers that support FreeBSD guests.

After provisioning a fresh host, preferably installed on ZFS, I like to tighten down some of the default security settings. It’s also a good idea to set up the firewall at your hosting provider (not on the VPS itself) to only allow SSH traffic from your home address. Just remember, if your home IP address is dynamic, it will often change at the most inconvenient times, and you’ll need to be able to access the firewall configuration to change the allowed IP.

PF

I start from a basic PF configuration for the firewall, something like:

# /etc/pf.conf
# Other hosting providers may have different names 
#   for the default interface on virtual machines.
#   Check your interfaces with ifconfig if uncertain.
ext_if = "vtnet0"

set block-policy drop

set skip on lo

# Block all traffic in by default
block in on $ext_if

# Allow all traffic out
pass out on $ext_if proto { tcp, udp, icmp } from any to any keep state

# Allow SSH Traffic in
pass in on $ext_if proto tcp from any to any port ssh keep state

# Allow HTTP and HTTPS Traffic in
pass in on $ext_if proto tcp from any to any port { http https } keep state
pass in on $ext_if proto udp from any to any port { http https }

Then you only need to enable the firewall. There are two ways to do this in FreeBSD - you can either edit the system configuration direction in /etc/rc.conf or you can use the sysrc tool. It’s generally recommended these days to use the sysrc tool, which can be done as follows:

# sysrc pf_enable=YES
# sysrc pf_rules="/etc/pf.conf"

All this does is set the specified variables in /etc/rc.conf, but sysrc has some validation that just editing the file with vim misses.

After enabling pf, you can start the service with service pf start. If PF was already running, you can reload the rules with pfctl -f /etc/pf.conf.

At this point, your server should be set up to only allow in ssh, http and https traffic. If you want to lock it down a little further, it’s a good idea to change the port sshd runs on in /etc/ssh/sshd_config (remember to change the allowed port in your firewall configuration too).

PF is capable of a lot more, and it’s worth spending more time tinkering with this configuration. With a bit more work PF can be set up to do things like rate limiting connections and maintaining tables of banned IPs, but I think that is a bit beyond the scope of this guide. PF rules are evaluated from top to bottom, unless the quick keyword is used, and the last matching rule is the one that wins. As long as you keep that in mind, configuring PF isn’t too bad.

Setting up Caddy

I like to use Caddy as a reverse proxy these days because it handles getting and rotating SSL certificates automatically. There are some disadvantages though, with one of the larger ones being that all the SSL certificates acquired through Caddy are added to the Certificate Transparency logs, which are pretty frequently crawled by bots. It’s not a major issue, but be aware that subdomains you add specifically in Caddy will effectively be added to a public log.

NOTE: In more recent versions of Caddy you can largely avoid this privacy concern by using a wildcard certificate.

Let’s start by installing dependencies:

pkg install caddy
pkg install security/portacl-rc

You’ll note that recent versions of caddy include a long message about appropriate UIDs and portacl when you install them. I think this is worth doing, so I’m installing both caddy and security/portacl-rc. Actually setting up portacl is not too complicated, and it’s pretty well detailed in the instructions from Caddy.

Adding some things to rc.conf, again using sysrc:

# sysrc portacl_users="www"
# sysrc portacl_user_www_tcp="http https"
# sysrc portacl_user_www_udp="https"
# sysrc portacl_enable="YES"

# sysrc caddy_user="www"
# sysrc caddy_group="www"
# sysrc caddy_enable="YES"

After setting up caddy, you can configure the caddyfile with something like this:

# /usr/local/etc/caddy/Caddyfile

wandering-grove.net, www.wandering-grove.net {
        reverse_proxy localhost:4000

        log {
                output file /var/log/caddy/access.log
                
                format console
        }
}

This is a minimal configuration which will route all traffic to an application running locally on port 4000. Caddy is pretty powerful and there are a lot of additional options listed in their Caddyfile docs, but this is sufficient for our needs. Note that it includes request logging which will land in /var/log/caddy/access.log.

Service Requirements

Now that we’ve got Caddy and PF configured, we need to get this host set up to build and run Elixir applications. In traditional deployment systems we would typically have a separate build machine, and that is important for most real services, but for blogs and personal projects there’s no need to actually get a separate machine. With that in mind, I set this up to build directly on the host.

Elixir

On FreeBSD, you can just do:

# pkg install elixir

Which as of today will give you Elixir 1.17.3 and Erlang 26.2.5, neither of which are particularly new but they will still work. If you need or want a more recent version of Elixir, the best path is typically to use something like asdf with a plugin for Elixir/Erlang/etc. This would make things a bit more complicated, so I will not be exploring that in this guide.

git

We’ll also want git, which can be installed with:

# pkg install git-lite

The full version of git includes a lot of extra things we don’t need, but git-lite will fit our needs.

Database

If you’re hosting a full Phoenix web application you will also need a database. Phoenix supports SQLite so that is a valid option for simple applications, but I tend to be a fan of PostgreSQL so I went ahead and installed that:

# pkg install postgresql18-client postgresql18-server

Then enable the server:

# sysrc postgresql_enable=YES

and initialize the database:

# service postgresql initdb

This will create the initial postgres database in the default location - /var/db/postgres/data18/. Next it’s a good idea to make sure PostgreSQL is relatively locked down, when I am running the database and the website on the same host I usually do this in pg_hba.conf, which controls authentication:

# /var/db/postgres/data18/pg_hba.conf
# You need this 'local' auth with 'trust' to set up any initial super user accounts in postgres,
#   but should comment it out later if you don't want to allow 'trust', which allows any user access
#   without credentials.

# Don't require any authentication for unix-domain access (using a unix socket under /tmp)
local    all             all                                     trust

# Require password authentication for network access from localhost
host     all             all             127.0.0.1/32            scram-sha-256
host     all             all             ::1/128                 scram-sha-256

# Reject all network access from anything besides localhost
host     all             all             0.0.0.0/0               reject
host     all             all             ::/0                    reject

Make sure there aren’t other enable authentication methods, and then in postgresql.conf find and change listen_addresses like this:

# /var/db/postgres/data18/postgresql.conf
listen_addresses = 'localhost'

This will set up postgres to require authentication on local connections, listen only on localhost, and reject all connections from any host that isn’t localhost.

Next start up postgres:

# service postgresql start

And then log into postgres:

# su postgres -c psql
psql (18.4, server 18.3)
Type "help" for help.

postgres=# 

and using the postgres prompt, create a user:

postgres=# CREATE ROLE my_blog_user WITH LOGIN CREATEDB PASSWORD 'a_strong_password';
postgres=# CREATE DATABASE my_blog_database OWNER my_blog_user;

Whew, ok, that’s postgres taken care of. We’ll let our deployed application will take care of any needed database migrations. Make sure to write down that password somewhere safe.

Service Accounts

Next let’s create a user for our actual application to run as. This user account will also handle deployment, so I’m creating it with git-shell as the login shell. This will come up later when we wire up git-based deployment.

Add a service user

root@www:~ # pw useradd blog -m -c "Blog User" -s /usr/local/bin/git-shell 

Create an ssh key on your regular computer and add the public key to the service user:

# cd /home/blog
# vim .ssh/authorized_keys

   -- Put your key in this file --

# chown blog:blog .ssh/authorized_keys 
# chmod 600 .ssh/authorized_keys 

If you limited the allowed users in sshd_config, make sure to add this new blog user to the list.

Service Configuration

Now it’s time to start wiring things together.

To wire up service administration on freebsd, you’ll need to create a custom rc.d configuration for your service. This is a pretty good skeleton, but the exact configuration will depend on where you ultimately deploy the released service. I am planning to keep everything under /home/blog, so that is the path included in the blog_command in this:

# /usr/local/etc/rc.d/blog
# PROVIDE: blog
# REQUIRE: epmd
# AFTER: epmd
# KEYWORD: shutdown
#
# Add the following line to /etc/rc.conf to enable the Wandering Grove Blog
#
# blog_enable="YES"

. /etc/rc.subr

name="blog"
rcvar="blog_enable"

load_rc_config $name

: ${blog_enable:="NO"}
: ${blog_user:="blog"}
: ${blog_database_url:=""}
: ${blog_database_pool_size:=10}
: ${blog_secret_key_base:=""}
: ${blog_port:=4000}
: ${blog_hostname:=""}
: ${blog_database_ipv6:=false}

blog_command="/home/blog/release/bin/wandering_grove"

cpidfile="/var/run/${name}/${name}.pid"
pidfile="/var/run/${name}/${name}d.pid"
logfile="/var/log/${name}.log"

command=/usr/sbin/daemon
command_args="-P ${pidfile} -p ${cpidfile} -f -S -t ${name}-super -T ${name} ${blog_command} start"

start_precmd="${name}_prestart"

blog_prestart()
{
    # Use install to create the runfile directory if it doesn't already exist
    if [ ! -d /var/run/${name} ]; then
        install -d -o ${blog_user} /var/run/${name}
    fi

    export DATABASE_URL="${blog_database_url}"
    export POOL_SIZE="${blog_database_pool_size}"
    export SECRET_KEY_BASE="${blog_secret_key_base}"
    export PORT="${blog_port}"
    export PHX_HOST="${blog_hostname}"
    export ECTO_IPV6="${blog_database_ipv6}"
    export PHX_SERVER=true
}

run_rc_command "$1"

There’s a lot going on here, but let’s look at a few pieces. These:

: ${blog_enable:="NO"}
: ${blog_user:="blog"}
: ${blog_database_url:=""}
: ${blog_database_pool_size:=10}
: ${blog_secret_key_base:=""}
: ${blog_port:=4000}
: ${blog_hostname:=""}
: ${blog_database_ipv6:=false}

are rc.d configuration variables and define default values. We’ll override these later in rc.conf.

The section under blog_prestart() is responsible for a few things including creating the runfile directory, which sorts PIDs, and exporting environment variables which will be used by the blog application itself.

This script uses daemon to actually run the deployed application, and uses the -t and -T flags to tag log messages that get sent to syslog. We’ll set up routing for those in a minute.

blog_command is the actual binary that daemon will start. If you are deploying your application somewhere other than /home/blog, make sure to point at the released binary.

I’m pretty sure this just touches the surface of what rc.d can do, and it’s probably not the best way to handle some of this, but I haven’t had much time to explore the rc.subr documentation further.

Next we need to set up the blog configuration in rc.conf. Just like postgres and pf, we can do this with sysrc:

# sysrc blog_database_url="ecto://my_blog_user:my_blog_password@localhost/my_blog_database"
# sysrc blog_enable=YES
# sysrc blog_hostname="wandering-grove.net"
# sysrc blog_port=4000
# sysrc blog_secret_key_base="..."

Make sure to set blog_secret_key_base to a secret actually generated with mix phx.gen.secret. Additionally, if you encounter difficulties, make sure that your phoenix application is loading the correct environment variables in runtime.exs.

Now wire up logging. This is mostly a matter of creating two configuration files - one for newsyslog, which handles log rotation - and one for syslog itself:

# /usr/local/etc/newsyslog.conf.d/blog.conf
/var/log/blog.log  640  7  *  @T00  Z
# /usr/local/etc/syslog.d/blog.conf
!blog
*.*    /var/log/blog.log

Automatic Deployment

Init Bare Repo:

git init --bare /home/blog/repo.git

Note - if you’re doing this as root, you’ll need to make sure to chown and chmod the repository and the authorized_keys files to their appropriate permissions before this will work.

Create a git hook at /home/blog/repo.git/hooks/post-receive:

#!/bin/sh

WORKTREE="/home/blog/worktree"
BLOG_REPO="/home/blog/repo.git"
RELEASE="/home/blog/release"

BRANCH="main"

set -e

while read oldrev newrev ref
do
  if [ "$ref" = "refs/heads/$BRANCH" ];
  then
    echo "Ref $ref received. Deploying ${BRANCH} on server..."
    git --work-tree="${WORKTREE}" --git-dir="${BLOG_REPO}" checkout -f ${BRANCH}

    echo "Building"

    unset GIT_DIR
    cd "${WORKTREE}" || exit 1

    export MIX_ENV=prod

    mix deps.get --only prod
    mix compile
    mix assets.setup
    mix assets.deploy

    echo "Releasing"
    mix release --overwrite --path "${RELEASE}"

    echo "Restarting Service"
    sudo -n /usr/sbin/service blog restart

    exit 0
  else
    echo "Ref $ref received. Doing nothing, only ${BRANCH} may deploy here."
  fi
done

Allow the blog user to restart the service in visudo:

blog ALL=(root) NOPASSWD: /usr/sbin/service blog restart

That’s pretty much it. There was a long diversion to get tailwind css v4 building on FreeBSD - I have included a link to the solution I found below - but I plan to pull tailwindcss out of this site and redo the styling in plain css so I’m not including it here.

At this point, pushing a reference to main on your server like git push publish main should connect to the server as the blog user, push the references, and run the post-receive hook which will build and deploy your application. You’ll see the logs directly in the terminal you push from.

Note that there is no automatic rollback in this if something breaks, and there’s no automatic testing. For a small project, though, it works quite well.

References