DIY Server Part 2: Serve Your Site With httpd

Posted on 2020-02-25

This is the second part in my series of how to set up your own server on OpenBSD. This guide will cover how to use httpd to serve your website and acme-client with Let's Encrypt for your TLS certificates.

Before You Read

This guide assumes that you have installed OpenBSD and set up doas as we did in the previous guide. You should also make sure that whichever domain name you want to use is pointing at the IP address of your server.

If a line starts with #, it is meant to be run with root privileges. If it starts with $, it is to be run as a regular user. This is just a visual indicator, so do not actually type these symbols! To run a single command as root in OpenBSD, you can simply add doas at the start of the command, like so:

$ vi /etc/httpd.conf       <--- This will run as a regular user
$ doas vi /etc/httpd.conf  <--- This will run with root privileges

If you have to execute multiple commands with root privileges, you can run doas -s to switch to a root shell:

$ vi /etc/httpd.conf       <--- This will run as a regular user
$ doas -s                  <--- This will switch to a root shell (note the "#"!)

# vi /etc/httpd.conf       <--- This will be run with root privileges
# rcctl reload httpd       <--- This will still be run with root privileges
# exit                     <--- This will exit the root shell

$ vi /etc/httpd.conf       <--- This will run as regular user again

Now that we have the basics covered, let's get to our webserver!

Introduction

At its simplest, a webserver such as OpenBSD's httpd serves files from your server to the outside world. Let's say that you have written your own website using HTML and CSS; a user that is opening your website would basically just be asking the server "Hey, can you give me the file(s) for this URL?" and the webserver would respond by sending the relevant files to your user.

To get httpd to do this for us, we need to tell it where the files for our website are and how to serve them. In the case of httpd, we do this in its configuration file httpd.conf, located in the /etc directory.

That's not all there is to the setup though: To serve the files securely, we want to also set up TLS by using acme-client. TLS ("Transport Layer Security") is a protocol which is used to encrypt the traffic between your users and your server. When an URL starts with https://..., it is using TLS or its predecessor, SSL. In addition to just asking for a file, in this case your user would also be asking for a certificate that is meant to prove the authenticity of the server and set up the encrypted connection. For a more correct description, you can read this article.

Now we will get to how we do all this!

Configuring httpd

OpenBSD provides many example configuration files in /etc/examples, including an example for our httpd.conf. Since we will need to execute a bunch of commands as root, we will switch to a root shell, copy the example file to the right place and adjust it to our needs later:

$ doas -s
# cp /etc/examples/httpd.conf /etc

For editing the configuration file, we will use the vi editor, as this editor comes pre-installed with OpenBSD. If you are jumping into vi for the first time you may be very confused and you may not even know how to exit it again, so use this site as a reference! In my opinion it is worth learning, there are only few basic commands that you really need (how do I add things, how do I delete things, how do I save and quit) and you can use the arrow keys for navigation. The rest will come with time :)

If this is too much for you right now though, you can also install a different editor such as nano, which may be more intuivive for beginners. It can be installed by executing # pkg_add nano.

Now let's start editing our httpd.conf:

# vi /etc/httpd.conf

I will first give you the full configuration as we will use it and then walk you through what each part of it does. This is what the httpd.conf looks like for this website:

# $OpenBSD: httpd.conf,v 1.20 2018/06/13 15:08:24 reyk Exp $

server "earthroot.city" {
        listen on * port 80
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
        location * {
                block return 302 "https://$HTTP_HOST$REQUEST_URI"
        }
}

server "earthroot.city" {
        listen on * tls port 443
        tls {
                certificate "/etc/ssl/earthroot.city.fullchain.pem"
                key "/etc/ssl/private/earthroot.city.key"
        }
        root "/htdocs/earthroot.city"
}

At this point I have to add that httpd serves from within a chroot at /var/www. So httpd only sees files in the /var/www directory, meaning that if we write /my_site/tomatos in our httpd.conf, this would correspond to the full path /var/www/my_site/tomatos. If you don't understand this yet, I will illustrate it with an example in a bit!

As we can see, this configuration file consists of two blocks. Let's take each of them apart a few lines at a time!

First block: Serve acme-challenge and redirect to HTTPS

  1. We are defining a server called earthroot.city.
server "earthroot.city" { 
        ...
}
  1. This server listens on any network interface (That's what * means) on port 80.
listen on * port 80
  1. If somebody requests any file (*) at the location /.well-known-acme-challenge/, set the root to the /acme directory of our server and strip two levels from the request.
location "/.well-known/acme-challenge/*" {
        root "/acme"
        request strip 2
}

So if somebody requests http://earthroot.city/.well-known/acme-challenge/example_file, the server will remove /.well-known/acme-challenge/ and serve the part thats left, in this case example_file from the /acme directory of our webserver.

To come back to the chroot thing I mentioned above; the /acme path in the httpd.conf would correspond to the full path /var/www/acme, since httpd operates from within /var/www. If you list all the files and folders at /var/www, you will see the acme folder there, which should illustrate how the chroot works:

# ls /var/www
acme      bin       cache     cgi-bin   conf      etc       htdocs    logs      run       tmp       usr
  1. The last part of the first block does the following:
    If you access any location (location *) besides the one we defined above, block the request and redirect to the https:// URL of whatever was requested.
location * {
        block return 302 "https://$HTTP_HOST$REQUEST_URI"
}

So if I access http://earthroot.city/example_post/, it would redirect me to https://earthroot.city/example_post/.

To summarize the first block: The server listens for any http connections on port 80. If anyone requests the acme-challenge (which is needed for TLS), serve it from /var/www/acme. If anyone requests anything else, redirect them to the https version of the site.

Second block: Serving our Website with HTTPS

  1. As above, we are defining a server called earthroot.city:
server "earthroot.city" {
        ...
}
  1. This server listens on any interface (*) with TLS on port 443:
listen on * tls port 443
  1. The tls {...} block contains information for establishing the TLS connection, namely the location of our certificate and our private key:
tls {
        certificate "/etc/ssl/earthroot.city.fullchain.pem"
        key "/etc/ssl/private/earthroot.city.key"
}
  1. Set the root for any requests to be /htdocs/earthroot.city:
root "/htdocs/earthroot.city"

So if a user requests https://earthroot.city/index.html, the server would look for index.html in the folder /var/www/htdocs/earthroot.city.

Configuring acme-client

As you saw above, we noted paths for our TLS certificate and key in our httpd.conf. Right now, these files don't exist yet though, so now we will go about getting them!

The configuration file for acme-client is called acme-client.conf. As with the httpd.conf, we will just take the example file in /etc/examples/ and adapt it to our needs.

# cp /etc/examples/acme-client.conf /etc
# vi /etc/acme-client.conf

We will change the example.com to the domain name of our site, in my case earthroot.city. Since we will just serve to earthroot.city for now, I will remove the alternative names {...} line. Our acme-client.conf will then look like this:

#
# $OpenBSD: acme-client.conf,v 1.2 2019/06/07 08:08:30 florian Exp $
#
authority letsencrypt {
        api url "https://acme-v02.api.letsencrypt.org/directory"
        account key "/etc/acme/letsencrypt-privkey.pem"
}

authority letsencrypt-staging {
        api url "https://acme-staging-v02.api.letsencrypt.org/directory"
        account key "/etc/acme/letsencrypt-staging-privkey.pem"
}

domain earthroot.city {
        domain key "/etc/ssl/private/earthroot.city.key"
        domain full chain certificate "/etc/ssl/earthroot.city.fullchain.pem"
        sign with letsencrypt
}

In this configuration file, we can see two authorities: letsencrypt and letsencrypt-staging. In our domain earthroot.city {...} section, we wrote sign with letsencrypt, meaning that we will get out certificates from the letsencrypt authority.

This is how you want things to be configured once your site is all done, but if you are still testing a bunch of different things and different domain names, you should use letsencrypt-staging instead, because you may otherwise run into rate limiting. letsencrypt is meant for stable production systems, so if you keep requesting certificates for the same domain over and over again, they may block you for a week. The letsencrypt-staging authority is meant to be used when you need to test your setup, as you may be doing right now. To learn more about this, check out the official site of Let's Encrypt on this topic.

Getting it all to run

Now that we are done with our configuration, let's check that all we wrote is correct. Both httpd and acme-client provide an option to check for errors in the configuration file:

# httpd -n
# acme-client -n earthroot.city

httpd will tell you that no certificates were found, but this is okay. Just make sure that it confirms your Configuration is OK. If not, check the line number it mentions for any errors. acme-client will just spit out its configuration and nothing else.

After we know that everything is okay with our setup, let's create the directory for the files we want to serve:

# mkdir -p /var/www/htdocs/earthroot.city

Now let's add a little "Hello World":

# echo "Hello from httpd!" >> /var/www/htdocs/earthroot.city/index.html

Then we can set httpd to start at boot and start it up:

# rcctl enable httpd && rcctl start httpd

And fetch our certificates:

# acme-client -v earthroot.city

Now reload httpd one last time to load the certificates:

# rcctl reload httpd

If you now visit your website in a browser, you should see the text "Hello from httpd!". Congratulations, you made it!

Switching from letsencrypt-staging to letsencrypt

If you did end up running with letsencrypt-staging first and then want to switch to letsencrypt, running acme-client on your domain may just return that your certificate is still valid. To force acme-client to renew your certificate and switch to letsencrypt, just run acme-client once with the -F (Force) option, like this:

# acme-client -Fv earthroot.city
# rcctl reload httpd

Automatic Certificate Renewal

Since certificates expire after some time, we want our server to automatically periodically check whether our certificate is still valid, and renew it if it is nearing its expiration date. The manual page of acme-client (man acme-client) tells us how to do this:

First, open crontab. cron is a system service that periodically does tasks that you can specify in various files, crontab is one of these files. To learn more about cron and crontab, you can have a look at their manual pages. :) (Warning: Crontab uses the vi editor by default, so be ready to open up the cheat sheet!)

# crontab -e

Now insert the text that is mentioned in the acme-client manual in a new line at the bottom of the crontab (change earthroot.city to your own domain name of course):

...

0       *       *       *       *       sleep $((RANDOM \% 2048)) && acme-client earthroot.city && rcctl reload httpd

Then, reload the cron service:

# rcctl reload cron

What's next?

The example page we used to verify whether it's working was pretty minimal, but your page does not have to be! The foundation we built here can easily be expanded upon. You can write your own HTML, CSS and JavaScript files to build your site, or use a static site generator like I do for generating the files that make up earthroot.city.

You have a lot of choices here on how to proceed and perhaps I will make a guide on how I set up Zola to create the website you are reading right now, but that is for the future. There are many other fantastic (arguably better and easier to set up) static site generators like Hugo, Jerkyll or Pelican, as well as countless guides to match. Look around and start building your site!

httpd can't just serve static files though, it can also serve dynamic content as for example in the case of Nextcloud, which I will likely cover in a future guide. Until then, feel free to contact me if you have any questions or suggestions, and I will do my best to answer!