HTTP Stack

A couple hundred users per host, a few domains each, security headers, HTTPS with Let’s Encrypt, WebSocket, pass-through, web backends, and .htaccess… At some point a web setup reaches a point, when there are too many feature to handle them all with just one web server software. That's why we're employing two of them back-to-back: Caddy and Apache httpd.

                 _
               (`  ).
              (     ).                 .-------.        .-------.
             _(       '`.  ----------> | caddy | -----> | httpd |
         .=(`( Internet )              '-------'        '-------'
         ((    (..__.:'-'                  |                => php (via php-fpm)
         `(       ) )                      |                => static files
           ` __.:'   )                     |                => .htaccess
               a:f--'                      |
                                           |            .--------------.
                                           '----------> | Web Backends |
                                                        '--------------'
                                                            => nodejs, python, ruby, ...
                                                            => gogs, mattermost, matrix, ...

Caddy

Caddy handles all the nitty gritty of accepting requests from browsers, making sure HTTPS works properly and passing requests through to configured web backends to serve apps based on Rust, Python, Ruby, etc. It also connects to Apache for more traditional web development needs, like PHP/WordPress.

Within Caddy each user domain gets their very own config block. On a fresh asteroid, it might look like this:

http://isabell.uber.space {
  import logs isabell
  redir https://isabell.uber.space{uri} 308
}

https://isabell.uber.space {
  root /var/www/virtual/isabell/isabell.uber.space
  import logs isabell
  import user-logs isabell

  tls {
    on_demand
  }

  encode

  handle `/*` {
    reverse_proxy `localhost:81` {
    }
  }

  # FALLBACK: show error page if domain has no backends configured
  handle {
    root /srv/http/user_has_no_backend
    import fileserver
    import assets
  }
}

By default, this block only contains a simple reverse_proxy statement processing all requests the same way: send everything to Apache httpd and report back whatever it said.

This configuration can be extended using uberspace web backend set <web-backends> commands.

[isabell@moondust ~]$ uberspace web backend add /etherpad-lite port 51922
OK: Added webbackend '/etherpad-lite' to your Asteroid

Now the handler part would look like this:

  handle `/*` {
    reverse_proxy `localhost:81` {
    }
  }

  handle `/etherpad-lite/*` {
    reverse_proxy `isabell.local.uberspace.de:51922` {
    }
  }

  handle `/etherpad-lite` {
    reverse_proxy `isabell.local.uberspace.de:51922` {
    }
  }

  # FALLBACK: show error page if domain has no backends configured
  handle {
    root /srv/http/user_has_no_backend
    import fileserver
    import assets
  }

The original configuration has been extended with a new handle/reverse_proxy section. By default, all requests are still routed to Apache, but requests intended for /etherpad are passed onto the service directly.

This enables you to get the direct, raw HTTP traffic - including the original headers and WebSocket connections.

The curious isabell.local.uberspace.de domain resolves to an IP in the Shared Address Space - in our networking setup. Feel free to read up on it, if you'd like to know more!

Other backend types like --apache or ones specific to a domain work in a very similar way. They are documented over in the web backends article.

Apache Httpd

Apache serves requests for more traditional development needs - like PHP and applications requiring .htaccess files. As you saw in the above examples, it is reverse–proxied using Caddy - just like other web backends. Since we try to handle as much as possible within Caddy, which makes our httpd configuration rather short:

<Directory /var/www/virtual/isabell/html>
  AllowOverride AuthConfig FileInfo Indexes Limit Options=ExecCGI,Includes,Indexes,MultiViews,SymLinksIfOwnerMatch
  Options +Includes
</Directory>

<Directory /var/www/virtual/isabell/isabell.uber.space>
  AllowOverride AuthConfig FileInfo Indexes Limit Options=ExecCGI,Includes,Indexes,MultiViews,SymLinksIfOwnerMatch
  Options +Includes
</Directory>

<VirtualHost *>
  ServerName https://isabell.uber.space
  ServerAdmin isabell@uber.space

  DocumentRoot /var/www/virtual/isabell/isabell.uber.space

  AllowEncodedSlashes NoDecode

  ErrorLog /readonly/isabell/logs/apache/error_log_apache
  LogLevel notice

  RewriteEngine On

  <FilesMatch "\.php$">
    SetHandler "proxy:unix:/var/lib/php-fpm/isabell/php-fpm.sock|fcgi://php-fpm-isabell"
  </FilesMatch>

  <Proxy "fcgi://php-fpm-isabell" max=10>
  </Proxy>
</VirtualHost>

As mentioned earlier httpd only handles .htaccess (=> AllowOverride), static files (=> DocumentRoot) and PHP (=> SetHandler).

Since all of those are rather ordinary duties for httpd, the configuration is rather simple in this case. Additionally, since everything else is handled within Caddy, the only dynamic parts of this configuration is the list of domains in ServerName - and the username, of course.

Some probably noticed that static files are handled within httpd, instead of Caddy. Even though Caddy easily outperforms httpd when it comes to serving static files, we need to use httpd in this case. Many applications like Wordpress rely on .htaccess files to rewrite URLs or protect certain directories from being accessed. Since those files can only be parsed by httpd, Caddy does not qualify for the job.

To serve files directly with Caddy, you can use a STATIC backend, though.

Acknowledgments

The ASCII art cloud has been copied from asciiart.eu. The artist goes by the name a:f. Thank you!