Recommended docker compose + nginx setup /w websocket support

Technical assistance
1

Infos:

  • Used Zammad version: 6.4.1
  • Used Zammad installation type: docker-compose
  • Operating system: Debian 11
  • Browser + version: Firefox

I’m trying to setup a docker compose install and saw two options, one of which I couldn’t get to work, the other I could, but without support for websockets. This could be founded in confusion on my part or a missing recommended install that supports websockets.

First, I tried modifying the given docker-compose.yml to use only my own nginx server, not the embedded zammad-nginx. For that, I removed that service from the docker-compose.yml, and adapted the provided nginx.conf to my needs. Notably, I replaced root /opt/zammad/public with a copy of the public folder in the repo on my servers file system.
This didn’t work since zammad-railsserver expects to be able to write the javascript files to that folder and expects nginx to serve them. I do not understand how this works in the default docker-compose.yml, since they only share the zammad-storage volume which wasn’t even mapped to the right folder, but it did work once I gave up on this path:

So next I used two nginx servers - my existing one which I already use as the reverse proxy, and the embedded zammad-nginx. This worked fine and was easy to setup. Until I was informed by errors in the log and this forum post that this setup would NOT work in the future since the nginx reverse proxy does not proxy websockets.

Now this implies the proper solution would be to get my first attempt running:
Instead of forwarding 443 straight to 8080, don’t use the embedded zammad-nginx and instead handle everything in one nginx server, proxying straight to 3000 and notably, to 6042 for websocket support.

How would I go about this? For now setup 2 seems to work, but the above mentioned forum posts indicates this doesn’t work for mobile right now, and won’t work for desktop in the future. But I can’t find a proper docker compose+nginx setup that actually supports websockets and works.

Thank you for your assistance.

2

Well, I did end up solving all my issues.

For once, I am still using two nginx servers. I’m merely using the embedded nginx server to handle the railsserver configuration, as I deemed that too complex to keep external with a docker-compose setup.

Second, I did reference the embedded nginx config to forward the two websockets in my external nginx config the relevant servers (directly to the legacy websockets server, but through the embedded nginx for the railsserver, though I think I could skip the embedded nginx with this one, too).

Finally, I had odd troubles with "CSRF token verification failed!” on login, varied in severity and when it would start occurring on my various trials, but I was able to solve it by adding another header to assure the rails valid_authenticity_token that the connection is proxied with SSL:
proxy_set_header X-Forwarded-Ssl on;
I believe this may be a new requirement in rails and should probably be added to the documentation. In fact, that header IS set in the apache2 example, but NOT for nginx.

This is my final config of the external proxy NGINX with letsencrypt (note that I am configuring SSL options and logging externally, since this is part of a larger setup):

upstream zammad-nginx {
  server localhost:8080;
}

upstream zammad-railsserver {
  server localhost:3000;
}

upstream zammad-websocket {
  server localhost:6042;
}

server {
    listen 80;
    server_name -----------;
    server_tokens off;

    location / {
        return 301 https://$host$request_uri;
    }

    location /.well-known/acme-challenge/ {
        root /var/www/-----------;/html;
    }
}

server {
    listen 443 ssl;
    server_name -----------;
    server_tokens off;

    ssl_certificate     /etc/letsencrypt/live/-----------/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/-----------/privkey.pem;

    add_header Strict-Transport-Security "max-age=63072000" always;

    client_max_body_size 50M;

    # legacy web socket server
    location /ws {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header CLIENT_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_read_timeout 86400;
        proxy_pass http://zammad-websocket;
    }

    # action cable
    location /cable {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $http_host;
        proxy_set_header CLIENT_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_read_timeout 86400;
        proxy_pass http://zammad-nginx;
    }

    location / {
        proxy_set_header Host $http_host;
        proxy_set_header CLIENT_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_read_timeout 180;
        proxy_pass http://zammad-nginx;

        gzip on;
        gzip_types text/plain text/xml text/css image/svg+xml application/javascript application/x-javascript application/json application/xml;
        gzip_proxied any;
    }
}

For this to work, I also slightly modified the docker-compose.yml:

  zammad-railsserver:
    ports:
      - "3000:3000"
  zammad-websocket:
    ports:
      - "6042:6042"

Notably, the nginx container is staying the same. You should probably put this in the docker-compose.override.yml as recommended.
I think that should be it. If I’m missing something, feel free to ask. I’ll be wanting to keep my setup up-to-date anyway.

3

This configuration opens ports 3000 and 6042 on the host, usually on 0.0.0.0.
This means others can bypass your proxy - this is potentially very insecure and should not be done at all.

4

Not in my reverse proxy case, no.
I of course have a firewall that only leaves port 80 and 443 open (plus VPN).
These are handled by the first nginx server that handles all other requests on the server as well.
This then forwards to the 3000, 6042 and 8080 ports that I expose in zammads docker-compose.yml.
This does leave it open for other users on the server, yes, this in my case acceptable. Maybe it is possible to forward these websocket requests through the embedded nginx as well, but that has no functional difference afaik.

5

Your statement is only correct if you have configured your firewall in a correct way (which you have, that’s good).

My note regarding the compose configuration however is still correct

...
   ports:
      - "2001:3000"
...

results in

tcp   LISTEN 0      4096         0.0.0.0:2001      0.0.0.0:*    users:(("docker-proxy",pid=2786746,fd=4)) ino:749805527 sk:1001 cgroup:/system.slice/docker.service <->                                                                                                                                                                                                                        
tcp   LISTEN 0      4096            [::]:2001         [::]:*    users:(("docker-proxy",pid=2786753,fd=4)) ino:749805532 sk:1002 cgroup:/system.slice/docker.service v6only:1 <->

on network level. If you need it on the local machine only anyway, consider doing it like so:

...
   ports:
      - "127.0.0.1:2001:3000"
...

That will, even if one fucks up their firewall configuration, not cause in the service being available to the public network interface

tcp   LISTEN 0      4096       127.0.0.1:2001      0.0.0.0:*    users:(("docker-proxy",pid=2787069,fd=4)) ino:749815054 sk:2001 cgroup:/system.slice/docker.service <->

In my opinion a much better approach, if you only need to access these ports locally on the same host machine any way.

6

Well there’s something new to learn every day. Thank you a lot, will immediately apply that to all my docker setups.

7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.