Swag Nginx Reverse Proxy

Infos:

  • Used Zammad version: 4.0.x
  • Used Zammad installation type: (source, package, docker-compose, …) Package
  • Operating system: Debian 10 Buster kernel 4.19 (VM) on Kubuntu 20.04.2 LTS kernel 5.4 host
  • Browser + version: FireFox Mozilla Firefox 88.0.1

Expected behavior:

Please note that I’m using a subdomain that I own and control and my.domain.com is NOT a valid real domain I’m attempting to use. I’ve simply swapped mine here since it is irrelevant.

Actual behavior:

Nextcloud in particular requires certain Headers be set in the directive on the Swag container’s Nginx proxy config, it’s possible that Zammad does too, but I’ve yet to find documentation on what those are. This is different from the server itself, which is running on another physical server from the Swag container.

Steps to reproduce the behavior:

  • Installed Zammad 4.0.x via apt-get package on Debian 10 VM running inside Kubuntu 20.04 host system on Intel NUC x86_64 (Host B).

  • Zammad can be accessed via http://HostB:8080 on local network, basic first admin account was setup and system can be used, however without https/SSL/TLS

  • Configured Swag with the following zammad.subfolder.conf configuration file:

    location /support {
    return 301 $scheme://$host/support/;
    }

    location ^~ /support/ {
    proxy_set_header Host $http_host;
    proxy_set_header CLIENT_IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    set $upstream_app hostB.lan;
    include /config/nginx/proxy.conf;
    resolver LAN-DNS-IP valid=30s;
    set $upstream_port 8080;
    set $upstream_proto http;
    proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    rewrite /support(.*) $1 break;
    }

    location ^~ /support/ws {
    include /config/nginx/proxy.conf;
    resolver LAN-DNS-IP valid=30s;
    set $upstream_app hostb.lan;
    set $upstream_port 6042;
    set $upstream_proto ws;
    proxy_pass $upstream_proto://$upstream_app:$upstream_port/ws;
    rewrite /support/ws(.*) $1 break;
    }

  • For comparison here’s a working Swag nginx proxy config for Gitea server (docker container on hostA):

location /gitea {
return 301 $scheme://$host/gitea/;
}

location ^~ /gitea/ {
include /config/nginx/proxy.conf;
resolver 127.0.0.11 valid=30s;
set $upstream_app gitea;
set $upstream_port 3000;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
rewrite /gitea(.*) $1 break;
}

Hypotheses

I’ve a few hunches about what may be going on or need to change, but I’m thinking that the problem exists somewhere in the Swag proxy config as LAN direct connect over http works fine to Zammad.
My guesses are:

  • Swag proxy config needs something else
  • There may be a CORS problem
  • There’s a trust issue on the origin server for Zammad where it doesn’t know to accept the request from the Swag proxy

Anyone have any thoughts on what’s wrong and how to fix it? I don’t mind grabbing some more relevant logs and such if you need those too.

I’m glad to be a part of this online community and look forward to being able to use Zammad and help out anyone else based on what I learn from hosting and using it.

Hi, we have zammad working through Nginx reverse proxy (not swag).

Our config is pretty simple.

The important part is

server {
  set $forward_scheme http;
  set $server         "localzammad-dns";
  set $port           8080;

  listen 80;
listen [::]:80;

listen 443 ssl http2;
listen [::]:443;

  server_name zammad.domain.reachable.from.wan;

And a bit further is where we do websockets:

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_http_version 1.1;

add_header       X-Served-By $host;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Scheme $scheme;
proxy_set_header X-Forwarded-Proto  $scheme;
proxy_set_header X-Forwarded-For    $remote_addr;
proxy_set_header X-Real-IP          $remote_addr;
proxy_pass       $forward_scheme://$server:$port;

Perhaps that helps you figure out what you need for your SWAG config.

Big thanks Floris, I’ll see what I can do, it helps to know that your implementation works. Swag is very similar to any Nginx reverse proxy so this ought to help.

Update on progress:
I still haven’t fully fixed this configuration in my own implementation but I’m getting closer.

What the web looks like in the Swag or Nginx reverse proxied site currently (with error, it never advances beyond that):

And the console with the errors that show as 404 errors, but I think it’s because it’s coming through Swag as HTTP 2 and zammad is expecting HTTP 1.1

And for comparison here’s the web console view of the working zammad webserver from a LAN PC:

Finally, some notes for understanding where I’m at:

  • Zammad server was moved to port 81 from port 80 directly in the zammad server config
  • Zammad port was changed due to using a different server on http://localzammad:80. The other server has web services that are on host machine, keeping in mind the zammad server is installed on a VM guest running on the same host. The other server is a container in docker.
  • Perhaps stating the obvious, but the LAN PC browsing to the LAN address works fine. Eventually I’ll block that connection, but I want the secure http WAN connection to be stable first.
  • Swag works fine for gitea, heimdall, nextcloud and so on. If I’m to change the configs, I mean to change only the zammad server’s proxy. Part of the problem I’ve had implementing changes to make my config more like Floris’s is that it seems to apply the changes to all the proxied hosts and configs, breaking them all simultaneously, including getting the http error 521 from zammad as well.
  • Because those changes break both current production servers and connectivity to zammad, I may not have it fully right. Don’t worry too much about outages on the production servers as I’m the only person using them right now.
  • The actual problem may be that my nginx reverse proxy/swag container needs to send the correct directive to allow communication over the ws protocol.

Thanks for anyone who has any ideas, feel free to share them here!

Hi @Zeronaut,

for me it looks like the problem is your subfolder “/support”. Subdomain should not be a problem, but it seems subfolder does.

If your Zammad installation is in /support also the asstes should be in /support/assets/… but your screenshots show, that /assets is requested.

I would suggest to use Zammad in the root folder of your subdomain.

Best Regards,
Bernhard

Okay I have to step in.
Subdirectories are not supported by Zammad.

Use the root directory.
No workarounds possible without hacks (again, not supported)

1 Like

Thanks @MrGeneration that clears this up. I’ll see what I can do with a subdomain instead. These will be eventually routed to the same network and I hope that doesn’t cause any other problems.

1 Like

Progress update:
After banging my head on the wall a bit yesterday (and honestly looking into alternatives) I was able to assimilate the good advice everyone has given thus far and identify some of the problems I was having.

Before jumping to the screenshots of the partial success, a few details and discussion:

  1. The reason my swag config kept busting up my other servers is entirely logical and I’ve figured out how to mitigate it while troubleshooting this issue. SWAG being Secure Web Access Gateway functions like any gateway where the traffic comes in through it as a gateway but also bottleneck. With any secure setup you want to limit traffic to only authorized use within it. So changes to the bottleneck that effected everything in turn broke everything. In my case this was a change that will be temporary, but needed at the moment - I had allowed an nginx config block that forced all traffic (particularly over port 80) into the https port 443. After commenting this out, my existing servers worked and further I was able to add the subdomain support.my.domain/ for use with the zammad origin server.
  2. The Mitigating strategy is that the nginx config update can be run by only touching the swag container via the command on the host machine: sudo docker restart swag following any config changes.
  3. I needed a better understanding about how websockets work which this video helped me with: Youtube Juriy Bura Proxying Websockets with NGINX
  4. Using that understanding and the examples you provided @Floris and suggestions from @bernhardk and @MrGeneration ONLY (that is removing some of the template/boilerplate config from Swag’s built-in suggestions) I was able to get this to work.
  5. The final step will be getting the connection to also use https with the SSL certificate on the Swag container. I suspect this will require adding a bit of the Swag nginx site-conf template’s directives back in, but not the ones that are redundant or break functionality.

Now, congrats to those of you who’ve commented and helped me with this project, here’s the screenshots of things so far:

I am really looking forward to being able to use this server in production. I’m working to provide more affordable IT services through my company to smaller organizations (startups, non-profits, NGO’s, etc.) Zammad provides a great tool for part of that stack. So thanks to the developers like @MrGeneration

1 Like

Just in case I read and understand correctly, in case you’re planning to provide Zammad as a service for customers, please have a look at Zammads Policies just to be extra sure:
https://zammad-foundation.org/policy/

1 Like

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.