CSRF token verification failed! // already did proxy_set_header X-Forwarded-Proto and replace $scheme by https

Technical assistance
1

Infos:

  • Used Zammad version: 6.2.0-1712728678.1fb595f2.jammy
  • Used Zammad installation type: package
  • Operating system:
  • ubuntu 22.04

Expected behavior:

I can use caddy in front to get ssl

Actual behavior:

when I access th epage while having http senable din settings I get CSRF token verification failed!

Steps to reproduce the behavior:

put caddy in front of nginx which is configured accordingly to:
https://docs.zammad.org/en/latest/getting-started/configure-webserver.html

nginx config:

This is the nginx config for zammad, with explicit settings for HTTPS forwarding.

upstream zammad-railsserver {
server 127.0.0.1:3000;
}

upstream zammad-websocket {
server 127.0.0.1:6042;
}

server {
listen 80;
listen [::]:80;

server_name support.domain.tld; # Replace with your FQDN

server_tokens off;

root /opt/zammad/public;

access_log /var/log/nginx/zammad.access.log;
error_log  /var/log/nginx/zammad.error.log;

client_max_body_size 50M;

location ~ ^/(assets/|robots.txt|humans.txt|favicon.ico|apple-touch-icon.png) {
    expires max;
}

location /ws {
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header CLIENT_IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https; # Ensure this is set to https
    proxy_set_header X-Forwarded-Ssl on; # Indicate SSL connection
    proxy_read_timeout 86400;
    proxy_pass http://zammad-websocket;
}

location /cable {
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header CLIENT_IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https; # Ensure this is set to https
    proxy_set_header X-Forwarded-Ssl on; # Indicate SSL connection
    proxy_read_timeout 86400;
    proxy_pass http://zammad-railsserver;
}

location / {
    proxy_set_header Host $http_host;
    proxy_set_header CLIENT_IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https; # Ensure this is set to https
    proxy_set_header X-Forwarded-Ssl on; # Indicate SSL connection
    proxy_read_timeout 300;
    proxy_pass http://zammad-railsserver;
    gzip on;
    gzip_types text/plain text/xml text/css image/svg+xml application/javascript application/x-javascript application/json application/xml;
    gzip_proxied any;
}

}

2

Your vHost is http. This can hardly be right.

3

the idea is that I put caddy in front of nginx. but caddy is on another host so I cannot use the port 3000 from caddy directly (it only listens on localhost). maybe you cna tell me how to change that, then I can go that route.

4

Read the documentation.

5

I did. and adjusted the ocnfig accoring to this:

This usually affects systems with more than one proxy server only. For this to function you may have to tell your web server directly which connection type was used.

Warning

Do not use below options if you’re unsure, they may technically be a security issue!

The following options expect HTTPs connections which should be your goal.

nginx:
Within your virtual host configuration, locate both directives proxy_set_header X-Forwarded-Proto and replace $scheme by https.

I tried this config now, I dont see what I can adjust more.

upstream zammad-railsserver {
    server 127.0.0.1:3000;
}

upstream zammad-websocket {
    server 127.0.0.1:6042;
}

server {
    listen 80;
    listen [::]:80;
    server_name support.domain.tld;
    server_tokens off;
    root /opt/zammad/public;
    access_log /var/log/nginx/zammad.access.log;
    error_log /var/log/nginx/zammad.error.log;

    client_max_body_size 50M;

    location ~ ^/(assets/|robots.txt|humans.txt|favicon.ico|apple-touch-icon.png) {
        expires max;
    }

    location /ws {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header CLIENT_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_read_timeout 86400;
        proxy_pass https://zammad-websocket;
    }

    location /cable {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header CLIENT_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_read_timeout 86400;
        proxy_pass https://zammad-railsserver;
    }

    location / {
        proxy_set_header Host $http_host;
        proxy_set_header CLIENT_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_read_timeout 300;
        proxy_pass https://zammad-railsserver;
        gzip on;
        gzip_types text/plain text/xml text/css image/svg+xml application/javascript application/x-javascript application/json application/xml;
        gzip_proxied any;
    }
}
6

the caddy in front of it is serving the page on https / 443

7

I’ve setup Caddy and Zammad. No issues and no additional configuration required. Everything worked as expected.

8

@Sal
how did you set it up?
for me it also works when I use http in the backend but that is not compatible with SAML-SSO! so I need to define https in Zammad so it understands that users are actually connecting via ssl. as soon as I do that, login fails.

9

I didn’t have to make any changes in Zammad and Caddy is just a normal reverse proxy setup:

zammad_url.com {
reverse_proxy Zammad_IP
}

10

do you run nginx on the zammad host? or do you run caddy directly on the zammad host?

can you share your exact config for nginx and caddy?

11

it works to login when I set http-type in zammad to http (not https) and it works over https BUT SSO will not work like that.

12

I’m not running SSO, so I’m not sure I can help there.

Also running on Apache instead of NGINX.

13

take a look.
This CSRF problem has been haunting you from the very beginning, but you are not getting to the bottom of the problem.

This error occurs when a user has not closed his tab after logging off or something else.

replace $scheme by https is not a panacea. which is written e.g. here or in the documentation.

https://zammad.panic.at/help/de-de/56-zammad/182-was-tun-bei-csrf-token-verification-failed

You have to build a mechanism in your application to avoid this error.

Best wishes
Beste Grüße

14

The CSRF functionality is a security functionality for HTTPs connections.
It is not the applications fault that people do not follow documentation (which is there in that order for over 2,5 years by now which says: install, configure web server, continue with first steps).

Any one installing Zammad from a third party documentation that does not include this step has “bad luck”. Sorry, but not our problem people seek other sources.

It’s not suggested to use Zammad via HTTP for (I guess) obvious reasons… right?
The documentation even explicitly mentions these CSRF token errors with a possible workaround if detection goes wrong (which is a web server issue btw):
https://docs.zammad.org/en/latest/getting-started/configure-webserver.html

Correct. Do you think this was done to bug people? The contrib file we provide is technically functioning on many installations. There are exclusions and I may re-visit it at some point to verify that this is still the way to do, but the way I’m being “asked” here is not how you motivate me to do things.

Little fun fact, $scheme works -without- any issues on over 1800 installations that I am aware of. Maybe that helps you to understand the scope.

That is not correct at all. Caching of the login page which is there several days until you login may be a situation this occurs too (that’s solvable by a simple reload and working as desired as of now). 99% of these problems are configuration issues.

15

Don’t get me wrong, I didn’t mean that you have no right. Internet is full of why this error occurs and it can be fixed by L8. And it happens again and again with many other applications.

But Zammad is not just any application. It is a helpdesk tool, you have a layman L8 who just wants to report his problem through this channel, and if maybe he wants to report this error at login, he can’t get through. Because he has never heard of CSRF Token Missmatch.

This is not a problem of the application, but the login mask of the WebGUI that consists of JS could go through these errors, e.g. perform a tab/page reload for users.

We already have the correct configurations from the original documentation. But still comes to these CSRF errors, after loggingoff if tab is not updated, or if users often leave their tab open for a long time, or build helpdesk home page as auto tab start etc.

Have a nice weekend

16

There. You have a suggestion for a possible improvement. That’s great! (No irony, for real)

As it’s technically no bug, it’s the best to actually put this into a feature request in my opinion:

Your suggestion may improve Zammad or PO says “no” and hopefully provides an answer of why.