CSRF Token (Apache)

Infos:

  • Used Zammad version: 6.1.0-1700492762.96ac5a60.bionic
  • Used Zammad installation type: package
  • Operating system: Ubuntu 20.04.6 LTS
  • Browser + version: Newest Chrome

Expected behavior:

I can put Tags in Tickets and refresh it

Actual behavior:

After putting a tag in a ticket the whole webpage refreshes and the tag is not set.
In F12 browser console I can see “401 unauthorized”
In the Zammad production.log I can see “CSRF token verification failed”:

I, [2024-03-21T09:17:12.693458#1657-6568960]  INFO -- : Started PUT "/api/v1/taskbar/87615" for [IP] at 2024-03-21 09:17:12 +0100
I, [2024-03-21T09:17:12.697559#1657-6568960]  INFO -- : Processing by TaskbarController#update as JSON
I, [2024-03-21T09:17:12.697636#1657-6568960]  INFO -- :   Parameters: {"key"=>"Ticket-8251", "callback"=>"TicketZoom", "state"=>{"ticket"=>{}, "article"=>{"form_id"=>"943761786"}}, "params"=>{"ticket_id"=>"8251", "init"=>true}, "prio"=>41, "notify"=>false, "active"=>false, "attachments"=>[], "updated_at"=>"2024-03-21T06:45:01.413Z", "id"=>"87615"}
I, [2024-03-21T09:17:12.706092#1657-6568960]  INFO -- : CSRF token verification failed
I, [2024-03-21T09:17:12.706232#1657-6568960]  INFO -- : CSRF token verification failed! (Exceptions::NotAuthorized)
app/controllers/application_controller/prevents_csrf.rb:36:in `verify_csrf_token'
app/controllers/application_controller/has_download.rb:17:in `block (4 levels) in <module:HasDownload>'
app/controllers/application_controller/has_download.rb:16:in `block (3 levels) in <module:HasDownload>'
app/controllers/application_controller/has_download.rb:15:in `block (2 levels) in <module:HasDownload>'
app/controllers/application_controller/handles_transitions.rb:16:in `handle_transaction'
I, [2024-03-21T09:17:12.707088#1657-6568960]  INFO -- : Completed 401 Unauthorized in 9ms (Views: 0.1ms | ActiveRecord: 2.1ms | Allocations: 5284)
I, [2024-03-21T09:17:13.777762#1657-6568860]  INFO -- : Started POST "/api/v1/signshow" for [IP] at 2024-03-21 09:17:13 +0100

Steps to reproduce the behavior:

Just add a tag.

Our zammad is running only in internal networks using a internal signed certificate(via Microsoft PKI) so all clients in our network accept it. That works without any problems.

This CSRF-Error happens since I changed the webserver from nginx to apache2 because we tried SSO. (I didnt got SSO to work and deleted the sso-lines but still would like to keep apache if possible)

Our /etc/apache2/sites-enabled/zammad.conf is copied from here: zammad/contrib/apache2/zammad_ssl.conf at develop · zammad/zammad · GitHub

Of course did I try to research how to fix this. And alot people said they fixed it using THE “zammad-le-ssl.conf”-file. This file seems unexistent on our machine.
I had also a look in sites-available.

Here our censored apache2 zammad.conf file:

ServerName [name].[domain]
LogLevel debug

# security - prevent information disclosure about server version
ServerTokens Prod

 <VirtualHost *:80>
    ServerAlias [name]
    Redirect permanent / https://[name].[domain]
</VirtualHost>

<VirtualHost *:443>
    ServerAlias [name]
    SSLEngine on
    SSLCertificateFile "/etc/ssl/certs/zammad.pem"
    SSLCertificateKeyFile "/etc/ssl/private/zammad.key"

    ## don't loose time with IP address lookups
    HostnameLookups Off

    ## needed for named virtual hosts
    UseCanonicalName On

    ## configures the footer on server-generated documents
    ServerSignature Off

    ProxyRequests Off
    ProxyPreserveHost On

    <Proxy 127.0.0.1:3000>
      Require local
    </Proxy>

    RequestHeader set X_FORWARDED_PROTO 'https'
    RequestHeader set X-Forwarded-Ssl on

    ProxyPass /assets !
    ProxyPass /favicon.ico !
    ProxyPass /apple-touch-icon.png !
    ProxyPass /robots.txt !
    # legacy web socket server
    ProxyPass /ws ws://127.0.0.1:6042/
    # action cable
    ProxyPass /cable ws://127.0.0.1:3000/cable
    ProxyPass / http://127.0.0.1:3000/

    # change this line in an SSO setup
    RequestHeader unset X-Forwarded-User

    # Use settings below if proxying does not work and you receive HTTP-Errror 404
    # if you use the settings below, make sure to comment out the above two options
    # This may not apply to all systems, applies to openSuse
    #ProxyPass /ws ws://127.0.0.1:6042/ "retry=1 acque=3000 timeout=600 keepalive=On"
    #ProxyPass /cable ws://127.0.0.1:3000/cable "retry=1 acque=3000 timeout=600 keepalive=On"
    #ProxyPass / http://127.0.0.1:3000/ "retry=1 acque=3000 timeout=600 keepalive=On"

    DocumentRoot "/opt/zammad/public"

    <Directory />
        Options FollowSymLinks
        AllowOverride None
    </Directory>

    <Directory "/opt/zammad/public">
        Options FollowSymLinks
        Require all granted
    </Directory>
	
</VirtualHost>

Im thankful for any help. I cant see my mistake here.

Hi @jimwendrich. What’s set for http_type and fqdn in your system? Any other issues, because I cannot imagine that only tags are affected.

What worked for me was to configure the above settings mentioned by @fliebe92, via Ruby console.

Check if you can run the following:

zammad run rails c

If you can’t, you will need to install the full ruby components.
https://docs.zammad.org/en/latest/admin/console.html#start-zammad-s-rails-console

The other method, via zammad environement variables, didn’t work for me.
https://docs.zammad.org/en/latest/appendix/configure-env-vars.html#performance-tuning

Hi,

@stormrider123 run rails c works.

@fliebe92 here is the censored output:
image

Btw by [Name].[Domain] I mean name.domain.de. As in the apache config.

We can work just fine in zammad except adding tags.
Also we experienced that sometimes adding tags works.
Yesterday overviews showed nothing to me. I pressed F5 and they showed their tickets.

Right now we experienced that a college is able to add tags without any problems. And if I try to add a tag it automaticly reloads the whole webpage without adding a tag.

failed for me:

I, [2024-03-22T09:44:55.091855#1657-13871480]  INFO -- : Completed 200 OK in 74ms (Views: 2.8ms | ActiveRecord: 12.0ms | Allocations: 13485)
I, [2024-03-22T09:44:57.049766#1657-13990040]  INFO -- : Started POST "/api/v1/tags/add" for [my IP] at 2024-03-22 09:44:57 +0100
I, [2024-03-22T09:44:57.053691#1657-13990040]  INFO -- : Processing by TagsController#add as JSON
I, [2024-03-22T09:44:57.053745#1657-13990040]  INFO -- :   Parameters: {"object"=>"Ticket", "o_id"=>8197, "item"=>"DevExpress Änderung"}
I, [2024-03-22T09:44:57.062527#1657-13990040]  INFO -- : CSRF token verification failed
I, [2024-03-22T09:44:57.062664#1657-13990040]  INFO -- : CSRF token verification failed! (Exceptions::NotAuthorized)
app/controllers/application_controller/prevents_csrf.rb:36:in `verify_csrf_token'
app/controllers/application_controller/has_download.rb:17:in `block (4 levels) in <module:HasDownload>'
app/controllers/application_controller/has_download.rb:16:in `block (3 levels) in <module:HasDownload>'
app/controllers/application_controller/has_download.rb:15:in `block (2 levels) in <module:HasDownload>'
app/controllers/application_controller/handles_transitions.rb:16:in `handle_transaction'
I, [2024-03-22T09:44:57.069569#1657-13990040]  INFO -- : Completed 401 Unauthorized in 16ms (Views: 0.1ms | ActiveRecord: 2.2ms | Allocations: 5788)
I, [2024-03-22T09:44:58.251496#1657-13872040]  INFO -- : Started POST "/api/v1/signshow" for [my IP] at 2024-03-22 09:44:58 +0100
I, [2024-03-22T09:44:58.256124#1657-13872040]  INFO -- : Processing by SessionsController#show as JSON
I, [2024-03-22T09:44:58.256179#1657-13872040]  INFO -- :   Parameters: {"fingerprint"=>"-1011084360"}

works for a college:

I, [2024-03-22T09:56:15.555334#1659-15156320]  INFO -- : Completed 200 OK in 9ms (Views: 0.2ms | ActiveRecord: 2.2ms | Allocations: 5294)
I, [2024-03-22T09:56:16.465365#1659-15156080]  INFO -- : Started GET "/api/v1/tag_search?term=ces" for [colleges IP] at 2024-03-22 09:56:16 +0100
I, [2024-03-22T09:56:16.469208#1659-15156080]  INFO -- : Processing by TagsController#search as JSON
I, [2024-03-22T09:56:16.469261#1659-15156080]  INFO -- :   Parameters: {"term"=>"ces"}
I, [2024-03-22T09:56:16.477745#1659-15156080]  INFO -- : Completed 200 OK in 8ms (Views: 0.1ms | ActiveRecord: 2.0ms | Allocations: 4925)
I, [2024-03-22T09:56:19.123557#1659-16622180]  INFO -- : Started POST "/api/v1/tags/add" for [colleges IP] at 2024-03-22 09:56:19 +0100
I, [2024-03-22T09:56:19.127390#1659-16622180]  INFO -- : Processing by TagsController#add as JSON
I, [2024-03-22T09:56:19.127448#1659-16622180]  INFO -- :   Parameters: {"object"=>"Ticket", "o_id"=>8197, "item"=>"CES"}
I, [2024-03-22T09:56:19.183988#1659-16622180]  INFO -- : Completed 201 Created in 56ms (Views: 0.1ms | ActiveRecord: 28.0ms | Allocations: 17285)
I, [2024-03-22T09:56:20.006851#1659-16622220]  INFO -- : Started GET "/api/v1/tickets/8197?full=true&_=1711097744047" for [colleges IP] at 2024-03-22 09:56:20 +0100
I, [2024-03-22T09:56:20.010779#1659-16622220]  INFO -- : Processing by TicketsController#show as JSON
I, [2024-03-22T09:56:20.010821#1659-16622220]  INFO -- :   Parameters: {"full"=>"true", "_"=>"1711097744047", "id"=>"8197"}
I, [2024-03-22T09:56:20.018895#1657-15314680]  INFO -- : Completed 200 OK in 26ms (Views: 2.2ms | ActiveRecord: 4.9ms | Allocations: 17957)
I, [2024-03-22T09:56:20.042476#1659-16622220]  INFO -- : Completed 200 OK in 32ms (Views: 4.6ms | ActiveRecord: 4.7ms | Allocations: 19338)
I, [2024-03-22T09:56:20.168043#1659-16622220]  INFO -- : Started PUT "/api/v1/tickets/8197?all=true" for [colleges IP] at 2024-03-22 09:56:20 +0100
I, [2024-03-22T09:56:20.175375#1659-16622220]  INFO -- : Processing by TicketsController#update as JSON
I, [2024-03-22T09:56:20.175479#1659-16622220]  INFO -- :   Parameters: {"number"=>"398192", "title"=>"Zammad - Tags hinzufügen geht nicht mehr", "group_id"=>"4", "owner_id"=>"12", "customer_id"=>9, "state_id"=>"2", "priority_id"=>"2", "updated_at"=>"2024-03-22T08:56:19.155Z", "preferences"=>{}, "organization_id"=>nil, "pending_time"=>nil, "ort_hardware"=>"", "id"=>"8197", "all"=>"true"}
I, [2024-03-22T09:56:20.417571#1659-16271680]  INFO -- : Started GET "/api/v1/tickets/8197?all=true&auto_assign=true&_=1711097744048" for [colleges IP] at 2024-03-22 09:56:20 +0100
I, [2024-03-22T09:56:20.424634#1659-16271680]  INFO -- : Processing by TicketsController#show as JSON
I, [2024-03-22T09:56:20.424899#1659-16271680]  INFO -- :   Parameters: {"all"=>"true", "auto_assign"=>"true", "_"=>"1711097744048", "id"=>"8197"}
I, [2024-03-22T09:56:20.480122#1657-13872040]  INFO -- : Completed 200 OK in 91ms (Views: 5.6ms | ActiveRecord: 15.7ms | Allocations: 53670)
I, [2024-03-22T09:56:20.558405#1659-16271680]  INFO -- : Completed 200 OK in 133ms (Views: 10.0ms | ActiveRecord: 35.3ms | Allocations: 92741)
I, [2024-03-22T09:56:20.586677#1659-16622220]  INFO -- : Completed 200 OK in 411ms (Views: 5.0ms | ActiveRecord: 75.3ms | Allocations: 149258)
I, [2024-03-22T09:56:20.658655#1659-16622220]  INFO -- : Started DELETE "/api/v1/upload_caches/097807059" for [colleges IP] at 2024-03-22 09:56:20 +0100
I, [2024-03-22T09:56:20.662406#1659-16622220]  INFO -- : Processing by UploadCachesController#destroy as JSON
I, [2024-03-22T09:56:20.662497#1659-16622220]  INFO -- :   Parameters: {"id"=>"097807059"}
I, [2024-03-22T09:56:20.671247#1659-16622220]  INFO -- : Completed 200 OK in 9ms (Views: 0.1ms | ActiveRecord: 1.8ms | Allocations: 5011)
I, [2024-03-22T09:56:22.001397#1659-16622180]  INFO -- :   Parameters: {"all"=>"true", "auto_assign"=>"true", "_"=>"1711095544608", "id"=>"8197"}
I, [2024-03-22T09:56:22.018606#901-64800]  INFO -- : 2024-03-22T09:56:22+0100: [Worker(host:huszammad pid:901)] Job TicketUserTicketCounterJob [04b8973e-ee9c-4ba7-b998-36541583fde5] from DelayedJob(default) with arguments: [9, 2419] (id=5241350) (queue=default) COMPLETED after 0.0269
I, [2024-03-22T09:56:22.022754#901-64800]  INFO -- : 2024-03-22T09:56:22+0100: [Worker(host:huszammad pid:901)] Job TransactionJob [658b13ff-6f1c-4549-9296-5c7cbcf8eab3] from DelayedJob(default) with arguments: [{"object"=>"Tag", "object_id"=>705, "user_id"=>2419, "created_at"=>{"_aj_serialized"=>"ActiveJob::Serializers::TimeWithZoneSerializer", "value"=>"2024-03-22T08:56:19.152369495Z"}, "type"=>"create", "_aj_symbol_keys"=>["object", "object_id", "user_id", "created_at", "type"]}, {"interface_handle"=>"application_server", "_aj_symbol_keys"=>["interface_handle"]}] (id=5241353) (queue=default) RUNNING
I, [2024-03-22T09:56:22.039796#901-64800]  INFO -- : 2024-03-22T09:56:22+0100: [Worker(host:huszammad pid:901)] Job TransactionJob [658b13ff-6f1c-4549-9296-5c7cbcf8eab3] from DelayedJob(default) with arguments: [{"object"=>"Tag", "object_id"=>705, "user_id"=>2419, "created_at"=>{"_aj_serialized"=>"ActiveJob::Serializers::TimeWithZoneSerializer", "value"=>"2024-03-22T08:56:19.152369495Z"}, "type"=>"create", "_aj_symbol_keys"=>["object", "object_id", "user_id", "created_at", "type"]}, {"interface_handle"=>"application_server", "_aj_symbol_keys"=>["interface_handle"]}] (id=5241353) (queue=default) COMPLETED after 0.0170
I, [2024-03-22T09:56:22.041637#901-64800]  INFO -- : 2024-03-22T09:56:22+0100: [Worker(host:huszammad pid:901)] Job TransactionJob [8802c6c4-1182-4d1f-9391-77865896e79d] from DelayedJob(default) with arguments: [{"object"=>"Ticket", "object_id"=>8197, "user_id"=>2419, "created_at"=>{"_aj_serialized"=>"ActiveJob::Serializers::TimeWithZoneSerializer", "value"=>"2024-03-22T08:56:20.420020200Z"}, "type"=>"update", "changes"=>{"updated_by_id"=>[12, 2419], "_aj_symbol_keys"=>[]}, "_aj_symbol_keys"=>["object", "object_id", "user_id", "created_at", "type", "changes"]}, {"interface_handle"=>"application_server", "_aj_symbol_keys"=>["interface_handle"]}] (id=5241354) (queue=default) RUNNING
I, [2024-03-22T09:56:22.088574#1659-16622180]  INFO -- : Completed 200 OK in 87ms (Views: 4.3ms | ActiveRecord: 15.4ms | Allocations: 53319)

I hope that I coppied the important lines. Not that easy to see because ~10 agents work right now.

I really tried to see what is different between us but it makes no sense.
e.g. in my local google chrome Im able to add tags, but a 2nd college on his local chrome isnt. That 2nd college uses tags the most.

Is there anything else I could check?

Is there a difference in how you both are accessing your Zammad instance? E.g. you’re using the fqdn, your colleague is using something else…

e.g. talking about my last information that I am able to add tags in my local google chrome and my college isnt able to in his local google chrome - we both access Zammad using the fqdn.
like https://name.domain.de

I also use zammad in a terminal server session (Windows Server 2022 via RDP). There also in Google Chrome and fqdn I cant add tags.
(I didnt wrote that yet because it really doesnt make any sense to me. But I guess with this information we could say that its not caused by the user).

Testing also seems easier via my local chrome vs my colleges local chrome

Make sure this isn’t a caching issue of your browser, if your browser is the only one that has this issue.

Force reload via CTRL + Shift + R will discard all existing cache.
If the issue still persists and only affects one / some browsers but not all, check what the difference is.

Thanks for your input too.

It persists after CTRL + Shift + R
(reloading took longer than f5 so the shortcut should have worked)

We try to investigate the differences more next week

Maybe you have an extension installed in your Chrome browser that somehow interferes with it… no idea right now.

Only chrome addon my college uses is from our anti virus software.
We disabled it, used the shortcut again, but zammad still refreshes when adding a tag to a ticket.
I will look about chrome versions or different browser next week

So my coworker suddenly was able to add tags using the same chrome browser as before (without any visible updates)
Now he got some weeks off from work so I tested it again on my client.

3 hours ago in a locally installed chrome 123.0.6312.86 (newest) and edge 123.0.2420.65 (newest) I wasnt able to add a tag.
In Firefox 124.0.1 (newest) it worked just fine.

Now (3hours later) the same chrome browser on the same version works.
Edge still doesnt work. Firefox still works.

In the browser F12 console I can see a (new?) error on every Browser. I dont know if it has anything todo with this CSRF thing.

Firefox console: