All logins suddenly being denied, no errors

Technical assistance
1

Infos:

  • Used Zammad version: latest zammad-docker-compose:zammad image
  • Used Zammad installation source: docker-compose
  • Operating system: Debian 9.5
  • Browser + version: Chrome

Actual behavior:

  • A few hours ago I was signed in to the portal viewing tickets. I clicked on another ticket and got the message " Opps… I’m sorry, but you have insufficient rights to open this Ticket.". I clicked back on the original ticket and got that message. I tried a bunch of tickets and got that message. I went to settings to look at the user list and it hanged with a loading message. I signed out and tried to sign back in and failed. I checked with users and other support reps and they couldn’t sign in.

I ran ‘docker-compose down’ and then ran ‘docker-compose up’ (without the -d flag to watch logs) and the server came up without errors. No one could sign in. No errors in the log.

Verified LDAP was accessible and up from the docker host.
Pulled up the console and verified I could see a list of users. Reset my password. Still no ability to log in.

Ran ‘docker pull zammad/zammad-docker-compose:zammad’ to update the image. Ran ‘docker-compuse up’. Still no ability to sign in. No error messages.

Logs from clicking on a ticket and getting denied:

zammad-railsserver_1    | I, [2018-12-12T03:00:00.673915 #1-69858014805460]  INFO -- : Started PUT "/api/v1/taskbar/3651" for ::ffff:172.21.0.10 at 2018-12-12 03:00:00 +0000
zammad-railsserver_1    | I, [2018-12-12T03:00:00.688336 #1-69858014805460]  INFO -- : Processing by TaskbarController#update as JSON
zammad-railsserver_1    | I, [2018-12-12T03:00:00.688530 #1-69858014805460]  INFO -- :   Parameters: {"key"=>"Ticket-702", "client_id"=>"123", "callback"=>"TicketZoom", "state"=>{"ticket"=>{}, "article"=>{"form_id"=>"547883448"}}, "params"=>{"ticket_id"=>"702", "init"=>true}, "prio"=>49, "notify"=>false, "active"=>true, "attachments"=>[], "updated_at"=>"2018-12-11T21:23:45.567Z", "id"=>"3651"}
zammad-railsserver_1    | I, [2018-12-12T03:00:00.691603 #1-69858014805460]  INFO -- : Completed 401 Unauthorized in 3ms (Views: 0.4ms | ActiveRecord: 0.0ms)
zammad-nginx_1          | 172.21.0.11 - - [12/Dec/2018:03:00:00 +0000] "PUT /api/v1/taskbar/3651 HTTP/1.1" 401 40 "https://support.--redacted--.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"
frontend_1              | nginx.1    | support.--redacted--.com x.x.x.x - - [12/Dec/2018:03:00:00 +0000] "PUT /api/v1/taskbar/3651 HTTP/2.0" 401 29 "https://support.--redacted--.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"
zammad-railsserver_1    | I, [2018-12-12T03:00:00.712607 #1-69858014811160]  INFO -- : Started PUT "/api/v1/taskbar/3582" for ::ffff:172.21.0.10 at 2018-12-12 03:00:00 +0000
zammad-railsserver_1    | I, [2018-12-12T03:00:00.724343 #1-69858014811160]  INFO -- : Processing by TaskbarController#update as JSON
zammad-railsserver_1    | I, [2018-12-12T03:00:00.724616 #1-69858014811160]  INFO -- :   Parameters: {"key"=>"Ticket-773", "client_id"=>"123", "callback"=>"TicketZoom", "state"=>{"ticket"=>{}, "article"=>{"form_id"=>"463948144"}}, "params"=>{"ticket_id"=>"773", "init"=>true}, "prio"=>36, "notify"=>false, "active"=>false, "attachments"=>[], "updated_at"=>"2018-12-12T00:58:37.753Z", "id"=>"3582"}
zammad-railsserver_1    | I, [2018-12-12T03:00:00.727107 #1-69858014811160]  INFO -- : Completed 401 Unauthorized in 2ms (Views: 0.4ms | ActiveRecord: 0.0ms)
zammad-nginx_1          | 172.21.0.11 - - [12/Dec/2018:03:00:00 +0000] "PUT /api/v1/taskbar/3582 HTTP/1.1" 401 40 "https://support.--redacted--.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"
frontend_1              | nginx.1    | support.--redacted--.com x.x.x.x - - [12/Dec/2018:03:00:00 +0000] "PUT /api/v1/taskbar/3582 HTTP/2.0" 401 29 "https://support.--redacted--.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36"
2

what’s happens to me is while the ldap sync is running and syncing all the users, I occasionally get a postgres deadlock timeout while syncing some random user, and zammad disables that user.

If you can log in as a different user with admin rights, look at settings -> Integration -> LDAP and scroll down to the LDAP log and see if you see an error with the locked user name.

If so you can fix it by clicking start new.

Also check your postgres logs for a deadlock timeout error. I believe it’s a bug when ldap sync and some other scheduled job runs at the same time, like ldap sync starts running, then while it’s still running the email fetch job starts running.

3

We had three admin users. None of them can log in. I tried using the admin console to reset passwords and ensure they were admins (Console — Zammad System Documentation documentation) to no avail.

4

I finally got in.

I messed around with User.create() until I was able to successfully create a new account and grant it admin privileges.

After I signed in, I noticed my normal account was disabled. I enabled it and signed in. A few minutes later I lost access again.

Turns out the LDAP plugin for some unknown reason disabled a bunch of accounts even though they are valid in AD.

When I run a manual sync, I can actually see errors in the console indicating that a bunch of accounts were disabled (including the admin accounts) and a few conflicts because the LDAP sync created new accounts and failed due to an email address already existing on a disabled account.

5

I guess there’s something wrong with the mappings, there was a thread on here a week or two ago about the required zammad fields, if one of the required fields is empty zammad doesn’t create the account, and maybe if the account is already created it disables it?

I don’t know though, but you found the cause for the accounts being disabled and that;s 80% of the issue solved, yo just got to fix the last 20%.

6

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.