Add comment as Ticket owner and Agent (not customer) to ticket should work.
Actual behavior:
I get following JS error in the console of multiple browsers. Service restarts and so on doesn’t change anything. Upgrade from 3.1.x to 3.3.x doesn’t help either
I can’t find any issue. In logs everything seems to work well. The PUT Request for the missing Article I could identify. So it seems the client is sending the update to the backend.
I, [2020-06-15T15:59:50.746526 #24751-70028961841480] INFO – : Started PUT “/api/v1/taskbar/30141” for xxx.xxx.xxx.xxx at 2020-06-15 15:59:50 +0200
I, [2020-06-15T15:59:50.753992 #24751-70028961841480] INFO – : Processing by TaskbarController#update as JSON
I, [2020-06-15T15:59:50.754142 #24751-70028961841480] INFO – : Parameters: {“key”=>“Ticket-9137”, “client_id”=>“123”, “callback”=>“TicketZoom”, “state”=>{“ticket”=>{}, “article”=>{“body”=>"[FILTERED]", “from”=>“xxxxxx”, “ticket_id”=>“9137”, “content_type”=>“text/html”, “sender_id”=>“1”, “type_id”=>“10”, “form_id”=>“965776769”}}, “params”=>{“ticket_id”=>“9137”, “init”=>true}, “prio”=>5, “notify”=>false, “active”=>false, “attachments”=>[], “updated_at”=>“2020-06-15T07:24:09.351Z”, “id”=>“30141”}
I, [2020-06-15T15:59:50.765014 #24751-70028961841480] INFO – : Enqueued UserDeviceLogJob (Job ID: e6a7c911-daf3-4f1a-a427-c0c0d1603223) to DelayedJob(default) with arguments: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.75 Safari/537.36”, “xxx.xxx.xxx.xxx”, 7, “-1486906726”, “session”
I, [2020-06-15T15:59:50.800230 #24751-70028961841480] INFO – : Completed 200 OK in 46ms (Views: 0.6ms | ActiveRecord: 8.3ms)
the client / frontend hang’s in update state and no new article is created.
Ticket.find_by(id:‘9137’).article_ids
=> [38011]
rails returns only the original ticket article but not the new one, as I expected.
i can’t find the error - has anyone an idea where to look next?
Personally I’d guess that the issues you mention are two different issues - at least it feels that way.
Regarding your second article:
Above logfile part looks totally fine, ensure that ticket with ID 9137 really doesn’t have articles and, if it doesn’t that your database wasn’t having a bad day. Looks like database was telling Zammad “yep, written!” but didn’t do that. But that’s fishing in the dark as we have no information on your database setup.
This sounds like the application is trying to communicate with your Zammad Server but fails for whatever reason. This can have various reasons:
database is overloaded or you’re having deadlocks (deadlocks can be found by searching for that term in production.log)
other database errors like max_connections can be found there too. Maybe your database server limits the remaining connections for administrative use
ensure your websocket server is up and running
If you reload Zammad and /ws upgrade fails, you’re using the ajax fallback which draws much more load on your system which should be avoided
First of all thanks a lot @MrGeneration for your reply. I’m sorry for the big information gap I left behind - caused by the JS error I thought the problem is located in the frontend section, so I didn’t post the exact server setup.
Setup
Zammad Server:
Single machine installation
VM on ESXi
Ubuntu 16.04
PostgreSQL 10.12
Apache 2.4.29 ReverseProxy
Zammad 3.3.x
The machine / VM is well dimensioned with:
10 GB RAM
4 vCPU
PostgreSQL:
max_connections: 2000 (limit not reached)
Size of temporary file: 3032 kB (rewrite from temp DB to master seems to work)
Deadlock Counter for last 2 month: 16 (I think that is acceptable)
On the setup nothing had changed for a while and the error started for all users at once.
If I look at the performance values of the machine and especially PostgreSQL, there is no heavy usage of resources.
At the moment I think, caused by your hint with the AJAX fallback, the problem is somewhere around the /ws endpoint or socket server itself. /var/log/zammad/websocket-server_err.log is totally empty. In ‘tail’ I can see backend is telling that there are no AJAX clients and a bunch of websocket using clients. In my opinion that says the socket is working properly.
The production log doesn’t show up any relevant errors. For days I ‘tail’ now it’s content, but nothing significant show up.
The stranged thing is, the closing of tickets and setting of other ticket attributes works well. Only the writing or saving of articles is not working.
Honestly, I don’t see why updates on a table should work and inserts don’t.
This doesn’t seemt o be related to Zammad and I can’t help further on that regard.
You could however fire my favorite greps I mentioned above:
Also a grep "Deadlock" logfile might help.
Logfile wants to be your production log in case that wasn’t clear yet.
I got your point but in the logs there is no error. production.log and websocket_err.log are clean of errors and warnings. The only error popping up is the one in the frontend JS,
The process is atm:
Post an article (mail, comment or internal comment doesn’t matter)
JS exception is thrown in Ticket._selectorMatch: Unknown operator: is,is
Backend recieves PUT request and obviously handle it without error
In Frontend nothing happens only ‘submit’ is greyed out (no feedback over established socket? - socket is definitv ‘online’)
.* May be the socket handling in frontend is aborted at the exception point?
The only way to work further is to discard changes by closing ticket tab and answer warning with ‘discard changes’ or push the button ‘discard changes’ on bottum of the ticket view.
PUT request is send to backend wich in my opinion is discarding the changes. I don’t know the exact internal process.
Thinking about the process is hardining my opinion that the JS error is the cause of the problem. Somehow it is realted to the socket server. While the debugging I saw my client switching on the socket side to AJAX, although the socket was establshed.
Is this issue constantly happening or only during high load situations?
This might be a performance issue of some sort. I currently suspect you’re using Firefox when having this issue.
You could try: zammad run rake assets:precompile to compile the javascript entirely from stretch.
If a clean reload of the application keeps providing this issue and it happens no matter of the situation, please provide a list of addons you’re using in that browser.
If possible also try another browser or try to do the same with completely deactivated addons.
If this only happens in high duty situations, please provide the output of the following commands in such a moment:
Unfortunately the issue is happening constantly at all users working with the system. In the meantime I had tried multiple Browsers: Chromium 73.0.3683.75, Chrome 83.0.4103.116, Firefox 77.0.1. Also I tried it with Chrome without plugins enabled and the result is the same.
Last week I had tried the following repair strategies for the assets without success:
Asset cleanup
1.1. zammad run rake assets:clean
1.2. zammad run rake assets:precompile
zammad run rake tmp:clear + Asset cleanup
Reinstallation of of assets
3.1. Download of 3.4. source
3.2. rm -R /opt/zammad/app/assets/*
3.3. rm -R /opt/zammad/public/assets/*
3.2. cp -R /tmp/zammad-3.4.0/app/assets/* /opt/zammad/app/assets
3.3. cp -R /tmp/zammad-3.4.0/public/assets/* /opt/zammad/public/assets
3.4. chown -R zammad:zammad /opt/zammad
3.5. zammad run rake assets:precompile
3.6. service zammad restart
This is an unchanged vanilla installation of Zammad that’s coming from us, Zammad GmbH provided via packager.io …?
Sounds very odd, because the javascript should be working just fine if it’s not changed or manipulated.
Ensure that no antivirus or “security suite” is playing man in the middle and potentially changing javascript files during loading it.
This could explain why parts of Zammad don’t work.
I’m afraid that this is as far as I can help you with, because anything else goes deep down to your client (doesn’t seem to be a server/application issue) which is out of scope for me.
I’m sorry.
Edit:
What you could do, do ensure your host is not at fault and your client isn’t as well - you could spin up a free trial instance on our hosted setup and check if you experience the issue there.
Only for completion and may be further investigation or to save some people’s time:
I installed a completly new server VM with the same setup. After finished this, I tested all features and everything worked well. Now I restored zammad with it’s own internal backup and restore scripts.
The new setup began to missbehave the same way. So I decided that the error have to be in the internal settings or the db. Finally, after disabling the ‘time tracking’, everything began to work again.
I can’t reproduce the error (it’s now working with time tracking enabled too) and I have absolutly no clue how exactly the error raises. There was absolutly no visible error message other then the JS error.
May be, the info will help and save someone’s time.
The backup holds the complete Zammad directory with source code files and attachments (if applicable).
I see two possible reasons for your issues that may be connected to the time accouting issue:
you disabled or removed an object which is referenced in the time accounting configuration which is why it blows up stuff
your source code files are somewhat broken, you could try to only restore /storage and the database
Another side note:
I don’t know if you followd the migration via documentation, but if you haven’t it’s worth mentioning that package installations require a forced reinstall for above reason.
