Check_MK integration page gives 500 error in 2.5.0

Infos:

Important:
If you are a Zammad Support or hosted customer and experience a technical issue, please refer to: support@zammad.com using your zammad-hostname / or company contract.

  • Used Zammad version: zammad-2.5.0-1529668954.708c86ba.centos7.x86_64
  • Used Zammad installation source: (source, package, …) repo
  • Operating system: CentOS Linux release 7.5.1804 (Core)
  • Browser + version: Google Chrome 67.0.3396.87

Expected behavior:

  • not getting a 500 when visiting the Check_MK integration page (usually I see the latest calls to debug them)

Actual behavior:

  • getting a 500 errror


I, [2018-06-27T07:58:04.020106 #1307]  INFO -- : Started GET "/api/v1/http_logs/check_mk?limit=50&_=1529941048387" for 185.121.33.41 at 2018-06-27 07:58:04 +0000
I, [2018-06-27T07:58:04.025242 #1307]  INFO -- : Processing by HttpLogsController#index as JSON
I, [2018-06-27T07:58:04.025331 #1307]  INFO -- :   Parameters: {"limit"=>"50", "_"=>"1529941048387", "facility"=>"check_mk"}
E, [2018-06-27T07:58:04.089893 #1307] ERROR -- : "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/core_ext/object/json.rb:36:in `encode'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/core_ext/object/json.rb:36:in `to_json'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/core_ext/object/json.rb:36:in `to_json'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/json/encoding.rb:55:in `to_json'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/json-2.1.0/lib/json/common.rb:224:in `generate'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/json-2.1.0/lib/json/common.rb:224:in `generate'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/json/encoding.rb:100:in `stringify'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/json/encoding.rb:33:in `encode'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/json/encoding.rb:20:in `encode'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/core_ext/object/json.rb:39:in `to_json'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/renderers.rb:156:in `block in <module:Renderers>'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/renderers.rb:149:in `block in _render_to_body_with_renderer'
/opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/set.rb:324:in `each_key'
/opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/set.rb:324:in `each'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/renderers.rb:145:in `_render_to_body_with_renderer'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/renderers.rb:141:in `render_to_body'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/abstract_controller/rendering.rb:24:in `render'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/rendering.rb:36:in `render'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/instrumentation.rb:44:in `block (2 levels) in render'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/core_ext/benchmark.rb:12:in `block in ms'
/opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/benchmark.rb:308:in `realtime'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/core_ext/benchmark.rb:12:in `ms'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/instrumentation.rb:44:in `block in render'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/instrumentation.rb:87:in `cleanup_view_runtime'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.5/lib/active_record/railties/controller_runtime.rb:29:in `cleanup_view_runtime'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/instrumentation.rb:43:in `render'
/opt/zammad/app/controllers/application_controller/renders_models.rb:152:in `model_index_render_result'
/opt/zammad/app/controllers/http_logs_controller.rb:14:in `index'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/abstract_controller/base.rb:186:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/rendering.rb:30:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:131:in `run_callbacks'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/abstract_controller/callbacks.rb:19:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/rescue.rb:20:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/notifications.rb:166:in `block in instrument'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/notifications.rb:166:in `instrument'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.5/lib/active_record/railties/controller_runtime.rb:22:in `process_action'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/abstract_controller/base.rb:124:in `process'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionview-5.1.5/lib/action_view/rendering.rb:30:in `process'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal.rb:189:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_controller/metal.rb:253:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/routing/route_set.rb:31:in `serve'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/journey/router.rb:33:in `each'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/journey/router.rb:33:in `serve'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/routing/route_set.rb:844:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:190:in `call!'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/strategy.rb:168:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/omniauth-1.7.1/lib/omniauth/builder.rb:63:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/etag.rb:25:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/conditional_get.rb:25:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/head.rb:12:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/session/abstract/id.rb:232:in `context'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/session/abstract/id.rb:226:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/cookies.rb:613:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/callbacks.rb:97:in `run_callbacks'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/callbacks.rb:24:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/rack/logger.rb:36:in `call_app'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/rack/logger.rb:24:in `block in call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:69:in `block in tagged'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:26:in `tagged'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/tagged_logging.rb:69:in `tagged'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/rack/logger.rb:24:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/request_id.rb:25:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/method_override.rb:22:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/runtime.rb:22:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.5/lib/active_support/cache/strategy/local_cache_middleware.rb:27:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/executor.rb:12:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.5/lib/action_dispatch/middleware/static.rb:125:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.5/lib/rack/sendfile.rb:111:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/engine.rb:522:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/puma-3.11.0/lib/puma/configuration.rb:225:in `call'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/puma-3.11.0/lib/puma/server.rb:624:in `handle_request'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/puma-3.11.0/lib/puma/server.rb:438:in `process_client'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/puma-3.11.0/lib/puma/server.rb:302:in `block in run'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/puma-3.11.0/lib/puma/thread_pool.rb:120:in `block in spawn_thread'
/opt/zammad/vendor/bundle/ruby/2.4.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
I, [2018-06-27T07:58:04.090690 #1307]  INFO -- : Completed 500 Internal Server Error in 65ms (Views: 0.2ms | ActiveRecord: 3.5ms)

Steps to reproduce the behavior:

  • Enable Check_MK integration, configure check_mk to send alerts via custom script, update from 2.4.0 to 2.5.0

I don’t know if this was introduced by the update or if just broke at a certain point, used to work when set up… Now I wanted to investigate on how to handle HOST_STATUS along the already working SERVICE_STATUS notifications… I went to the integration page to check the contents of the API calls but got the error.

Hi @penzoiders - It looks like there is an entry in your DB with an unsupported/wrong character encoding. Let’s find out which one it is by doing the following:

1.) Create a file called debug_issue.rb in your Zammad directory (usually /opt/zammad)
2.) Run the file from your Zammad directory via zammad run rails r debug_issue.rb or rails r debug_issue.rb as zammad user, depending on your installation source (package/source)
3.) Post the output here. Make sure all sensitive data is anonymized! If you want you can drop it as an email to support@zammad.com. Please refer to this topic and me :slight_smile:
4.) Delete the debug_issue.rb file

HttpLog.where(facility: 'check_mk').order('created_at DESC').limit(50).each do |entry|
  JSON.generate(entry)
rescue => e
  p entry
end

thanks @thorsteneckel,
maybe a typo in your debug script??

 /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/commands/runner/runner_command.rb:34:in `load': debug_issue.rb:3: syntax error, unexpected keyword_rescue, expecting keyword_end (SyntaxError)
rescue => e
      ^
debug_issue.rb:5: syntax error, unexpected keyword_end, expecting end-of-input
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/commands/runner/runner_command.rb:34:in `perform'
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/command/base.rb:63:in `perform'
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/command.rb:44:in `invoke'
	from /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.5/lib/rails/commands.rb:16:in `<top (required)>'
	from /opt/zammad/bin/rails:4:in `require'
	from /opt/zammad/bin/rails:4:in `<main>'

Sorry, my bad: Fixed code:

HttpLog.where(facility: 'check_mk').order('created_at DESC').limit(50).each do |entry|
  begin
    JSON.generate(entry)
  rescue => e
    p entry
  end
end

don’t worry…
zammad run rails r debug_issue.rb
gives no error but even no output to the shell… should I look somewhere else?

Looks like the code takes a different approach. Please try this:

HttpLog.where(facility: 'check_mk').order('created_at DESC').limit(50).each do |entry|
  begin
    entry.to_json
  rescue => e
    p entry
  end
end

Ok, looking at the string, probably there’s an accented à in the body of the Check_MK service notification due to the Italian language of the Windows monitored host.

remoto \xC3\xA8 stato emesso da unAutorit\xC3\xA0

comes from this string in check_mk

remoto è stato emesso da un'Autorità

heres the anonymized output:

#<HttpLog id: 225, direction: "in", facility: "check_mk", method: "POST", url: "https://zammad.fqdn.com/api/v1/integration/...", status: "200", ip: "xIP.xIP.xIP.xIP", request: {"content"=>"HTTP_VERSION: HTTP/1.0\nHTTP_HOST: zammad.fqdn.com\nHTTP_CLIENT_IP: xIP.xIP.xIP.xIP\nHTTP_X_FORWARDED_FOR: xIP.xIP.xIP.xIP\nHTTP_X_FORWARDED_PROTO: https\nHTTP_CONNECTION: close\nHTTP_USER_AGENT: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2\nHTTP_ACCEPT: */*\n\n------------------------------9b34f2a406fa\r\nContent-Disposition: form-data; name=\"event_id\"\r\n\r\n16599\r\n------------------------------9b34f2a406fa\r\nContent-Disposition: form-data; name=\"host\"\r\n\r\nsql01.testing.fqdn.com\r\n------------------------------9b34f2a406fa\r\nContent-Disposition: form-data; name=\"service\"\r\n\r\nLog System\r\n------------------------------9b34f2a406fa\r\nContent-Disposition: form-data; name=\"state\"\r\n\r\nCRITICAL\r\n------------------------------9b34f2a406fa\r\nContent-Disposition: form-data; name=\"text\"\r\n\r\nCRIT - 8 CRIT, 1 WARN messages (Last worst: Jun 18 11:28:53 0.36882 Schannel Il certificato ricevuto dal server remoto \xC3\xA8 stato emesso da unAutorit\xC3\xA0 di certificazione non disponibile nellelenco locale. Non \xC3\xA8 pertanto possibile convalidare i dati contenuti nel certificato. La richiesta di connessione SSL non \xC3\xA8 riuscita. Il certificato server \xC3\xA8 incluso nei dati allegati.)\r\n------------------------------9b34f2a406fa--\r\n", "content_type"=>"multipart/form-data; boundary=----------------------------9b34f2a406fa", "content_encoding"=>nil, "source"=>"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.27.1 zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}, response: {"code"=>"200", "content"=>"X-Frame-Options: SAMEORIGIN\nX-XSS-Protection: 1; mode=block\nX-Content-Type-Options: nosniff\nContent-Type: application/json; charset=utf-8\n\n{\"result\":\"ticket already open, added note\",\"ticket_ids\":[46]}", "content_type"=>nil, "content_encoding"=>nil, "source"=>nil}, updated_by_id: 1, created_by_id: 1, created_at: "2018-06-18 14:10:57", updated_at: "2018-06-18 14:10:57">

This looks like a bug to me. Could you please combine the information and create an issue over at Github with them, fill out the issue template accordingly and link to this thread? We will verify and fix it :+1:

1 Like

done:

Perfect! I’ll close the thread now :vulcan_salute: