Infos:
- Used Zammad version: 4.1
- Used Zammad installation type: Docker orchestrated with Zammad Helm Chart
- Operating system: GKE
- Browser + version: any
Expected behavior:
- rebuilding the elasticsearch index should work
Actual behavior:
- rebuilding the elasticsearch index does not work
Steps to reproduce the behavior:
- Try to rebuild your elasticsearch index under zammad 4.1
I was running Zammad 4.0.x before and didnt have any issue with rebuilding the elasticsearch index.
With 4.1.x now, the reindexing does not work.
The following log is taken from a small Zammad installation, bascially used for upgrade testing. There is not much data in this system and only around 100 tickets.
Trying to rebuild the index fails with the following errors.
zammad@zammad-0:~$ rails searchindex:rebuild
I, [2021-06-23T13:00:16.124884 #44-47193338788220] INFO -- : Setting.set('models_searchable', ["KnowledgeBase::Answer::Translation", "Ticket", "Organization", "User", "Chat::Session"])
drop indexes...done
delete pipeline (pipeline)... done
create indexes...done
create pipeline (pipeline)... done
reload data...
reload KnowledgeBase::Answer::Translation
- started at 2021-06-23 13:00:18 UTC
W, [2021-06-23T13:00:18.983078 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
3/3
- took 0 seconds
reload KnowledgeBase::Translation
- started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.536410 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
1/1
- took 0 seconds
reload KnowledgeBase::Category::Translation
- started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.627173 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
4/4
- took 0 seconds
reload Ticket::Priority
- started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.781907 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
3/3
- took 0 seconds
reload Ticket::State
- started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.810643 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
7/7
- took 0 seconds
reload Ticket
- started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.987032 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
86/86
- took 5 seconds
reload Organization
- started at 2021-06-23 13:00:25 UTC
W, [2021-06-23T13:00:25.181196 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
2/2
- took 0 seconds
reload StatsStore
- started at 2021-06-23 13:00:25 UTC
W, [2021-06-23T13:00:25.470830 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
7/7
- took 0 seconds
reload Group
- started at 2021-06-23 13:00:25 UTC
W, [2021-06-23T13:00:25.936436 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
4/4
- took 0 seconds
reload User
- started at 2021-06-23 13:00:26 UTC
W, [2021-06-23T13:00:26.113638 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
14/14
- took 0 seconds
reload Chat::Session
- started at 2021-06-23 13:00:26 UTC
W, [2021-06-23T13:00:26.992556 #44-47193338788220] WARN -- : Scoped order is ignored, it's forced to be batch order.
rails aborted!
SystemStackError: stack level too deep
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each_with_index'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject_join'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:639:in `visit_Arel_Nodes_And'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:601:in `visit_Arel_Nodes_On'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:593:in `visit_Arel_Nodes_InnerJoin'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:824:in `block in inject_join'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each_with_index'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject_join'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:550:in `visit_Arel_Nodes_JoinSource'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:251:in `visit_Arel_Nodes_SelectCore'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:217:in `block in visit_Arel_Nodes_SelectStatement'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:216:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:216:in `inject'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:216:in `visit_Arel_Nodes_SelectStatement'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:8:in `accept'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/determine_if_preparable_visitor.rb:10:in `accept'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/abstract/database_statements.rb:25:in `to_sql_and_binds'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/abstract/database_statements.rb:61:in `select_all'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/abstract/query_cache.rb:106:in `select_all'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/calculations.rb:197:in `block in pluck'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation.rb:584:in `skip_query_cache_if_necessary'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/calculations.rb:197:in `pluck'
/opt/zammad/app/models/tag.rb:153:in `tag_list'
/opt/zammad/app/models/concerns/has_tags.rb:55:in `tag_list'
/opt/zammad/app/models/chat/session/search_index.rb:22:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
----snip-----
removed pages over pages containing the identical lines:
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
----snip-----
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:177:in `search_index_update_backend'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:226:in `block (2 levels) in search_index_reload'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:222:in `each'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:222:in `block in search_index_reload'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:222:in `loop'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:222:in `in_batches'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:221:in `with_index'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:221:in `search_index_reload'
/opt/zammad/lib/tasks/search_index_es.rake:48:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:44:in `each'
/opt/zammad/lib/tasks/search_index_es.rake:44:in `block (2 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:63:in `block (2 levels) in <main>'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/commands/rake/rake_command.rb:23:in `block in perform'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/commands/rake/rake_command.rb:20:in `perform'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/command.rb:48:in `invoke'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/commands.rb:18:in `<main>'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `block in require_with_bootsnap_lfi'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/loaded_features_index.rb:65:in `register'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:20:in `require_with_bootsnap_lfi'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:29:in `require'
/usr/local/bundle/gems/activesupport-5.2.4.6/lib/active_support/dependencies.rb:291:in `block in require'
/usr/local/bundle/gems/activesupport-5.2.4.6/lib/active_support/dependencies.rb:257:in `load_dependency'
/usr/local/bundle/gems/activesupport-5.2.4.6/lib/active_support/dependencies.rb:291:in `require'
bin/rails:9:in `<main>'
Tasks: TOP => searchindex:rebuild
(See full trace by running task with --trace)
zammad@zammad-0:~$