(another) Rake searchindex:rebuild Issues since 4.1 upgrade

Infos:

  • Used Zammad version: 4.1
  • Used Zammad installation type: Docker orchestrated with Zammad Helm Chart
  • Operating system: GKE
  • Browser + version: any

Expected behavior:

  • rebuilding the elasticsearch index should work

Actual behavior:

  • rebuilding the elasticsearch index does not work

Steps to reproduce the behavior:

  • Try to rebuild your elasticsearch index under zammad 4.1

I was running Zammad 4.0.x before and didnt have any issue with rebuilding the elasticsearch index.
With 4.1.x now, the reindexing does not work.

The following log is taken from a small Zammad installation, bascially used for upgrade testing. There is not much data in this system and only around 100 tickets.

Trying to rebuild the index fails with the following errors.

zammad@zammad-0:~$ rails searchindex:rebuild
I, [2021-06-23T13:00:16.124884 #44-47193338788220]  INFO -- : Setting.set('models_searchable', ["KnowledgeBase::Answer::Translation", "Ticket", "Organization", "User", "Chat::Session"])
drop indexes...done
delete pipeline (pipeline)... done
create indexes...done
create pipeline (pipeline)... done
reload data...
 reload KnowledgeBase::Answer::Translation
  - started at 2021-06-23 13:00:18 UTC
W, [2021-06-23T13:00:18.983078 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	3/3
  - took 0 seconds
 reload KnowledgeBase::Translation
  - started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.536410 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	1/1
  - took 0 seconds
 reload KnowledgeBase::Category::Translation
  - started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.627173 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	4/4
  - took 0 seconds
 reload Ticket::Priority
  - started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.781907 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	3/3
  - took 0 seconds
 reload Ticket::State
  - started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.810643 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	7/7
  - took 0 seconds
 reload Ticket
  - started at 2021-06-23 13:00:19 UTC
W, [2021-06-23T13:00:19.987032 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	86/86
  - took 5 seconds
 reload Organization
  - started at 2021-06-23 13:00:25 UTC
W, [2021-06-23T13:00:25.181196 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	2/2
  - took 0 seconds
 reload StatsStore
  - started at 2021-06-23 13:00:25 UTC
W, [2021-06-23T13:00:25.470830 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	7/7
  - took 0 seconds
 reload Group
  - started at 2021-06-23 13:00:25 UTC
W, [2021-06-23T13:00:25.936436 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	4/4
  - took 0 seconds
 reload User
  - started at 2021-06-23 13:00:26 UTC
W, [2021-06-23T13:00:26.113638 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	14/14
  - took 0 seconds
 reload Chat::Session
  - started at 2021-06-23 13:00:26 UTC
W, [2021-06-23T13:00:26.992556 #44-47193338788220]  WARN -- : Scoped order is ignored, it's forced to be batch order.
rails aborted!
SystemStackError: stack level too deep
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each_with_index'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject_join'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:639:in `visit_Arel_Nodes_And'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:601:in `visit_Arel_Nodes_On'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:593:in `visit_Arel_Nodes_InnerJoin'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:824:in `block in inject_join'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each_with_index'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:820:in `inject_join'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:550:in `visit_Arel_Nodes_JoinSource'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:251:in `visit_Arel_Nodes_SelectCore'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:217:in `block in visit_Arel_Nodes_SelectStatement'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:216:in `each'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:216:in `inject'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/to_sql.rb:216:in `visit_Arel_Nodes_SelectStatement'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:15:in `visit'
/usr/local/bundle/gems/arel-9.0.0/lib/arel/visitors/reduce.rb:8:in `accept'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/determine_if_preparable_visitor.rb:10:in `accept'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/abstract/database_statements.rb:25:in `to_sql_and_binds'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/abstract/database_statements.rb:61:in `select_all'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/connection_adapters/abstract/query_cache.rb:106:in `select_all'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/calculations.rb:197:in `block in pluck'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation.rb:584:in `skip_query_cache_if_necessary'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/calculations.rb:197:in `pluck'
/opt/zammad/app/models/tag.rb:153:in `tag_list'
/opt/zammad/app/models/concerns/has_tags.rb:55:in `tag_list'
/opt/zammad/app/models/chat/session/search_index.rb:22:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'

----snip-----
removed pages over pages containing the identical lines:
/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:81:in `search_index_value_by_attribute'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `each'
/opt/zammad/app/models/application_model/can_lookup_search_index_attributes.rb:23:in `search_index_attribute_lookup'
/opt/zammad/app/models/chat/session/search_index.rb:29:in `block in search_index_attribute_lookup'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/delegation.rb:71:in `each'
----snip-----

/opt/zammad/app/models/chat/session/search_index.rb:26:in `search_index_attribute_lookup'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:177:in `search_index_update_backend'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:226:in `block (2 levels) in search_index_reload'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:222:in `each'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:222:in `block in search_index_reload'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:222:in `loop'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:222:in `in_batches'
/usr/local/bundle/gems/activerecord-5.2.4.6/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:221:in `with_index'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:221:in `search_index_reload'
/opt/zammad/lib/tasks/search_index_es.rake:48:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:44:in `each'
/opt/zammad/lib/tasks/search_index_es.rake:44:in `block (2 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:63:in `block (2 levels) in <main>'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/commands/rake/rake_command.rb:23:in `block in perform'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/commands/rake/rake_command.rb:20:in `perform'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/command.rb:48:in `invoke'
/usr/local/bundle/gems/railties-5.2.4.6/lib/rails/commands.rb:18:in `<main>'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `block in require_with_bootsnap_lfi'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/loaded_features_index.rb:65:in `register'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:20:in `require_with_bootsnap_lfi'
/usr/local/bundle/gems/bootsnap-1.3.2/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:29:in `require'
/usr/local/bundle/gems/activesupport-5.2.4.6/lib/active_support/dependencies.rb:291:in `block in require'
/usr/local/bundle/gems/activesupport-5.2.4.6/lib/active_support/dependencies.rb:257:in `load_dependency'
/usr/local/bundle/gems/activesupport-5.2.4.6/lib/active_support/dependencies.rb:291:in `require'
bin/rails:9:in `<main>'
Tasks: TOP => searchindex:rebuild
(See full trace by running task with --trace)
zammad@zammad-0:~$

SOLVED - with the latest zammad 4.1 release (docker tags 4.1.0-17) my problem seems to be solved.
I’m now able to rebuild elasticsearch index again:

zammad@zammad-0:~$ rails searchindex:rebuild
I, [2021-06-25T13:16:07.660223 #20-47095670749540]  INFO -- : Setting.set('models_searchable', ["KnowledgeBase::Answer::Translation", "Ticket", "Organization", "User", "Chat::Session"])
drop indexes...done
delete pipeline (pipeline)... done
create indexes...done
create pipeline (pipeline)... done
reload data...
 reload KnowledgeBase::Answer::Translation
  - started at 2021-06-25 13:16:12 UTC
W, [2021-06-25T13:16:12.794639 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	3/3
  - took 0 seconds
 reload KnowledgeBase::Translation
  - started at 2021-06-25 13:16:13 UTC
W, [2021-06-25T13:16:13.674145 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	1/1
  - took 0 seconds
 reload KnowledgeBase::Category::Translation
  - started at 2021-06-25 13:16:13 UTC
W, [2021-06-25T13:16:13.831017 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	4/4
  - took 0 seconds
 reload Ticket::Priority
  - started at 2021-06-25 13:16:14 UTC
W, [2021-06-25T13:16:14.433571 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	3/3
  - took 0 seconds
 reload Ticket::State
  - started at 2021-06-25 13:16:14 UTC
W, [2021-06-25T13:16:14.549435 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	7/7
  - took 0 seconds
 reload Ticket
  - started at 2021-06-25 13:16:15 UTC
W, [2021-06-25T13:16:15.114550 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	86/86
  - took 10 seconds
 reload Organization
  - started at 2021-06-25 13:16:25 UTC
W, [2021-06-25T13:16:25.358730 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	2/2
  - took 0 seconds
 reload StatsStore
  - started at 2021-06-25 13:16:25 UTC
W, [2021-06-25T13:16:25.798828 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	8/8
  - took 0 seconds
 reload Group
  - started at 2021-06-25 13:16:26 UTC
W, [2021-06-25T13:16:26.469638 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	4/4
  - took 0 seconds
 reload User
  - started at 2021-06-25 13:16:26 UTC
W, [2021-06-25T13:16:26.698982 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	15/15
  - took 1 seconds
 reload Chat::Session
  - started at 2021-06-25 13:16:27 UTC
W, [2021-06-25T13:16:27.857349 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
	55/55
  - took 1 seconds
 reload Cti::Log
  - started at 2021-06-25 13:16:29 UTC
W, [2021-06-25T13:16:29.424189 #20-47095670749540]  WARN -- : Scoped order is ignored, it's forced to be batch order.
  - took 0 seconds
zammad@zammad-0:~$
1 Like

Just for reference, you’ve been affected by this bug:

It has been fixed yesterday

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.