ElasticSearch Index Rebuild after upgrade from 6.3.1 to 6.5

Technical assistance
1

Infos:

  • Used Zammad version: 6.3.1 to 6.5
  • Used Zammad installation type: apt upgrade packages
  • Operating system: Ubuntu 22.04
  • Browser + version:

Expected behavior:

  • Index rebuild completes properly

Actual behavior:

  • After upgrading Zammad using the package manager from version 6.3.1 to 6.5 I attempted to run the ElasticSearch index rebuild as recommended and required by the release notes. When attempting this I received an error so I completed the same using the --trace parameter to get more details. The output is below:

root@zticket:/home/zammad# zammad run rake zammad:searchindex:rebuild --trace
** Invoke zammad:searchindex:rebuild (first_time)
** Invoke zammad:searchindex:version_supported (first_time)
** Invoke zammad:searchindex:configured (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute zammad:searchindex:configured
** Execute zammad:searchindex:version_supported
** Execute zammad:searchindex:rebuild
** Execute zammad:searchindex:drop
Dropping indexes… done.
** Execute zammad:searchindex:drop_pipeline
Deleting pipeline… done.
** Execute zammad:searchindex:create
Creating indexes… rake aborted!
Unable to process put request to elasticsearch URL ‘http://localhost:9200/zammad_production_chat_session’. Check the response and payload for detailed information:

Response:
#<UserAgent::Result:0x00007f6e3abea500 @success=false, @body=“{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [flattened] declared on field [preferences]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_doc]: No handler for type [flattened] declared on field [preferences]","caused_by" :{"type":"mapper_parsing_exception","reason":"No handler for type [flattened] declared on field [preferences]"}},"status":400}”, @data=nil, @code=“400”, @content_type=nil, @err or=“Client Error: #<Net::HTTPBadRequest 400 Bad Request readbody=true>!”, @header={“content-type”=>“application/json; charset=UTF-8”, “content-length”=>“414”}>

Payload:
{“mappings”:{“properties”:{“id”:{“type”:“integer”},“chat_id”:{“type”:“integer”},“session_id”:{“type”:“text”,“fields”:{“keyword”:{“type”:“keyword”,“ignore_above”:5012}}},“name”:{“type”:“t ext”,“fields”:{“keyword”:{“type”:“keyword”,“ignore_above”:5012}}},“state”:{“type”:“text”,“fields”:{“keyword”:{“type”:“keyword”,“ignore_above”:5012}}},“user_id”:{“type”:“integer”},“prefer ences”:{“type”:“flattened”},“updated_by_id”:{“type”:“integer”},“created_by_id”:{“type”:“integer”}},“_source”:{“excludes”:[“attachment”,“article.attachment”]}},“settings”:{“index.mapping. total_fields.limit”:2000,“index.max_result_window”:1000000}}

Payload size: 0M
/opt/zammad/lib/search_index_backend.rb:812:in make_request_and_validate' /opt/zammad/lib/search_index_backend.rb:135:in index’
/opt/zammad/lib/search_index_backend.rb:982:in block in create_index' /opt/zammad/lib/search_index_backend.rb:981:in each’
/opt/zammad/lib/search_index_backend.rb:981:in create_index' /opt/zammad/lib/tasks/zammad/search_index_es.rake:17:in block (3 levels) in ’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:281:in block in execute' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:281:in each’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:281:in execute' /opt/zammad/lib/tasks/zammad/search_index_es.rake:59:in block (3 levels) in ’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:281:in block in execute' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:281:in each’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:281:in execute' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:219:in block in invoke_with_call_chain’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:199:in synchronize' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:199:in invoke_with_call_chain’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/task.rb:188:in invoke' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:188:in invoke_task’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:138:in block (2 levels) in top_level' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:138:in each’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:138:in block in top_level' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:147:in run_with_threads’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:132:in top_level' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:83:in block in run’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:214:in standard_exception_handling' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/lib/rake/application.rb:80:in run’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/rake-13.2.1/exe/rake:27:in <top (required)>' /opt/zammad/vendor/bundle/ruby/3.2.0/bin/rake:25:in load’
/opt/zammad/vendor/bundle/ruby/3.2.0/bin/rake:25:in <top (required)>' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:58:in load’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:58:in kernel_load' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:23:in run’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:492:in exec' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/command.rb:28:in run’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor.rb:527:in dispatch’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:34:in dispatch' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/base.rb:584:in start’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:28:in start' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/exe/bundle:37:in block in <top (required)>’
/opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/lib/bundler/friendly_errors.rb:117:in with_friendly_errors' /opt/zammad/vendor/bundle/ruby/3.2.0/gems/bundler-2.4.22/exe/bundle:29:in <top (required)>’
/opt/zammad/bin/bundle:121:in load' /opt/zammad/bin/bundle:121:in ’
Tasks: TOP => zammad:searchindex:rebuild
root@zticket:/home/zammad# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2025-05-01 20:11:12 EDT; 5min ago
Docs: https://www.elastic.co
Main PID: 6767 (java)
Tasks: 45 (limit: 28702)
Memory: 1.2G
CPU: 26.298s
CGroup: /system.slice/elasticsearch.service
└─6767 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.h>

I saw reference to a db:migrate command and attempted that as well, but it did not work either.

Any help or advice would be greatly appreciated. As this instance is a VM and I created a snapshot before the upgrade I was able to easily roll back to a good working condition, so my server is still running version 6.3.1 for now.

Thank you in advance.

2

Unfortunately, I’ve been having the same problem since the update and would be very grateful if someone had a solution!

3

What Elasticsearch version are you guys using? If you have issues with it, that would be the minimum information one would expect.

4

I’m running version 7.10.2 here.

5

That is quite old.
Latest stable Elasticsearch 7.x version is 7.17.28.

Cannot reproduce this on the latest stable version tbh.

6

My version is 7.10.2 as well. I’ll attempt to upgrade that to 7.17.28 first and try the upgrade again. Will post update after I do that w/ results. Thank you @MrGeneration!

7

I’m facing the same problem. I used elasticsearch oss (its maximum version is 7.10.2). After removing oss 7.10.2 and installing 8.18.1, the index was successfully recreated.

8

Alright, I’ve solved the problem for myself by installing Zammad on a new machine and restoring my backup there.

9

Got this solved last night. The problem that I had run into was that after upgrading the underlying Ubuntu OS from 20.04 to 22.04 the Elastic repo had been disabled. I re-enabled then ran the install for Elasticsearch again before upgrading Zammad packages. After that the upgrade and the re-index ran smoothly.

Commands that I used to update Elasticsearch (specific to Ubuntu 22.04):

First, rebuild the repo. Here’s a good guide which explains how to do that for version 7.x: How To Install and Configure Elasticsearch on Ubuntu 22.04 | DigitalOcean

Next, get the updated packages and install the newest version of elasticsearch 7.x:

sudo apt update

sudo apt install elasticsearch

One last thing to note, since Elastic has moved away from the OSS 7.x license you have to do an apt install instead of an apt upgrade in order to get the Elastic packages up to date.