Cannot rebuild elasticsearch index

Technical assistance
1
  • Used Zammad version: 5.0.3
  • Used Zammad installation type: package
  • Operating system: Debian Stretch
  • Browser + version:

Expected behavior:

  • I have to rebuild the index, because it was build with ES 5.x, to update ES to version 7

Actual behavior:

  • Running zammad run rake searchindex:rebuild gave me the error “Version 6.8.21 of configured elasticsearch is not supported.” Below is the output from the command:

** Invoke searchindex:rebuild (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke searchindex:configured (first_time)
** Invoke environment
** Execute searchindex:configured
** Invoke searchindex:version_supported (first_time)
** Invoke environment
** Execute searchindex:version_supported
rake aborted!
Version 6.8.21 of configured elasticsearch is not supported.
/opt/zammad/lib/search_index_backend.rb:29:in info' /opt/zammad/lib/search_index_backend.rb:1065:in version’
/opt/zammad/lib/search_index_backend.rb:1075:in version_int' /opt/zammad/lib/search_index_backend.rb:1085:in version_supported?’
/opt/zammad/lib/tasks/search_index_es.rake:69:in block (2 levels) in <main>' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in block in execute’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in each' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in execute’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in block in invoke_with_call_chain' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in synchronize’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in invoke_with_call_chain' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:243:in block in invoke_prerequisites’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:241:in each' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:241:in invoke_prerequisites’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:218:in block in invoke_with_call_chain' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in synchronize’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in invoke_with_call_chain' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in invoke’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in invoke_task' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in block (2 levels) in top_level’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in each' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in block in top_level’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in run_with_threads' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in top_level’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in block in run' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in standard_exception_handling’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in run' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in <top (required)>’
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in load' /opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in <top (required)>’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in load' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:63:in kernel_load’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/cli/exec.rb:28:in run' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:494:in exec’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in invoke_command’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor.rb:392:in dispatch' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:30:in dispatch’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/vendor/thor/lib/thor/base.rb:485:in start' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/cli.rb:24:in start’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/exe/bundle:49:in block in <top (required)>' /opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/lib/bundler/friendly_errors.rb:130:in with_friendly_errors’
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.16/exe/bundle:37:in <top (required)>' /opt/zammad/bin/bundle:5:in load’
/opt/zammad/bin/bundle:5:in `’
Tasks: TOP => searchindex:rebuild => searchindex:version_supported

In the search_index_backend.rb there is a comment “# only versions greater/equal than 6.5.0 are supported”, but this doesn’t seem to work. I Have tried to install several ES Version up from 6.5, but the error is every time the same.

2

Hey @richie_77 ,

considering elasticsearch you have to follow a certain upgrade path while upgrading zammad:
https://docs.zammad.org/en/latest/prerequisites/software.html#elasticsearch-optional

Now the neat part - you also have to follow a certain upgrade path while upgrading elasticsearch itself:
https://www.elastic.co/guide/en/elasticsearch/reference//current/setup-upgrade.html

What most folks tend to ignore is this:

Elasticsearch can read indices created in the previous major version. If you have indices created in 5.x or before, you must reindex or delete them before upgrading to 7.16.2. Elasticsearch nodes will fail to start if incompatible indices are present. Snapshots of 5.x or earlier indices cannot be restored to a 7.x cluster even if they were created by a 6.x cluster. For information about upgrading old indices, see Reindex to upgrade.

So my best guess is: you tried to go from 5.x to 7.x in a rush(?) and your indices are completely messed up.

The easy way out would be to simply delete your es indices and let zammad rebuild them from scratch. Keep in mind that this can take some while (depending on your environment) and while there are no indices zammad could use several functions will be limited (f. e. search).

cheers

3

No, i have a system with ES 6.8.x and Zammad 5.0.x and i cannot run the reindex procedure due to the error above. When i update ES to 7.x, it says, that i have to reindex with a version 6.x in order to start the 7.x-version.

Are there any guides, how to delete the ES index without using the ES api? Or are there procedures to do this with the ruby console?

As a last hope, i have a snapshot with Zammad 4.1, where i can reindex without any problems. So i have to backup the database from my current instance, update the 4.1 snapshot to 5.0.3 and reimport the database. I hope, this works…

4

I’m just wondering because search_index_backend.rb checks for a minimum version of ES with at least 6.5.

  def self.version_supported?

    # only versions greater/equal than 6.5.0 are supported
    return if version_int < 6_005_000

    true
  end

Maybee, i miss something in other files, where other version checks are?

5

If i look deeper in the search_index_backend.rb, i found the following version checks:

if response.success?

      installed_version = response.data.dig('version', 'number')

      raise "Unable to get elasticsearch version from response: #{response.inspect}" if installed_version.blank?

      installed_version_parsed = Gem::Version.new(installed_version)

      version_supported = installed_version_parsed < Gem::Version.new('8')

      raise "Version #{installed_version} of configured elasticsearch is not supported." if !version_supported

      version_supported = installed_version_parsed >= Gem::Version.new('7.8')

      raise "Version #{installed_version} of configured elasticsearch is not supported." if !version_supported

      return response.data

    end

This looks to me like only versions from 7.8 and smaller then 8 are supported. I’m not so firm with ruby, but the syntax looks clear and understandable.

6

Seems like you didn’t follow the upgrade path that’s written in the zammad docs i linked:

Zammad Elasticsearch
5.0+ 7.8+
4.0-4.1 6.5-7.12
3.4-3.6 5.5–7.9
3.3 2.4–7.6
3.2 2.4–7.5
3.1 2.4–7.4
2.0–3.0 2.4–5.6

You should’ve upgraded to es 7.8+ prior to upgrading zammad to 5.x.

Here’s what i would do:

  • drop your current indices
  • update to es 7.8+
  • let zammad recreate the indices

Sure there are plenty of es guides. F. e. their own:

Yeah you could go back to zammad 4.1, update es to 7.8+ and after that update zammad to 5.x.

Whatever floats your boat is right. There are multiple ways to get this working again.

Edit:
Keep in mind that when you’re fiddling around with db backups/restores from different versions etc. things can go downtown pretty fast. Especially with a TTS.

I can’t stress this enough: the easy way would be to simply delete the indices and recreate them after the es update.

cheers

1 Like
7

Thank you for your answer. I will try to delete the indices. If anything goes wrong - i have multiple snapshots of the system, it is running in a vm and i take a new snapshot every time i change something on the system.

1 Like
8

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.