Zammad 6: Opensearch instead of Elasticsearch - Zammad 6 refuses

[alfredosola]

I have tried to run Zammad 6 with OpenSearch, so far un successfully.

Configured OpenSearch 2.9.0 (also tested 2.8.0) and Zammad 6.0.0-1690378408.79860681.bookworm.

I configured OpenSearch to pretend it is ElasticSearch, according to this post:

$ grep compatibility /etc/opensearch/opensearch.yml
compatibility.override_main_response_version: true

OpenSearch is up and available. But if I try to have Zammad talk to OpenSearch, Zammad aborts:

$ zammad run rake zammad:searchindex:rebuild
rake aborted!
Unable to process GET request to elasticsearch URL 'http://localhost:9200'. Check the response and payload for detailed information:

Response:
#<UserAgent::Result:0x00007f32ac69ff10 @success=false, @body=nil, @data=nil, @code=0, @content_type=nil, @error="#<EOFError: end of file reached>", @header=nil>

Payload:
null

Payload size: 0M
/opt/zammad/lib/search_index_backend.rb:36:in `info'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:64:in `block (3 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/3.1.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'
Tasks: TOP => zammad:searchindex:rebuild => zammad:searchindex:version_supported
(See full trace by running task with --trace)

So, is anybody running Zammad version 6 with OpenSearch? What did you have to do?

Thanks.

[fliebe92]

Hi @alfredosola. OpenSearch is not supported right now.

I played around with it yesterday evening, but there are some changes needed in the code.

[sebix]

OpenSearch is made to be compatible with ElasticSearch. It works perfectly fine after setting
compatibility.override_main_response_version: true
With this setting, OpenSearch reports the version number as it would be ES.
Also mentioned here:

Opensearch + Zammad: Version 2.0.0 of configured elasticsearch is not supported

Using this opensearch configuration setting works for me, using Zammad 5.2.1: compatibility.override_main_response_version: true OpenSearch’s documentation for this feature can be found at Agents and ingestion tools - OpenSearch documentation

@fliebe92

I played around with it yesterday evening, but there are some changes needed in the code.

This is an interesting observation because we had it running with Zammad 5 and now it works with Zammad 6. What did you find in your analysis?

[fliebe92]

@sebix> I set compatibility.override_main_response_version: true and afterward the command to create the needed pipelines failed.

[sebix]

Is your opensearch actually listening on http://localhost:9200 correctly?

[fliebe92]

Yeah, I checked that twice.

[sebix]

If the version check failed, the error message would be different, but yours is:

Unable to process GET request to elasticsearch URL 'http://localhost:9200'
...
@body=nil, @data=nil, @code=0 ... @error="#<EOFError: end of file reached>", @header=nil>

All that suggests an empty response and that puzzles me. Are username/password in OpenSearch and Zammad set correctly (the same)?
What is the result of curl -X GET https://localhost:9200 -u '$user:$pass' --insecure (fill in the configured username/password combination, standard is admin/admin)?
For me on a working installation the result of the request is

{
  "name" : "openslides-zammad",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "B7rcTWEoQDW6KO9Hyg_I9w",
  "version" : {
    "number" : "7.10.2",
    "build_type" : "deb",
    "build_hash" : "db90a415ff2fd428b4f7b3f800a51dc229287cb4",
    "build_date" : "2023-06-03T06:22:11.808047829Z",
    "build_snapshot" : false,
    "lucene_version" : "9.6.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

[alfredosola]

Make sure you are talking the right protocol. I think the .deb package published by OpenSearch (not sure if this is the default everywhere) talks https out of :9200, but by default Zammad expects http.

[sebix]

Then zammad run rails r "Setting.set('es_url', 'https://localhost:9200')" helps.

[fliebe92]

You’re mixing up things here.

I was not talking about end of file reached etc., this was @alfredosola initial post I guess.

The error I saw was really related to creating the needed pipeline for attachments, this fails.

[sebix]

Ah, okay. Isn’t the creation of pipelines included in zammad:searchindex:rebuild? What error did you get?

This is what the latest run showed:

# zammad run rake zammad:searchindex:rebuild
Dropping indexes... done.
Deleting pipeline... done.
Creating indexes... done.
Creating pipeline... done.
Reloading data...
  - Chat::Session...
    done in 0 seconds.
  - Cti::Log...
    done in 0 seconds.
  - Group...
    done in 0 seconds.
  - KnowledgeBase::Answer::Translation...
    done in 8 seconds.
  - KnowledgeBase::Category::Translation...
    done in 0 seconds.
  - KnowledgeBase::Translation...
    done in 0 seconds.
  - Organization...
    done in 8 seconds.
  - StatsStore...
    done in 0 seconds.
  - Ticket::Priority...
    done in 0 seconds.
  - Ticket::State...
    done in 0 seconds.
  - Ticket...
    done in 1044 seconds.
  - User...
    done in 39 seconds.   

[fliebe92]

@sebix> … after a restart of my OpenSearch Docker container it works. Thx for your help

[sebix]

FWIW: I have also run the complete Zammad CI test suite against OpenSearch and it succeeded:

docker: set user and password to admin by default

(using GitHub - sebix/zammad at opensearch and GitHub - sebix/zammad-docker-compose at opensearch, no modifications in the code, only in the tests)

[fliebe92]

It’s also important to install the needed ingest-attachment plugin in OpenSearch.

[sebix]

Of course. That’s equivalent like with ES:

/usr/share/opensearch/bin/opensearch-plugin install ingest-attachment

[YetAnotherGerrit]

ES8 doesn’t need the ingest-attachment anymore. Is there also an OS version like that?