Rebuilding the searchindex ist not possible: No processor type exists with name [attachment]

[bitshift]

Infos:

• Used Zammad version:

  $ rpm -q zammad
  zammad-3.3.0-1586347592.893c528d.centos7.x86_64
  
  $ rpm -q elasticsearch
  elasticsearch-7.6.2-1.x86_64
  

  However, previous two versions were also affected on this instance.

• Used Zammad installation source: package

• Operating system: CentOS Linux release 7.7.1908 (Core)

• Browser + version: does not matter in this context.

Expected behavior:

Searchindex is functional and can be rebuilt using zammad run rake searchindex:rebuild.

Actual behavior:

zammad run rake searchindex:rebuild fails with

drop indexes...done
delete pipeline (pipeline)... done
create indexes...done
create pipeline (pipeline)... rake aborted!
Unable to process put request to elasticsearch URL 'http://localhost:9200/_ingest/pipeline/zammad878688337864'. Check the response and payload for detailed information: 

Response:
#<UserAgent::Result:0x000055f37a3cb2c0 @success=false, @body="{\"error\":{\"root_cause\":[{\"type\":\"parse_exception\",\"reason\":\"No processor type exists with name [attachment]\",\"processor_type\":\"foreach\",\"suppressed\":[{\"type\":\"parse_exception\",\"reason\":\"No processor type exists with name [attachment]\",\"processor_type\":\"foreach\"}]}],\"type\":\"parse_exception\",\"reason\":\"No processor type exists with name [attachment]\",\"processor_type\":\"foreach\",\"suppressed\":[{\"type\":\"parse_exception\",\"reason\":\"No processor type exists with name [attachment]\",\"processor_type\":\"foreach\"}]},\"status\":400}", @data=nil, @code="400", @content_type=nil, @error="Client Error: #<Net::HTTPBadRequest 400 Bad Request readbody=true>!">

Payload:
{:description=>"Extract zammad-attachment information from arrays", :processors=>[{:foreach=>{:field=>"article", :processor=>{:foreach=>{:field=>"_ingest._value.attachment", :processor=>{:attachment=>{:target_field=>"_ingest._value", :field=>"_ingest._value._content", :ignore_failure=>true, :ignore_missing=>true}}, :ignore_failure=>true, :ignore_missing=>true}}, :ignore_failure=>true, :ignore_missing=>true}}, {:foreach=>{:field=>"attachment", :processor=>{:attachment=>{:target_field=>"_ingest._value", :field=>"_ingest._value._content", :ignore_failure=>true, :ignore_missing=>true}}, :ignore_failure=>true, :ignore_missing=>true}}]}

Payload size: 0M
/opt/zammad/lib/search_index_backend.rb:889:in `make_request_and_validate'
/opt/zammad/lib/search_index_backend.rb:84:in `block (2 levels) in processors'
/opt/zammad/lib/search_index_backend.rb:68:in `each'
/opt/zammad/lib/search_index_backend.rb:68:in `block in processors'
/opt/zammad/lib/search_index_backend.rb:65:in `each'
/opt/zammad/lib/search_index_backend.rb:65:in `processors'
/opt/zammad/lib/tasks/search_index_es.rake:95:in `block (2 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:67:in `block (2 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:178:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)>'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
/opt/zammad/vendor/bundle/ruby/2.5.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)>'
/opt/zammad/bin/bundle:3:in `load'
/opt/zammad/bin/bundle:3:in `<main>'
Tasks: TOP => searchindex:rebuild
(See full trace by running task with --trace)

Steps to reproduce the behavior:

Sadly I don’t know… which kind of data I have to deed an instance to trigger that. If I shall try something, I am happy to experiment

Regards,
Andreas

[dvnkln]

Kinda sounds like your ingest-attachment for elasticsearch is missing.
Can you please make sure it’s installed correctly?

cheers

[bitshift]

Hi,

thanks for helping me out. You are 100% correct.

Obviously, the vendor repository shipped a new version via elasticsearch package and Elasticsearch got upgraded from 7.5 to 7.6. During this, the ingest-attachment plugin got lost.

So the following indeed fixed the problem:

$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-attachment
$ sudo systemctl restart elasticsearch.service
$ zammad run rake searchindex:rebuild

Regards,
Andreas