Email ticket doesn't work when Storage Mechanism is changed to S3

inevitable · December 24, 2023, 7:20am

Infos:

Used Zammad version: 6.2.0-1
Used Zammad installation type: docker-compose
Operating system: AMI 2023
Browser + version:

Expected behavior:

After moving the files(attachments) to the S3 bucket, be able to send and create tickets via email.

Actual behavior:

For some reason, when files(attachments) are moved successfully to the S3 bucket and changing the Storage Mechanism to S3, the creation of tickets via email doesn’t work. I checked the emails, and Zammad every time move emails from inbox.If I change back to the DB, works but attachments then will go to the DB again.

I tried both, with migrated data and without, and it’s the same. For migrated data, attachments work great with the correct endpoint to S3.

Steps to reproduce the behavior:

Create tickets via email when Storage Mechanism is S3(docker-compose).

fliebe92 · December 27, 2023, 12:26pm

Hi @inevitable. Could you please share some excerpts of your production.log file when the ticket creation fails?

dominikklein · December 27, 2023, 12:38pm

Did you really restart all the services after you switched the storage provider?

inevitable · December 27, 2023, 12:53pm

Yes. And somehow, creating tickets via API(Postman or custom Form tested) worked. Strangely, everything works great except making the tickets with mail. As I said, after changing to DB, Zammad can create tickets from email.

@fliebe92 Right now, I don’t have it. I needed to set up Zammad for production at that time so I removed it completely. Also, I checked the docker-compose logs and didn’t see anything about it. I’ll try to run the test again in a few days.

dominikklein · December 27, 2023, 1:04pm

API and Form are working with the Webserver and the Email-Handling is working with the Background-Worker, so this could be the difference.
But yes, in the end when the complete email handling is no longer working, the log file would be needed, to see additional information.
Are the emails afterward in the unprocessable-mail folder?

inevitable · December 27, 2023, 1:13pm

Nope. I tracked and saw the incoming emails, and they quickly vanished because of Zammad’s Email conf not to keep emails in the mailbox.
I believe that Zammad gets them and, for some reason, gets stuck with creating.

dominikklein · December 27, 2023, 1:15pm

I mean on Zammad side, normally when the mail handling fails, they are moved to the var/spool/unprocessable_mail folder.

inevitable · December 27, 2023, 2:00pm

I tried now, and there’s the unprocessed emails:

zammad@a7f54cac6be2:~$ ls var/spool/unprocessable_mail/
540f8fcda385a50a179ab41de767762e.eml  6ea565e099df8c1bcd9c92b93dbba2c8.eml

inevitable · December 27, 2023, 2:21pm

I succeed with Rails command Channel::EmailParsers.process_unprocessable_mails
Also, I see the attachments on the S3 bucket from now processed emails.
Interestingly, it seems to be an issue with the S3 configuration:

zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:13:02.311708#1-143360]  INFO -- : fetching imap (mail.DOMAIN.com/salestest@DOMAIN.com port=993,ssl=true,starttls=false,folder=INBOX,keep_on_server=false,auth_type=LOGIN)
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:13:07.765324#1-142220]  INFO -- : 2023-12-27T14:13:07+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [385c47e9-765f-4b95-b953-9675705f9d5e] from DelayedJob(default) with arguments: [246596] (id=328796) (queue=default) RUNNING
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:13:10.697026#1-142220] ERROR -- : 2023-12-27T14:13:10+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [385c47e9-765f-4b95-b953-9675705f9d5e] from DelayedJob(default) with arguments: [246596] (id=328796) (queue=default) FAILED (3 prior attempts) with Store::Provider::S3::Error: Simple Storage Service configuration not found or invalid.
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:13:10.697235#1-142220] ERROR -- : 2023-12-27T14:13:10+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [385c47e9-765f-4b95-b953-9675705f9d5e] from DelayedJob(default) with arguments: [246596] (id=328796) (queue=default) FAILED permanently because of 4 consecutive failures
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:13:15.135067#1-142220]  INFO -- : 2023-12-27T14:13:15+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [6f252f90-1d57-4c27-934f-7e2eac0ce74d] from DelayedJob(default) with arguments: [246598] (id=328802) (queue=default) RUNNING
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:13:17.346448#1-142220] ERROR -- : 2023-12-27T14:13:17+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [6f252f90-1d57-4c27-934f-7e2eac0ce74d] from DelayedJob(default) with arguments: [246598] (id=328802) (queue=default) FAILED (3 prior attempts) with Store::Provider::S3::Error: Simple Storage Service configuration not found or invalid.
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:13:17.346532#1-142220] ERROR -- : 2023-12-27T14:13:17+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [6f252f90-1d57-4c27-934f-7e2eac0ce74d] from DelayedJob(default) with arguments: [246598] (id=328802) (queue=default) FAILED permanently because of 4 consecutive failures
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:13:21.616187#1-142220]  INFO -- : 2023-12-27T14:13:21+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [e695a5f3-dc64-4641-9b34-7f22f07f2528] from DelayedJob(default) with arguments: [246600] (id=328808) (queue=default) RUNNING
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:13:23.491158#1-142220] ERROR -- : 2023-12-27T14:13:23+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [e695a5f3-dc64-4641-9b34-7f22f07f2528] from DelayedJob(default) with arguments: [246600] (id=328808) (queue=default) FAILED (3 prior attempts) with Store::Provider::S3::Error: Simple Storage Service configuration not found or invalid.
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:13:23.491421#1-142220] ERROR -- : 2023-12-27T14:13:23+0000: [Worker(host:34c8038ef0a2 pid:1)] Job TicketArticleCommunicateEmailJob [e695a5f3-dc64-4641-9b34-7f22f07f2528] from DelayedJob(default) with arguments: [246600] (id=328808) (queue=default) FAILED permanently because of 4 consecutive failures
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:13:34.447163#1-143360]  INFO -- : fetching imap (mail.DOMAIN.com/salestest@DOMAIN.com port=993,ssl=true,starttls=false,folder=INBOX,keep_on_server=false,auth_type=LOGIN)
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:14:06.163304#1-143360]  INFO -- : fetching imap (mail.DOMAIN.com/salestest@DOMAIN.com port=993,ssl=true,starttls=false,folder=INBOX,keep_on_server=false,auth_type=LOGIN)
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:14:37.660575#1-143360]  INFO -- : fetching imap (mail.DOMAIN.com/salestest@DOMAIN.com port=993,ssl=true,starttls=false,folder=INBOX,keep_on_server=false,auth_type=LOGIN)
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:15:09.471029#1-143360]  INFO -- : fetching imap (mail.DOMAIN.com/salestest@DOMAIN.com port=993,ssl=true,starttls=false,folder=INBOX,keep_on_server=false,auth_type=LOGIN)
zammad-docker-compose-zammad-scheduler-1      | I, [2023-12-27T14:15:11.414633#1-143360]  INFO -- : Process email with msgid '<AS2PR08MB10374088D894B177906A0954AF19FA@AS2PR08MB10374.eurprd08.prod.outlook.com>'
zammad-docker-compose-zammad-scheduler-1      | "ERROR: Can't process email, you will find it for bug reporting under /opt/zammad/var/spool/unprocessable_mail/14d42cef8dd5a77604216bb2ec44c9be.eml, please create an issue at https://github.com/zammad/zammad/issues"
zammad-docker-compose-zammad-scheduler-1      | E, [2023-12-27T14:15:11.676525#1-143360] ERROR -- : Can't process email, you will find it for bug reporting under /opt/zammad/var/spool/unprocessable_mail/14d42cef8dd5a77604216bb2ec44c9be.eml, please create an issue at https://github.com/zammad/zammad/issues
zammad-docker-compose-zammad-scheduler-1      | app/models/channel/email_parser.rb:299:in `block in _process'
zammad-docker-compose-zammad-scheduler-1      | app/models/channel/email_parser.rb:186:in `_process'
zammad-docker-compose-zammad-scheduler-1      | app/models/channel/email_parser.rb:125:in `block in process'
zammad-docker-compose-zammad-scheduler-1      | app/models/channel/email_parser.rb:124:in `process'

After running the S3.ping command, I get the true:

zammad@a7f54cac6be2:~$ rails r 'Rails.logger = Logger.new(STDOUT); pp Store::Provider::S3.ping?'
I, [2023-12-27T14:23:35.908264 #139]  INFO -- : ActionCable is using the redis instance at redis://zammad-redis:6379.
I, [2023-12-27T14:23:35.921141#139-5980]  INFO -- : Using memcached as Rails cache store.
I, [2023-12-27T14:23:35.921488#139-5980]  INFO -- : Using the Redis back end for Zammad's web socket session store.
true

Here’s the storage.yml config file:

zammad@a7f54cac6be2:~$ cat config/zammad/storage.yml
# This is the Zammad storage provider config sample. Copy this file to
# config/zammad/storage.yml and adopt your configuration.

# Amazon AWS S3 compatible simple storage service.
#
# For further information please see the official Amazon AWS S3 guide
# https://docs.aws.amazon.com/AmazonS3/latest/userguide
s3:
  access_key_id: 'fdsafpdsafds'
  secret_access_key: 'fdsafdsakoifsad'
  region: 'us-west-2'
  endpoint: 'https://s3.us-west-2.amazonaws.com'
  force_path_style: true
  bucket: 'zammad.test'

# The above configuration may also be provided in a URL-like format in this
# configuration file or as an environment variable named S3_URL.
# If the url is provided in the configuration file the environment variable is
# ignored.
#s3:
#  url: 'https://key:secret@s3.eu-central-1.amazonaws.com/zammad-storage-bucket?region=eu-central-1&force_path_style=true'

dominikklein · December 27, 2023, 6:31pm

I think when all services are restarted after the s3-storage config was added and the storage was changed it should be working.
But maybe it’s also something related to the docker-compose setup, I think we need at least to check, if there is a reproducible problem for such a switch.

As a side note, I think when it is a “fresh” installation you could also use the S3_URL to setup it up directly for S3.

system · December 21, 2024, 6:32pm

This topic was automatically closed 360 days after the last reply. New replies are no longer allowed.