Zammad Upgrade to 7.0 - i-doit integration SSL error 80

Infos:

  • Used Zammad version: 7.0.0-1772600692.f3228274.centos9
  • Used Zammad installation type: package
  • Operating system: AlmaLinux 9.7
  • Browser + version: Brave, Firefox, Edge.

Expected behavior:

  • Browsing and attaching i-doit CIs in tickets

Actual behavior:

  • “Loading failed”

Steps to reproduce the behavior:

  • Update Zammad from last 6.5.2 to 7.0.0

Checking i-doit connection from integrations show the following error:

Can’t fetch objects from https://idoit.mydomain.com/i-doit/src/jsonrpc.php: #<OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 peeraddr=[xxxx:xxx:xxxx:xxxx::x]:443 state=error: tlsv1 alert internal error (SSL alert number 80)>

With VM rollback to 6.5.2, integration is working again.

Could be that it’s related to the OpenSSL update in Zammad 7. Which means the new OpenSSL versions are now using the current, more modern defaults which should be used these days.

Seems to be that the Idoit-Server is using an older OpenSSL version and should be updated.

The ciphers on the target system looking good, it is an AlmaLinux 9.7 System using Apache. Apache is using the systems ciphers.

SSLCipherSuite PROFILE=SYSTEM
SSLProxyCipherSuite PROFILE=SYSTEM

update-crypto-policies --show
DEFAULT

Ok, interesting.

Maybe you can check the Apache logs for some errors related to the requests from the Zammad side?

Another idea would be to check the connection outside of the Zammad context:

“openssl s_client -connect idoit.mydomain.com:443

Zammad tries to connect the wrong server, for whatever reason, it falls for a *.mydomain.com DNS WIldcard and not idoit.mydomain.com.

Did DNS resolve change in 7.0? if i roll back, it resolves idoit. On my Zammad Testserver in see the same issues, as soon as a update to 7.0, Zammad prefers to resolve to a wildcard and connects an unrelated reverse proxy server in the internet.

From command line, i can properly resolve idoit, modifing hosts has no effect. Is Zammad maybe using public DNS Servers? or something is really wrong with resolve in ruby.

If the dns records are the same, then it‘s most likely the virtual host that is somewhat incompatible.

SNI is a possible culprit here.

Hi @tineidae. Could you please post more information? What are you doing in the command line, what do you see in the logs of Zammad etc.?

We’re not able to help you with the given information, sorry.

I found the Problem, let me explain:

All my systems are dualstack IPv4 and IPv6, same for Zammad and i-doit. I have a wildcard in my domain (*.mydomain.com), it is related to some netbird proxy stuff and has nothing to do with Zammad/i-doit, but leads to an interesting issue with 7.0.

idoit.mydomain.com currently only has an A record. Normal behaviour on linux is to look for AAAA, and then A. If nothing is found, then the wildcard in my domain wins.

What happens in Zammad?

Behaviour on 7.0: AAAA record idoit.mydomain.com does not exist, it uses the wildcard in my domain and talks to a wrong server where the wildcard points to.

Behaviour on 6.5.2: AAAA record idoit.mydomain.com does not exist, Zammad uses the A records and connects via IPv4 to i-doit.

My solution: provide a proper AAAA record for idoit.mydomain.com, now 6.5.x and 7.0 can connect to i-doit, and both do this via IPv6 now.

For me this looks like a bug in the resolving mechanism between 6.5.2 and 7.0, Resolver should always look for AAAA and A records, but never use a wildcard if one of the records A or AAAA exist. I can go forth and back between 6.5.2 and 7.0.0, always repeatable.

So the issue is essentialy fixed for me, by putting AAAA records in my idoit.mydomain.com, but the behaviour is imho wrong.

You were luck it worked by ‚accident‘. For me this is invalid DNS configuration not a bug personally. Possibly rails behaves slightly different - but esentially Zammad always prefers IPv6 over v4 if applicable.

For “DNS configuration issue”..

I just created foobar.mydomain.com to show you how a linux system is handling this situation. The notexist.mydomain.com does neither A or AAAA records.

foobar.mydomain.com only has an A record to simulate the issue i had…

ping -4 foobar.mydomain.com
PING foobar.mydomain.com (10.49.xx.xx) 56(84) bytes of data.
64 bytes from foobar.mydomain.com (10.49.xx.xx): icmp_seq=1 ttl=64 time=0.178 ms

ping -6 foobar.mydomain.com
ping: foobar.ba.valantic.com: Address family for hostname not supported

Correct behaviour, dns server tells the resolver that there is no AAAA record, but it does not send the wildcard information because it knows there is an A record and that the hostname exist, but not for the requested type AAAA.

Now lets use something that does not have A or AAAA.

ping -4 notexist.mydomain.com
PING (91.xx.xx.xx) 56(84) bytes of data.
64 bytes from static.xx.xx.xx.xx.clients.your-server.de (91.xx.xx.xx): icmp_seq=1 ttl=50 time=13.7 ms

ping -6 notexist.mydomain.com
PING notexist.mydomain.com(2a01:xxx:xxxx:xxxx::x (2a01:xxx:xxxx:xxxx::x)) 56 data bytes
64 bytes from 2a01:xxx:xxxx:xxxx::x (2a01:xxx:xxxx:xxxx::x): icmp_seq=1 ttl=50 time=12.1 ms

Now dns provides the adresses of the wildcard, that makes sense as there is no A or AAAA record.

So, you may be right and rails do stuff differently, but only affects 7.0, so something changed, new rails? i dont know what does resolve in rails.

I know how wildcard DNS is working. :slight_smile:

Yes, ruby and rails were bumped with 7.0.

Possible this one, but i am not a coder, IPv6 fallback to IPv4 broken by a8d1d94 in 0.5.0 · Issue #68 · ruby/resolv · GitHub

7.0 has resolv 0.6.2
6.5.2 has resolv 0.2.2

I am stop nagging you guys, for me it looks like a ruby issue, thanks for the help :slight_smile:

1 Like

Yes, Ruby 3.4 is doing things differently; because of that, you’re seeing the difference after the update.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.