I would also say I have this problem.
For my system I had to do a workaround where everyone except for guest users is technically an agent.
You’ve got a base tier of agents that can send tickets to a first tier of groups. Give them at least visibility into the other groups so that they can see tickets that get escalated. For tickets that get escalated, I add roles that correspond to that agent’s function so that they can edit/take ownership of the ticket.
Hopefully this makes sense. I think it’s a very mid workaround, as most ticketing systems I’ve seen have a customer and an agent portal that one can switch between. Maybe one day