That’s correct. Agents of different can only see a ticket in another group, if they at least have “read” rights. This would enforce your agent to keep the affected ticket open in a tab, because after wards he won’t be able to find it further.
Usually when taking turns it’s not required to restrict your agents from seeing the other groups content.
If it does, you can’t allow them to view the ticket. This is in my opinion technically required, because other wise you’d break restrictions in permissions.