Not on Zammad, but on another system we use I have hidden the login form to push people towards the 3rd party login.
On zammad’s login page there is a form id='login' which contains just the form - if you add/modify the CSS so #login has display:none; then it should disappear.
You could add a custom CSS to /assets/custom and use nginx to inject it onto the login page - a trick I picked up from this thread which has the details.