Freshdesk migration fails on missing customer_id

Infos:

  • Used Zammad version: 6.4
  • Used Zammad installation type: docker-compose
  • Operating system: MacOS
  • Browser + version: Chrome (Latest)

Expected behavior:

  • Freshdesk migration should complete successfully

Actual behavior:

  • Freshdesk migration of most of the tickets fails on database errors

Steps to reproduce the behavior:

I’m testing the freshdesk migration on my local setup with docker compose.
I have tried both the browser and console migration option according to the documentation.

I currently have the Growth plan activated on Freshdesk. There are about 3000 users and 4500 tickets in the freshdesk.

When I run the migration, it starts by importing Users. I noticed that instead of importing all users it only imports 106 of them.

Then it tries to import the tickets but it fails on most of them. See this import_status after 10 minutes:

{
    "id": 1,
    "name": "Import::Freshdesk",
    "dry_run": false,
    "payload": {},
    "result": {
        "Groups": {
            "skipped": 0,
            "created": 1,
            "updated": 0,
            "unchanged": 0,
            "failed": 0,
            "deactivated": 0,
            "sum": 1,
            "total": 1
        },
        "Organizations": {
            "skipped": 0,
            "created": 1,
            "updated": 0,
            "unchanged": 0,
            "failed": 0,
            "deactivated": 0,
            "sum": 1,
            "total": 1
        },
        "Users": {
            "skipped": 0,
            "created": 106,
            "updated": 0,
            "unchanged": 0,
            "failed": 0,
            "deactivated": 0,
            "sum": 106,
            "total": 106
        },
        "Tickets": {
            "skipped": 0,
            "created": 16,
            "updated": 0,
            "unchanged": 0,
            "failed": 548,
            "deactivated": 0,
            "sum": 564,
            "total": 600
        }
    },
    "started_at": "2024-11-17T11:40:51.240Z",
    "finished_at": null,
    "created_at": "2024-11-17T11:40:48.396Z",
    "updated_at": "2024-11-17T11:58:44.366Z"
}

Looking into logs using docker-compose logs -f zammad-scheduler I can see two things happening:

Insert is failing because customer_id is missing:

PG::NotNullViolation: ERROR:  null value in column "customer_id" of relation "tickets" violates not-null constraint
zammad-docker-compose-zammad-scheduler-1  | DETAIL:  Failing row contains (669, 1, 1, 4, null, 1687, Some ticket subject, 5, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, Request, null, null, 1, 1, 2022-12-21 19:33:35, 2022-12-23 15:43:27, null).

I’m also getting rate limited, which is I guess expected:

Rate limit: {"date"=>["Sun, 17 Nov 2024 12:02:25 GMT"], "content-length"=>["0"], "connection"=>["keep-alive"], "x-fw-ratelimiting-managed"=>["true"], "retry-after"=>["54"], "x-envoy-ratelimited"=>["true"], "x-envoy-upstream-service-time"=>["6"], "x-trace-id"=>["###"], "nel"=>["{ \"report_to\": \"nel-endpoint-freshdesk\", \"max_age\": 2592000, \"include_subdomains\": true}"], "report-to"=>["{ \"group\": \"nel-endpoint-freshdesk\", \"max_age\": 2592000, \"include_subdomains\": true, \"endpoints\": [{\"url\": \"https://edge-admin.eu-central-1.freshedge.net/nelreports/freshdesk\"}]}"], "server"=>["fwe"], "x-request-id"=>["#####"]} (429 Too Many Requests). Sleeping 64 seconds and retry (#1/10)

My guess is that the users import somehow failed, and now the system fails to find customer_id for most of the tickets and fails to import the tickets. Has anyone experienced similar issue?

Thanks a lot
Tomas

You should check the related ticket in freshdesk. Which requester is present?

Since last year, there has been a new flag “skip_initial_contacts”, but this is off per default. So should not influence the import of the users. Normally all users should be imported before the ticket import is started.

You could check the API directly:

“https://???.freshdesk.com/api/v2/contacts?per_page=100”

Hello Dominik,
thanks for your reply.

I have wrote a simple script accessing the contacts from Freshdesk api and I’m able to access all 3000 contacts using the endpoint you mention. So I don’t think the problem is on the Freshdesk side.

Any other idea how else I could debug this?

Thanks

How many agents are included in these 3000 contacts? It would be nice to understand what is not fetched. Normally without changing some “hidden” features in a manual “import” it should fetch all contacts before importing any ticket.