Extractions - Performance issues
Hello,
I am extracting data from a ServiceNow system. Sometimes it's quick and sometimes it takes a long time.
To the point
"Final Count: Loaded..."
everything is always ok, but for the last steps
"Starting load incident$sys_audit to target ..."
until
"Extraction successful"
it is very different.
2022-01-11 08:59:15 INFO Extraction successful
2022-01-11 08:59:15 INFO Table incident$sys_audit loaded ..
2022-01-10 21:36:52 INFO Inserting extracted data into the Data Pool (this may take a while)
2022-01-10 21:36:52 INFO Starting load incident$sys_audit to target ...
2022-01-10 21:34:18 INFO Final Count: Loaded 5,450,395 record(s) for table incident$sys_audit
2022-01-10 21:33:53 INFO Loading 12,276 records for table incident$sys_audit...
In the best case, it takes 20 minutes and in the worst case >14 hours for extraction to be successful
What can be the reason? How can it be fixed
Thank you
Heiko