Hi Dany,
We faced the same issue and it was due to different extraction configurations for the same table.
So let's say you have Extraction1 (for daily delta loads) and Extraction2 (for weekly full loads). And now you changed the extraction configuration in Extraction2 for a table so that certain columns won't be laoded.
As soon as you try to load Extraction1 in delta load it tries to fill the table, which was created during Extraction2 with data. But since you're not extracting all the necessary columns, this error shows up.
So make sure that tables that are extracted in several data loads are always configurated the same.
Best regards,
Johannes
Thank you Johannes for the response.
In my case I am using exactly the same data Job where I configure Delta Filters, to extract only what I need. There was no change on the extraction configuration (at least from me). I was wondering whether it might have a link with the system I am extracting from. The full extraction works without any error, but shortly after, without any modification, the Delta extraction fails.
This also happens if you don't "configure columns" (select specific columns within a table extraction)
and the underlying source system structure changes.
For example, if you're extracting a table and you don't configure your columns, your Celonis data table will have an underlying structure of N columns. If you are delta loading that table, and the source system table is now N+1, that '1' will not be in your Celonis data table and will throw an error because Celonis is trying to insert records into a table whose structures do not match.
To fix this, you can do two things.
- Execute a full load and reset the data structure
- In a transformation, execute this statement
- ALTER TABLE Ltable name] ADD COLUMN Mnew column] ndatatype]
The alter table statement will change the Celonis data structure to match what the source system structure is set to.
Hope this helps!