Running a standard transformation, Im encountering an occasional source data error (e.g. an alpha in an INTEGER column). Of course, the script halts immediately. The source environment is highly customized and there will be a few of these.
DataException: [Vertica]VJDBC ERROR: Could not convert R03B802 from column table.numeric_field to a float8
This error message does not reveal a row number.
Im loading over 20M records, so going back to source to locate and fix each data anomaly and then going through a long reload each time ugh.
I cant get to the juicy part of this great tool because of the time spent returning to source data, one element at a time, cleansing and reloading.
Im new to this, but it seems like Im missing something simple here. Is there a way to patch data post-extraction?