DVLinkToSQL_IncrementalCopy - Not processed records and missing deletion records

[juanitodynamo]

Hello! We have implemented the incremental csv from synapse serverless to azure sql dedicated. We experience outdated records in target db, which do not appear in dynamics.

There are records that appear in the dedicated server which do not have a deletion record in synapse, even though there is no functional data in dynamics for it.

Shouldn't we have gotten a deletion record given this scenario? Even by searching this particular id we do not get the IsDelete = 'True' record.

There are also records that appear in the target db for which there is a deletion record in synapse, but it was not processed somehow.

We have implemented the pipeline as described in the documentation. Nonetheless, we want to validate if the Dedup_and_Merge step is being executed correctly based on the result logs we are getting (given the second scenario we are experiencing)

{ "resultSetCount": 0, "recordsAffected": 1062, "resultSets": [], "outputParameters": {}, "outputLogs": "--De-duplicate the data in dbo._new_purchline--\n-- Merge data from _new_purchline to purchline----\n", "outputLogsLocation": "", "outputTruncated": false, "effectiveIntegrationRuntime": "AutoResolveIntegrationRuntime (UK South)", "executionDuration": 6, "durationInQueue": { "integrationRuntimeQueue": 10 }, "billingReference": { "activityType": "PipelineActivity", "billableDuration": [ { "meterType": "AzureIR", "duration": 0.016666666666666666, "unit": "Hours" } ] } }

Are the resultSetCount and resultSets attributes expected to have some values? As far as we can see, all the steps have 0 on both, despite all having run successfully.

Thanks!