Add endpoint to manually enrich match with full details.

[apognu]

Open Sanctions's match API returns trimmed down information for each match, we need to provide a way (automatic or manual, depending on whether the API is self-hosted or paid) to enrich it with the full information we can retrieve from the API.

TODO:

• Manual endpoint to enrich a match

The periodic worker will be added in a separate PR.

[apognu]

I am changing the scope of this PR to only include the manual action or enriching the data (the part the frontend depends on).

The worker part will be handled in a separate PR.

[apognu]

Do we want to store that an entity was enriched? My initial thinking was that there might be use cases of enriching data again (if there are updates to a profile), but I am asking myself the question, now.

[Pascal-Delange]

Do we want to store that an entity was enriched?

It depends, if we keep a copy of the initial payload (for audit etc) then we implicitly know it was enriched ?

My initial thinking was that there might be use cases of enriching data again (if there are updates to a profile), but I am asking myself the question, now.

For that part I'd say definitely no, indeed we'd like the enriched version to be stored as soon as possible after the initial one.

[Pascal-Delange]

the logic looks sound, let me just confirm with Arnaud if he thinks the original payload should be kept as part of an audit trail

[apognu]

For that part I'd say definitely no, indeed we'd like the enriched version to be stored as soon as possible after the initial one.

Which will not work in the case of manual enrichment. Depending on the user's process for review, it might be enriched quite some time after the fact.

In any case, if we need to keep the previous payload, I'll set this back to draft, since right now I overwrite the data. Same thing if we only want to allow one enrichment (be it manual or automatic), I will need to store the fact that it was enriched.

[Pascal-Delange]

Or we could - but let me know if you think this is a weird pattern or is going to be hard to maintain - just get the json keys from the enriched payload that are not in the old one, and add them to the stored json?

[apognu]

I do not see how that would solve our issues, we would still not know which attributes were updated due to enrichment, or if it was enriched at all. And it adds complexity for not a lot of gain.

My guess is, if we need to keep the original, we will copy it in an original_payload column or similar?

[Pascal-Delange]

🚀