How to make plugin delete entry? WHERE filtering

[L4stIdi0t]

pgsync 4.1.0
es 9.x.x
pgs 15

I was wondering if there is a way to delete entries using plugins

class ActivePlugin(plugin.Plugin):
    """Plugin to filter only active."""

    name = "Active"

    def transform(self, doc, **kwargs):
        """Filter out non-active listings."""

        # Return None to skip indexing this document
        if doc.get("status") != "ACTIVE" or doc.get("processing_state") != "SUCCESS":
            return None # So that this would not make it index the entry or delete it if it was indexed in ES

        return doc

but when an item gets set to inactive it is not removed from the elasticsearch, how is this possible?

[toluaina]

This isn't fully supported yet, but I've made a start on it.

Are you looking to run a plugin method when a document is deleted?
NB: since the document has already been removed from the database at that point, only the _id will be available to the plugin.
Also, it would make sense for the plugin method to be named delete() for delete operations

Let me know your thoughts.

[L4stIdi0t]

Are you looking to run a plugin method when a document is deleted?
No, i have a table with all items, i only want a certain subset to be searchable with elasticsearch

So i want to filter which items are added to ES, but when an item should no longer be in ES because of the filters i can not/dont know how to delete it

So if only green items can be in ES and an item changes from green to red i want to delete it from ES

_For the on delete function with only id available, it would be useless to me and in my opinion any logic for that should be handled around the postgres deletion call, like logging, archiving, etc

Or if you mean something is deleted from ES and then a plugin function is called? That could be useful for other people but then you should be able to customise the id in my opinion of just having all primary keys added together as id
(I would like to be able to set custom ids no matter if used for delete because it makes more sense to me to use that in my setup and with querying ES and then doing comparisons)

But i agree that it should be called delete or onDelete

[Askaiant]

@L4stIdi0t
Not sure if this is what you're looking for, but this might help you out:

class ActivePlugin(plugin.Plugin):
    """Plugin to filter only active."""

    name = "Active"

    def transform(self, doc, **kwargs):
        """Filter out non-active listings."""

        # Sets operation type to DELETE
        if not doc.get("active", True):
            return {'_op_type': 'delete'}

        # So if only green items can be in ES and an item changes from green to red, we delete it from ES
        if doc.get('colour', 'green') == 'red':
            return {'_op_type': 'delete'}

        return doc