Deleting a nested object deletes the root object in index

[boccheciampe]

Hi,

I have an index with nested objects (relationships), something like this:

@myindex.doc_type
class MyIndex(DocType):
    data = fields.NestedField(properties={
        'prop1': fields.StringField(),
        'prop2': fields.DateField(),
        [...]
    })

    class Meta:
        model = MyObject
        fields = [...]
        related_models = [Data]

    def get_instances_from_related(self, related_instance):
        return related_instance.myobject

Whenever a "data" object in inserted or updated in database, the "MyObject" index is updated. But when I delete a "data" object, the entire "MyObject" is deleted from the index, it is not updated with the "data" object removed.

Any idea ?

Thanks.

[sabricot]

Yes there is some bugs with deletion on related_models. I am working on it but i have some issues to manage every related objects deletion cases, especially the m2m. Thanks for the reporting.

[jezeniel]

@sabricot is this possible to resolve? Where are the problem areas?

[sabricot]

I made an update on this, is it working ?

[boccheciampe]

Sorry, it looks like it's not working as expected : adding or deleting a nested object does not update the root object index, I have to manually save() it to update the entry.

[sabricot]

can you show me your models.py (only the relations part) please ?

[boccheciampe]

Here it is :

class Person(models.Model):
[...]

class EmployeeState(models.Model):
[...]
person = models.ForeignKey(Person, related_name='employee_states',
db_index=True, to_field='id')

The 'Person' class is my root object to be indexed :

@persons.doc_type
class PersonIndex(DocType):
id = fields.IntegerField()
[...]
employee_states = fields.ObjectField()

[sabricot]

I don't know, with this kind of relations, is it working for me.

Can you show me the complete class PersonIndex(DocType): with the def get_instances_from_related(self, related_instance):

And give me your django-elasticsearch-dsl, elasticsearch-dsl, django, elasticsearch and python version ? please :)

[boccheciampe]

Here are the requested details. I'll try to regenerate my index asap, just to be sure.

django-elasticsearch-dsl==0.4.4
elasticsearch-dsl==5.4.0
elasticsearch-5.6.2
Python 3.5.4
Django==1.8.18

@persons.doc_type
class PersonIndex(DocType):
id = fields.IntegerField()
last_updated = fields.DateField()
data = fields.ObjectField()

employee_states = fields.ObjectField()
external_states = fields.ObjectField()
student_states = fields.ObjectField()
addresses = fields.ObjectField()
accounts = fields.ObjectField()
functions = fields.ObjectField()
degrees = fields.ObjectField()

def __init__(self, *args, **kwargs):
    functions_list = ['employee_states', 'external_states', 'addresses',
                      'accounts','functions','degrees']

    # Generate the prepare_* functions
    for f in functions_list:
        _method = self.make_method(f, attrs[f])
        setattr(PersonIndex, "prepare_"+f, _method)

    super(PersonIndex, self).__init__(*args, **kwargs)

def make_method(self, name, attrs):
    def _method(self, instance):
        objects_list = []

        for o in getattr(instance, name).all():
            d = {}
            for attr in attrs:
                d[attr] = getattr(o,attr)

            for k in o.data.keys():
                d[k] = o.data[k]

            objects_list.append(d)

        return objects_list
    return _method

def prepare_data(self, instance):
    keys = Person.GOLDEN_RECORDS_FIELDS

    d = {}

    for k in keys:
        d[k] = instance.data.get(k, None)

    return d

def get_instances_from_related(self, related_instance):
    return related_instance.person

class Meta:
    model = Person
    related_models = [EmployeeState]

[boccheciampe]

I may be wrong, but when a nested object is deleted (or added), it looks like the root object is reindexed without the nested object change (eg. before applying the add/delete). It would explain why the index still looks the same.

When I delete a nested object, in class DocType(DSLDocType) (documents.py), line 223, the self._get_actions(object_list, action) returns the "previous" data, the nested object is still there.