Error with Django internal when using pytest >= 4.2

[debnet]

Hello,

Since I updated pytest to 4.2, I've got errors inside Django internals for some tests.
These tests worked fine with 4.1 and still work fine with Django embedded unittests.

Here the error:

______________________________________________________________________________ UserPageCheckTest.test_get_view_user_by_organisation _______________________________________________________________________________

self = <drop.front.tests.test_pages.UserPageCheckTest testMethod=test_get_view_user_by_organisation>, url = <URLPattern 'user/<int:pk>/' [name='view_user']>, pk_required = True, login_required = True

    def test_get_by_organisation(self, url=url, pk_required=pk_required, login_required=login_required):
>       base_url = self.get(url, pk_required, login_required)

drop/front/tests/test_pages.py:69: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
drop/front/tests/test_pages.py:49: in get
    self.client.force_login(self.user)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/test/client.py:611: in force_login
    self._login(user, backend)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/test/client.py:624: in _login
    login(request, user, backend)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/contrib/auth/__init__.py:132: in login
    user_logged_in.send(sender=user.__class__, request=request, user=user)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/dispatch/dispatcher.py:175: in send
    for receiver in self._live_receivers(sender)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/dispatch/dispatcher.py:175: in <listcomp>
    for receiver in self._live_receivers(sender)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/contrib/auth/models.py:20: in update_last_login
    user.save(update_fields=['last_login'])
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/contrib/auth/base_user.py:73: in save
    super().save(*args, **kwargs)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/db/models/base.py:718: in save
    force_update=force_update, update_fields=update_fields)
../../.pyenv/versions/drop/lib/python3.7/site-packages/django/db/models/base.py:748: in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <User: admin>, raw = False, cls = <class 'drop.core.models.base.User'>, force_insert = False, force_update = False, using = 'default', update_fields = frozenset({'last_login'})

    def _save_table(self, raw=False, cls=None, force_insert=False,
                    force_update=False, using=None, update_fields=None):
        """
        Do the heavy-lifting involved in saving. Update or insert the data
        for a single table.
        """
        meta = cls._meta
        non_pks = [f for f in meta.local_concrete_fields if not f.primary_key]
    
        if update_fields:
            non_pks = [f for f in non_pks
                       if f.name in update_fields or f.attname in update_fields]
    
        pk_val = self._get_pk_val(meta)
        if pk_val is None:
            pk_val = meta.pk.get_pk_value_on_save(self)
            setattr(self, meta.pk.attname, pk_val)
        pk_set = pk_val is not None
        if not pk_set and (force_update or update_fields):
            raise ValueError("Cannot force an update in save() with no primary key.")
        updated = False
        # If possible, try an UPDATE. If that doesn't update anything, do an INSERT.
        if pk_set and not force_insert:
            base_qs = cls._base_manager.using(using)
            values = [(f, None, (getattr(self, f.attname) if raw else f.pre_save(self, False)))
                      for f in non_pks]
            forced_update = update_fields or force_update
            updated = self._do_update(base_qs, using, pk_val, values, update_fields,
                                      forced_update)
            if force_update and not updated:
                raise DatabaseError("Forced update did not affect any rows.")
            if update_fields and not updated:
>               raise DatabaseError("Save with update_fields did not affect any rows.")
E               django.db.utils.DatabaseError: Save with update_fields did not affect any rows.

../../.pyenv/versions/drop/lib/python3.7/site-packages/django/db/models/base.py:816: DatabaseError

As you can see, the error occurs when updating the last login date inside Django, but it happens only for few tests, which is very odd.

Here my current requirements:

Django==2.1.7
Pillow==5.4.1
pytest==4.3.1
pytest-cov==2.6.1
pytest-django==3.4.8
pytest-env==0.6.2

Thanks for any help or insight provided. ;)

[blueyed]

Please git-bisect it.
Might be related to 5a9405b.
A minimal reproducible example would also be helpful.

[debnet]

Trying to bisect both pytest and pytest-django with no success because of the dependency between pytest>4.1 and pytest-django>3.4.4.

When bisecting pytest until 4.1.1 with latest pytest-django, I got errors with database transactions.
And bisecting pytest-django with the latest pytest leads to incompatibilities, too noisy.

As I said, the last version of pytest-django works fine with pytest==4.1.1 but not with pytest>=4.2, but it works fine with older versions of pytest-django too.

I'm kind of lost here, and I'm not a specialist unfortunalty.

[debnet]

By the way, ignoring the others fails and errors by git bisect and focusing on the tests which fails with the newer version tends to give the following commit as guilty of charges:

(python) [marc@marcbook pytest-django]$ git bisect good
5a9405ba8c1f2f042042c3c7e5a16679dfdb7e8a is the first bad commit
commit 5a9405ba8c1f2f042042c3c7e5a16679dfdb7e8a
Author: Daniel Hahler <github@thequod.de>
Date:   Sun Feb 3 19:54:38 2019 +0100

    Fix disabling/handling of unittest methods with pytest 4.2+ (#700)
    
    Fixes https://github.com/pytest-dev/pytest-django/issues/698.

:100644 100644 c73d1107d289aea3582665968fea6d45da3c0031 413fac5b2cd2da30970d6345ed700a6114a5c688 M	.travis.yml
:040000 040000 daa7c0b05d433bfa5b737cc16cca94c5dc68899b 3738a9fd2957b249bf78d1900980ccf35326d390 M	pytest_django
:100755 100755 c6e66fd87b0fc2db60903513f202917f7c7deec8 d38a89020a1ce66d38e966888e80e79f9251cad0 M	setup.py
:040000 040000 5439ab125e9feca105cfeff3845c6f9772183e6b 9112fbe0f45ff011b313c5999c48d8ab44a140f4 M	tests
:100644 100644 0582d0f7786f91712502283d83b805c9f4db5b4b 1265bda1d4dd4882f1f222ea3bad388ca6852787 M	tox.ini```

[blueyed]

Thanks for the info - this is what I suspected.

1. Can you provide a minimal example to reproduce this?
2. Does it this fail with latest pytest/pytest-django if you force this to be True? 5a9405b#diff-87c0c2cb9262dd6c4384e140e51d8e64R54

[debnet]

OK, thanks. I'll try the second tomorrow and give you a feedback soon.
For the first, I encountered the same issue in another project, I will try to remove unnecessary parts.

[debnet]

Sorry for the delay. I confirm the tests pass with this constant forced to True.
But previously passing tests in my case now fail (and some are in error due to MPTT). :D
Do you have any clue?

[evenicoulddoit]

I also just came across this when bumping dependencies for my DRF package.

I didn't manage to bisect, only to confirm that the issue was definitely present in 4.2.0, and not in 4.1.1. I was going to look into it further, and started reviewing the changes between said versions, but before going deeper I decided to bump this package to its latest, and it worked fine, so I guess I'd consider the issue solved.

The test failure I had only occurred when running the entire suite, not on a subset. I was using the Django override_settings() decorator to provide settings on a per-test basis, and the non-overloaded test was the one which was failing, which suggests some kind of setup/teardown issues. Before bumping the versions I was able to fix the issue by applying the decorator with the setting I wanted to test missing, but that would have been an ugly solution.

I could apply a bisect should you need, but I'd consider this closed?

[blueyed]

@debnet
Please try it with latest pytest, too.

[debnet]

Same error with pytest==4.4.1.

[blueyed]

@debnet
Thanks for the update.
Can you provide a minimal example?

It would be great to have it in a test for pytest-django, e.g. based on pytest's testdir fixture.

[blueyed]

Sorry for the delay. I confirm the tests pass with this constant forced to True.

Interesting.

But previously passing tests in my case now fail (and some are in error due to MPTT). :D
Do you have any clue?

How do those fail then?

The problem here is that pytest itself added handling of unittest classes, but apparently it behaves a bit different.

We really need a way to reproduce this to being able to fix this. The best would be to have a failing test in pytest-django's testsuite, of course.
I am not using unittest myself typically, but could imagine that e.g. using pytest with Django's own tests might also run into this (I've started experimenting with using pytest there, but there is not much interest apparently (https://code.djangoproject.com/ticket/30415)).