-
-
Notifications
You must be signed in to change notification settings - Fork 182
Refactor TestCases to mixins #5961
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Using UserProfileWithParentsFactory means you are not a SimpleTestCase. This is the cause of a number of test failures.
😱 This is why SimpleTestCase isn't supposed to have `databases` set. Although most of the tests that inherited from these also inherited from TestCase, wrapping their database access in transactions, cl.search.tests.tests_es_opinion.OpinionSearchAPICommonTests, cl.search.tests.tests_es_oral_arguments.OASearchAPICommonTests, cl.search.tests.tests_es_person.PeopleSearchAPICommonTests, and cl.search.tests.tests_es_recap.RECAPSearchAPICommonTests did not. I recognize a few of those from random test failures I've run into.
RecapFetchApiSerializationTestCase has created database objects and lost its Simple privileges.
RECAPDocumentSignalTests and RECAPDocumentReceiverTests have been found guilty of saving to the database and have been stripped of their Simple rank.
AlertAPITests goes directly to TestCase jail, does not change the database, and does not save 200ms.
For the crime of ruining countless test runs over the last four years, SimpleTestCase is sentenced to summary erasure by backspace key.
APITests was trying to secretly access the database without anyone knowing about it, but its malfeasance has been exposed.
DocketAlertAPITests broke the database and has to pay extra test time for restitution.
APIVisualizationTestCase has been hiding its true nature for a while, but it can no longer deny its attraction to modifying databases and it is out and proud to be a TestCase.
No longer will WebhooksHTMXTests terrorize test results with its carelessly created users and profiles.
Added an import alias to avoid having two APITestCase symbols.
OneDatabaseMixin is replicating what Django test cases other than SimpleTestCase have done for six years and unfortunately extending database access to tests without any the protections of TransactionTestCase or TestCase. This is the root cause of a lot of random test failures.
FilteringCountTestCase isn't a test case, it's a mixin for test cases that provides a new assert method. Now its name matches its function.
It works! I guess I missed CitationTextTest, but it failed when it tried to modify the database as a SimpleTestCase. This upgrades it to a TestCase so it can finish its mission.
AudioTestCase already inherits from TestCase, so NoteTest shouldn't directly inherit from it.
Add some missing super() calls for setup and teardown test methods.
You should never cross the streams of TransactionTestCase and TestCase. All of OpinionSearchFunctionalTest's data objects were being put in a transaction that Selenium couldn't see. This is fallout from AudioTestCase previously just adding stuff to the live database. I switched it from TestCase to TransactionTestCase and verified that nothing else importing it was also using TestCase.
This isn't good. I've seen a bunch of these missing and I'm guessing there's still more.
All that and OpinionSearchFunctionalTest doesn't use AudioTestCase. I looked at other classes using AudioTestCase and decided to switch it back to a TestCase.
4113c16
to
972fac7
Compare
The reason the parent Django class, SerialMixin, doesn't define the `__lockfile__` is because everything that inherits from it is then using the same lock. So your Selenium tests would be locked when you're doing ElasticSearch tests, etc. Better to just use the Django one and deal with `__lockfile__` collisions as they occur.
for more information, see https://pre-commit.ci
972fac7
to
a7a6851
Compare
I got my ordering wrong. Because Django TestCase doesn't call super ().setUpTestData(), it ends the chain. Any mixins have to go ahead of it or their methods won't be called.
RestartSentEmailQuotaMixin is only used by a couple tests in users, so I moved it next to where it is used. If it gets used elsewhere, then we call pull it out into the common mixins.
I've added an ESIndexTransactionTestCase and made ESIndexTestCase a subclass of that and TestCase. I then converted all the subclasses to the appropriate class. Also converted CountESTasksTestCase to a mixin.
615321b
to
b3e3e7f
Compare
cc6748f
to
02214ca
Compare
for more information, see https://pre-commit.ci
I think the conversion to mixins feels about right. Two thoughts I had:
|
Yeah, now that I've got most things converted to mixins, I'm going to start pulling out ones that are more model specific. If they're only used by one app's tests, I'm just moving them into the tests file like this. Although I should move that to the top of the file. The
I forked this from #5954 for two reasons: I didn't want to duplicate work from it and this isn't worth looking at until that is merged, anyway: that one actually fixes broken tests, this one is just a refactor to simplify some things. I just put it up as a draft to be able to point to some specific commits to make sure this was going in the right direction. I don't want to ambush you with a massive refactor that you don't want. And I don't expect this to be the final change. If you decide it's better to spread them across apps, we can do that. But this is a large step on the path to enabling that. |
This all sounds good. Hold off on going too deep here, I think, until we've caught up with you. That point we'll have a better sense of where things stand? I'm +1 on mixins, but I think the dev team wants a chance to catch up before having opinions. |
There are a lot of
TestCase
that basically just setup some useful Factory objects and don't really need to be TestCases. This can cause problems when you want to use a TransactionTestCase and one of these TestCases and makes the class hierarchy more complicated than it needs to be. Converting them into mixin classes simplifies things quite a bit.