-
-
Notifications
You must be signed in to change notification settings - Fork 71
Description
Describe the bug
I made a new pull request #1569 to try to narrow down what problem I had introduced that was causing CI system test failures in #1548. However, before I replicated any of the changes I had made, the fresh new pull request already showed a difference between CI and local tests. I've included screenshots below that show that basic_user sees 7 posts in the Main category when system tests are run locally, but sees 0 posts when run on the CI.
To Reproduce
Steps to reproduce the behavior:
I added the following test to a new branch from develop
:
require 'application_system_test_case'
class CIFailuresTest < ApplicationSystemTestCase
test 'temporary test to analyse CI failures what does basic user see in main' do
category = categories(:main)
log_in :basic_user
visit category_path(category)
assert_text 'Nonexistent to force screenshot'
end
end
Making a pull request then triggered the CI tests to run, and I compared the forced screenshot from the CI with the forced screenshot from running the system tests locally with rails test:system
.
Expected behavior
The test on CI and locally should start with the same state, being based on the same fixtures. CI and local tests cannot be expected to have consistent output if they do not start with consistent state.
Screenshots
Locally:
On CI:
Metadata
Metadata
Assignees
Labels
Type
Projects
Status