Skip to content

testing.md

Latest commit

 

History

History
133 lines (87 loc) · 4.14 KB

testing.md

File metadata and controls

133 lines (87 loc) · 4.14 KB

Tests and linting

The Rails application uses RSpec and RuboCop for testing and linting, as well as Brakeman for security scanning and Slim-Lint to lint Slim templates.

# Run tests and linting
bundle exec rake

# Run tests only
bundle exec rspec

# Run linters only
bundle exec rails lint

The frontend Javascript code uses Jest and ESLint for testing and linting (using Airbnb rules), as well as Stylelint for SASS linting (with the default ruleset):

# Run tests and linting
yarn test

# Run tests only
yarn run js:test

# Generate a coverage report
yarn run js:test:coverage

# Run JS linter only
yarn run js:lint

# Run SASS linter only
yarn run sass:lint

RSpec parallel testing

The service include parallel_tests gem, that allows to split the RSpec test suite run accross multiple CPU cores.

Each group will run against a separate database. To set it up:

RAILS_ENV=test rails parallel:create

To run the test suite in parallel:

RAILS_ENV=test rake parallel:spec

or

RAILS_ENV=test rails parallel:spec

or using Spring binstub:

parallel_rspec

RSpec test coverage reports

We use Simplecov and Undercover gems for measuring and enforcing our test coverage.

The configuration and minimum coverage requirements are set on the .simplecov file

How to generate coverage reports locally?

Set COVERAGE=1 in your environment or as a prefix the full rspec run.

EG: COVERAGE=1 rspec

After running the full test suite, the coverage report should be available at the project's /coverage/index.html path.

Finding code coverage on the PR runs

The code coverage is generated as an artifact on the projects Github PR check test -> Run RSpec Tests in parallel -> Upload coverage report

When the step is expanded, there is a link to download it as: Artifact download URL: https://github.com/DFE-Digital/teaching-vacancies/actions/runs/111111111/artifacts/22222222

The link will download a ZIP file containing the coverage report for the PR tests run.

Undercover alerts

Once having a coverage report generated, undercover can be used to alert us about code that we added or changed without test coverage.

This happens automatically in the PRs test -> Run RSpec Tests in parallel workflow. But can also be run locally invoking the undercover command.

Visual regression testing

The visual layout and appearance of defined scenarios (pages and common content) can be tested using BackstopJS

This works by testing snapshot images in a branch or environment against accepted baseline/reference snapshot images. Tests are run across 3 viewports - mobile, tablet and desktop. Also if abtest variants are added to config/ab_test.yml then the corresponding page will be tested with each variant.

This is currently a tool for developers to test changes locally against the QA environment. Running the test suite will create and load a UI in the browser to report on and for you to compare visual changes.

Setup

Install the backstopjs library as a global dependency

npm install -g backstopjs@6.1.0

Add the following variables to your local .env file:

VISUAL_TEST_JOBSEEKER_USERNAME=xxx
VISUAL_TEST_JOBSEEKER_PASSWORD=xxx
VISUAL_TEST_PUBLISHER_USERNAME=xxx
VISUAL_TEST_PUBLISHER_PASSWORD=xxx

Usage

If you have never run tests before, ensure you have created an backstop/lib/.tmp folder for authentication tokens. (TODO could do with automating)

# Create reference snapshot images
yarn run visual:test:init

# Run test suite to compare your changes to reference snapshots
yarn run visual:test:run

# approve changes
yarn run visual:test:approve

This will approve changes to create new baseline snapshots and clear your file system of snapshots created by previous test suite runs.