diff --git a/CHANGELOG.md b/CHANGELOG.md index c12bd0ed0e2c..1dc7d24d7fe2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ - `[jest-reporters]` Fix issue where console output not displayed for GHA reporter even with `silent: false` option ([#15864](https://github.com/jestjs/jest/pull/15864)) - `[jest-runtime]` Fix issue where user cannot utilize dynamic import despite specifying `--experimental-vm-modules` Node option ([#15842](https://github.com/jestjs/jest/pull/15842)) - `[jest-test-sequencer]` Fix issue where failed tests due to compilation errors not getting re-executed even with `--onlyFailures` CLI option ([#15851](https://github.com/jestjs/jest/pull/15851)) +- `[jest-reporters]` apply global coverage threshold to unmatched pattern files in addition to glob/path thresholds ([#15879](https://github.com/jestjs/jest/pull/15879)), fixes ([#5247](https://github.com/jestjs/jest/issues/5427)) ### Chore & Maintenance diff --git a/docs/Configuration.md b/docs/Configuration.md index 2e10910b5f44..e4b1771cc07a 100644 --- a/docs/Configuration.md +++ b/docs/Configuration.md @@ -392,7 +392,11 @@ For more information about the options object shape refer to `CoverageReporterWi Default: `undefined` -This will be used to configure minimum threshold enforcement for coverage results. Thresholds can be specified as `global`, as a [glob](https://github.com/isaacs/node-glob#glob-primer), and as a directory or file path. If thresholds aren't met, jest will fail. Thresholds specified as a positive number are taken to be the minimum percentage required. Thresholds specified as a negative number represent the maximum number of uncovered entities allowed. +This will be used to configure minimum threshold enforcement for coverage results. Thresholds can be specified as `global`, as a [glob](https://github.com/isaacs/node-glob#glob-primer), and as a directory or file path. If thresholds aren't met, jest will fail. + +- If a threshold is set to a **positive** number, it will be interpreted as the **minimum** percentage of coverage required. + +- If a threshold is set to a **negative** number, it will be treated as the **maximum** number of uncovered items allowed. For example, with the following configuration jest will fail if there is less than 80% branch, line, and function coverage, or if there are more than 10 uncovered statements: @@ -402,9 +406,13 @@ const {defineConfig} = require('jest'); module.exports = defineConfig({ coverageThreshold: { global: { + // Requires 80% branch coverage branches: 80, + // Requires 80% function coverage functions: 80, + // Requires 80% line coverage lines: 80, + // Require that no more than 10 statements are uncovered statements: -10, }, }, @@ -417,18 +425,48 @@ import {defineConfig} from 'jest'; export default defineConfig({ coverageThreshold: { global: { + // Requires 80% branch coverage branches: 80, + // Requires 80% function coverage functions: 80, + // Requires 80% line coverage lines: 80, + // Require that no more than 10 statements are uncovered statements: -10, }, }, }); ``` -If globs or paths are specified alongside `global`, coverage data for matching paths will be subtracted from overall coverage and thresholds will be applied independently. Thresholds for globs are applied to all files matching the glob. If the file specified by path is not found, an error is returned. +#### coverageThreshold.global.lines [number] + +Global threshold for lines. + +#### coverageThreshold.global.functions [number] + +Global threshold for functions. + +#### coverageThreshold.global.statements [number] + +Global threshold for statements. + +#### coverageThreshold.global.branches [number] + +Global threshold for branches. -For example, with the following configuration: +#### coverageThreshold[glob-pattern] \[object] + +Default: `undefined` + +Sets thresholds for files matching the [glob](https://github.com/isaacs/node-glob#glob-primer) pattern. This allows you to enforce a high global standard while also setting specific thresholds for critical files or directories. + +:::info + +When globs or paths are defined together with a global threshold, Jest applies each threshold independently — specific patterns use their own limits, while the global threshold applies only to files not matched by any pattern. + +If the file specified by path is not found, an error is returned. + +::: ```js tab title="jest.config.js" const {defineConfig} = require('jest'); @@ -488,10 +526,10 @@ export default defineConfig({ Jest will fail if: -- The `./src/components` directory has less than 40% branch or statement coverage. +- The `./src/components` directory has less than 40% branch/statement coverage. - One of the files matching the `./src/reducers/**/*.js` glob has less than 90% statement coverage. - The `./src/api/very-important-module.js` file has less than 100% coverage. -- Every remaining file combined has less than 50% coverage (`global`). +- All files that are not matched with `./src/components`, `./src/reducers/**/*.js`, `'./src/api/very-important-module.js'` has less than 50% coverage (`global`). ### `dependencyExtractor` \[string] diff --git a/e2e/__tests__/__snapshots__/coverageThreshold.test.ts.snap b/e2e/__tests__/__snapshots__/coverageThreshold.test.ts.snap index 2f3a61c0ff10..ecd238681297 100644 --- a/e2e/__tests__/__snapshots__/coverageThreshold.test.ts.snap +++ b/e2e/__tests__/__snapshots__/coverageThreshold.test.ts.snap @@ -1,28 +1,41 @@ // Jest Snapshot v1, https://jestjs.io/docs/snapshot-testing -exports[`excludes tests matched by path threshold groups from global group 1`] = ` -"PASS __tests__/banana.test.js - ✓ banana +exports[`exists with 1 if coverage threshold of matched paths is not met independently from global threshold 1`] = ` +"Test Suites: 1 passed, 1 total +Tests: 1 passed, 1 total +Snapshots: 0 total +Time: <> +Ran all test suites." +`; -Jest: "global" coverage threshold for lines (100%) not met: 0%" +exports[`exists with 1 if coverage threshold of matched paths is not met independently from global threshold: stdout 1`] = ` +"------------|---------|----------|---------|---------|------------------- +File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s +------------|---------|----------|---------|---------|------------------- +All files | 75 | 50 | 100 | 75 | + product.js | 75 | 50 | 100 | 75 | 6 +------------|---------|----------|---------|---------|-------------------" `; -exports[`excludes tests matched by path threshold groups from global group 2`] = ` -"Test Suites: 1 passed, 1 total -Tests: 1 passed, 1 total +exports[`exists with 1 if coverage threshold of the rest of non matched paths is not met 1`] = ` +"Test Suites: 5 passed, 5 total +Tests: 5 passed, 5 total Snapshots: 0 total Time: <> Ran all test suites." `; -exports[`excludes tests matched by path threshold groups from global group: stdout 1`] = ` -"-----------|---------|----------|---------|---------|------------------- -File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s ------------|---------|----------|---------|---------|------------------- -All files | 50 | 100 | 50 | 50 | - apple.js | 0 | 100 | 0 | 0 | 1-2 - banana.js | 100 | 100 | 100 | 100 | ------------|---------|----------|---------|---------|-------------------" +exports[`exists with 1 if coverage threshold of the rest of non matched paths is not met: stdout 1`] = ` +"------------|---------|----------|---------|---------|------------------- +File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s +------------|---------|----------|---------|---------|------------------- +All files | 91.66 | 50 | 100 | 91.66 | + product.js | 75 | 50 | 100 | 75 | 6 + sum-01.js | 100 | 100 | 100 | 100 | + sum-02.js | 100 | 100 | 100 | 100 | + sum-03.js | 100 | 100 | 100 | 100 | + sum-04.js | 100 | 100 | 100 | 100 | +------------|---------|----------|---------|---------|-------------------" `; exports[`exits with 0 if global threshold group is not found in coverage data: stdout 1`] = ` @@ -38,7 +51,7 @@ exports[`exits with 1 if coverage threshold is not met 1`] = ` "PASS __tests__/a-banana.js ✓ banana -Jest: "global" coverage threshold for lines (90%) not met: 50%" +Jest: Coverage for lines (50%) does not meet "global" threshold (90%)" `; exports[`exits with 1 if coverage threshold is not met 2`] = ` @@ -86,9 +99,9 @@ exports[`file is matched by all path and glob threshold groups 1`] = ` "PASS __tests__/banana.test.js ✓ banana -Jest: "./" coverage threshold for lines (100%) not met: 50% -Jest: "<>" coverage threshold for lines (100%) not met: 50% -Jest: "./banana.js" coverage threshold for lines (100%) not met: 50%" +Jest: Coverage for lines (50%) does not meet "./" threshold (100%) +Jest: Coverage for lines (50%) does not meet "<>" threshold (100%) +Jest: Coverage for lines (50%) does not meet "./banana.js" threshold (100%)" `; exports[`file is matched by all path and glob threshold groups 2`] = ` diff --git a/e2e/__tests__/coverageThreshold.test.ts b/e2e/__tests__/coverageThreshold.test.ts index 60b6850c602f..e575f6ef4185 100644 --- a/e2e/__tests__/coverageThreshold.test.ts +++ b/e2e/__tests__/coverageThreshold.test.ts @@ -129,38 +129,150 @@ test('exits with 0 if global threshold group is not found in coverage data', () expect(stdout).toMatchSnapshot('stdout'); }); -test('excludes tests matched by path threshold groups from global group', () => { +test('exists with 1 if coverage threshold of the rest of non matched paths is not met', () => { const pkgJson = { jest: { collectCoverage: true, collectCoverageFrom: ['**/*.js'], coverageThreshold: { - 'banana.js': { - lines: 100, + '**/*.js': { + branches: 50, + functions: 50, + lines: 50, + statements: 50, }, global: { - lines: 100, + branches: 80, + functions: 80, + lines: 80, + statements: 80, }, }, + testRegex: '.*\\.test\\.js$', }, }; - writeFiles(DIR, { - '__tests__/banana.test.js': ` - const banana = require('../banana.js'); - test('banana', () => expect(banana()).toBe(3)); + 'package.json': JSON.stringify(pkgJson, null, 2), + 'product.js': ` + function product(a, b) { + // let's simulate a 50% code coverage + if (a > 0) { + return a * b; + } else { + return a * b; + } + } + + module.exports = product; `, - 'apple.js': ` - module.exports = () => { - return 1 + 2; - }; + 'product.test.js': ` + test('multiplies 2 * 3 to equal 6', () => { + const sum = require('./product'); + expect(sum(2, 3)).toBe(6); + }); `, - 'banana.js': ` - module.exports = () => { - return 1 + 2; - }; + 'sum-01.js': ` + function sum(a, b) { + return a + b; + } + + module.exports = sum; + `, + 'sum-01.test.js': ` + test('adds 1 + 2 to equal 3', () => { + const sum = require('./sum-01'); + expect(sum(1, 2)).toBe(3); + }); + `, + 'sum-02.js': ` + function sum(a, b) { + return a + b; + } + + module.exports = sum; + `, + 'sum-02.test.js': ` + test('adds 1 + 2 to equal 3', () => { + const sum = require('./sum-02'); + expect(sum(1, 2)).toBe(3); + }); + `, + 'sum-03.js': ` + function sum(a, b) { + return a + b; + } + + module.exports = sum; + `, + 'sum-03.test.js': ` + test('adds 1 + 2 to equal 3', () => { + const sum = require('./sum-03'); + expect(sum(1, 2)).toBe(3); + }); + `, + 'sum-04.js': ` + function sum(a, b) { + return a + b; + } + + module.exports = sum; `, + 'sum-04.test.js': ` + test('adds 1 + 2 to equal 3', () => { + const sum = require('./sum-04'); + expect(sum(1, 2)).toBe(3); + }); + `, + }); + + const {stdout, stderr, exitCode} = runJest( + DIR, + ['--coverage', '--ci=false'], + {stripAnsi: true}, + ); + const {summary} = extractSummary(stderr); + + expect(exitCode).toBe(1); + expect(summary).toMatchSnapshot(); + expect(stdout).toMatchSnapshot('stdout'); +}); + +test('exists with 1 if coverage threshold of matched paths is not met independently from global threshold', () => { + const pkgJson = { + jest: { + collectCoverage: true, + collectCoverageFrom: ['**/*.js'], + coverageThreshold: { + global: { + lines: 70, + }, + 'product.js': { + lines: 80, + }, + }, + testRegex: '.*\\.test\\.js$', + }, + }; + writeFiles(DIR, { 'package.json': JSON.stringify(pkgJson, null, 2), + 'product.js': ` + function product(a, b) { + // let's simulate a 50% code coverage + if (a > 0) { + return a * b; + } else { + return a * b; + } + } + + module.exports = product; + `, + 'product.test.js': ` + test('multiplies 2 * 3 to equal 6', () => { + const sum = require('./product'); + expect(sum(2, 3)).toBe(6); + }); + `, }); const {stdout, stderr, exitCode} = runJest( @@ -168,10 +280,9 @@ test('excludes tests matched by path threshold groups from global group', () => ['--coverage', '--ci=false'], {stripAnsi: true}, ); - const {rest, summary} = extractSummary(stderr); + const {summary} = extractSummary(stderr); expect(exitCode).toBe(1); - expect(rest).toMatchSnapshot(); expect(summary).toMatchSnapshot(); expect(stdout).toMatchSnapshot('stdout'); }); diff --git a/packages/jest-reporters/src/CoverageReporter.ts b/packages/jest-reporters/src/CoverageReporter.ts index 4a26d17b33a1..3562773784c1 100644 --- a/packages/jest-reporters/src/CoverageReporter.ts +++ b/packages/jest-reporters/src/CoverageReporter.ts @@ -249,7 +249,7 @@ export default class CoverageReporter extends BaseReporter { } } else if (actual < threshold) { errors.push( - `Jest: "${name}" coverage threshold for ${key} (${threshold}%) not met: ${actual}%`, + `Jest: Coverage for ${key} (${actual}%) does not meet "${name}" threshold (${threshold}%)`, ); } } @@ -273,6 +273,11 @@ export default class CoverageReporter extends BaseReporter { const pathOrGlobMatches = thresholdGroups.reduce< Array<[string, string]> >((agg, thresholdGroup) => { + // Skip 'global' here as it will be handled separately for all files + if (thresholdGroup === THRESHOLD_GROUP_TYPES.GLOBAL) { + return agg; + } + // Preserve trailing slash, but not required if root dir // See https://github.com/jestjs/jest/issues/12703 const resolvedThresholdGroup = path.resolve(thresholdGroup); @@ -316,23 +321,20 @@ export default class CoverageReporter extends BaseReporter { if (pathOrGlobMatches.length > 0) { files.push(...pathOrGlobMatches); - return files; - } - - // Neither a glob or a path? Toss it in global if there's a global threshold: - if (thresholdGroups.includes(THRESHOLD_GROUP_TYPES.GLOBAL)) { - groupTypeByThresholdGroup[THRESHOLD_GROUP_TYPES.GLOBAL] = - THRESHOLD_GROUP_TYPES.GLOBAL; - files.push([file, THRESHOLD_GROUP_TYPES.GLOBAL]); - return files; } - // A covered file that doesn't have a threshold: + // A covered file that doesn't have a threshold (or only has global): files.push([file, undefined]); return files; }, []); + // Mark global threshold group if it exists + if (thresholdGroups.includes(THRESHOLD_GROUP_TYPES.GLOBAL)) { + groupTypeByThresholdGroup[THRESHOLD_GROUP_TYPES.GLOBAL] = + THRESHOLD_GROUP_TYPES.GLOBAL; + } + const getFilesInThresholdGroup = (thresholdGroup: string) => coveredFilesSortedIntoThresholdGroup .filter(fileAndGroup => fileAndGroup[1] === thresholdGroup) @@ -363,9 +365,8 @@ export default class CoverageReporter extends BaseReporter { for (const thresholdGroup of thresholdGroups) { switch (groupTypeByThresholdGroup[thresholdGroup]) { case THRESHOLD_GROUP_TYPES.GLOBAL: { - const coverage = combineCoverage( - getFilesInThresholdGroup(THRESHOLD_GROUP_TYPES.GLOBAL), - ); + // Global threshold applies to ALL covered files + const coverage = combineCoverage(coveredFiles); if (coverage) { errors = [ ...errors, diff --git a/packages/jest-reporters/src/__tests__/CoverageReporter.test.js b/packages/jest-reporters/src/__tests__/CoverageReporter.test.js index 59965635bf0b..dd5838c0e59c 100644 --- a/packages/jest-reporters/src/__tests__/CoverageReporter.test.js +++ b/packages/jest-reporters/src/__tests__/CoverageReporter.test.js @@ -135,7 +135,7 @@ describe('onRunComplete', () => { })); }); - test('getLastError() returns an error when threshold is not met for global', () => { + test('getLastError() returns an error when threshold is not met for global', async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -150,14 +150,13 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); }); - test('getLastError() returns an error when threshold is not met for file', () => { + test('getLastError() returns an error when threshold is not met for file', async () => { const covThreshold = {}; const paths = [ 'global', @@ -179,14 +178,13 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError().message.split('\n')).toHaveLength(5); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError().message.split('\n')).toHaveLength(5); }); - test('getLastError() returns `undefined` when threshold is met', () => { + test('getLastError() returns `undefined` when threshold is met', async () => { const covThreshold = {}; const paths = [ 'global', @@ -208,14 +206,13 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError()).toBeUndefined(); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError()).toBeUndefined(); }); - test('getLastError() returns an error when threshold is not met for non-covered file', () => { + test('getLastError() returns an error when threshold is not met for non-covered file', async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -230,14 +227,13 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); }); - test('getLastError() returns an error when threshold is not met for directory', () => { + test('getLastError() returns an error when threshold is not met for directory', async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -252,14 +248,13 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); }); - test('getLastError() returns `undefined` when threshold is met for directory', () => { + test('getLastError() returns `undefined` when threshold is met for directory', async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -274,14 +269,13 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError()).toBeUndefined(); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError()).toBeUndefined(); }); - test('getLastError() returns an error when there is no coverage data for a threshold', () => { + test('getLastError() returns an error when there is no coverage data for a threshold', async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -296,16 +290,15 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); }); - test(`getLastError() returns 'undefined' when global threshold group - is empty because PATH and GLOB threshold groups have matched all the - files in the coverage data.`, () => { + test(`getLastError() returns an error when global threshold is not +met by other non matched PATH and GLOB files, even when PATH and GLOB threshold +groups have matched all the files in the coverage data.`, async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -326,15 +319,16 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError()).toBeUndefined(); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + // With new behavior, global threshold applies to ALL files + // Total coverage is ~50.5%, which fails the 100% threshold + expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); }); test(`getLastError() returns 'undefined' when file and directory path - threshold groups overlap`, () => { +threshold groups overlap`, async () => { const covThreshold = {}; for (const path of [ './path-test-files/', @@ -360,16 +354,15 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError()).toBeUndefined(); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError()).toBeUndefined(); }); - test(`that if globs or paths are specified alongside global, coverage - data for matching paths will be subtracted from overall coverage - and thresholds will be applied independently`, () => { + test(`that if globs or paths are specified alongside global, global +threshold applies to all files while path/glob thresholds +are applied independently`, async () => { const testReporter = new CoverageReporter( { collectCoverage: true, @@ -390,13 +383,11 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - // 100% coverage file is removed from overall coverage so - // coverage drops to < 50% - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError().message.split('\n')).toHaveLength(1); - }); + // With new behavior, global threshold checks ALL files + // Total coverage is ~50.5%, which passes the 50% threshold + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(testReporter.getLastError()).toBeUndefined(); }); test('that files are matched by all matching threshold groups', () => { @@ -423,29 +414,26 @@ describe('onRunComplete', () => { }, ); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(testReporter.getLastError()).toBeUndefined(); - }); + return testReporter.onRunComplete(new Set(), mockAggResults).then(() => { + expect(testReporter.getLastError()).toBeUndefined(); + }); }); - test('that it passes custom options when creating reporters', () => { + test('that it passes custom options when creating reporters', async () => { const testReporter = new CoverageReporter({ coverageReporters: ['json', ['lcov', {maxCols: 10, projectRoot: './'}]], }); testReporter.log = jest.fn(); - return testReporter - .onRunComplete(new Set(), {}, mockAggResults) - .then(() => { - expect(istanbulReports.create).toHaveBeenCalledWith('json', { - maxCols: process.stdout.columns || Number.POSITIVE_INFINITY, - }); - expect(istanbulReports.create).toHaveBeenCalledWith('lcov', { - maxCols: 10, - projectRoot: './', - }); - expect(testReporter.getLastError()).toBeUndefined(); - }); + + await testReporter.onRunComplete(new Set(), mockAggResults); + + expect(istanbulReports.create).toHaveBeenCalledWith('json', { + maxCols: process.stdout.columns || Number.POSITIVE_INFINITY, + }); + expect(istanbulReports.create).toHaveBeenCalledWith('lcov', { + maxCols: 10, + projectRoot: './', + }); + expect(testReporter.getLastError()).toBeUndefined(); }); });