feat: Consider bulk suppressions when running Lint via the Node.js API

[sushichan044]

Summary

This RFC proposes integrating bulk suppressions support into the Node.js API via the ESLint and LegacyESLint classes, specifically focusing on considering existing bulk suppressions when linting files or text through the API. This change ensures that suppression files (eslint-suppressions.json) created via CLI commands are automatically respected when using the programmatic API, maintaining consistency between CLI and API behavior.

The scope is limited to applying existing suppressions during linting and does not include suppression file manipulation features (such as --suppress-all, --suppress-rule, or --prune-suppressions), which remain CLI-exclusive functionalities.

Related Issues

• Change Request: Support bulk suppression in Node.js API eslint#19603

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: sushichan044 / name: Kentaro Suzuki (40473fd, f63e63f, bc82f31, fa25623, 8b6e973, 00ba3e9, c5b4134)

[sushichan044]

Specific code examples for LegacyESLint are WIP, but will be added soon.

[mdjermanovic]

Thanks for the RFC! I left some initial questions about the design.

[mdjermanovic]

Suppose someone has a task to fix previously suppressed violations. How will they do this, since the violations no longer appear in their IDE?

[sushichan044]

The motivation for using bulk suppression may be “I want to ignore existing errors and give priority to activating new errors”.
With that assumption, existing errors that should be ignored should not be shown in the IDE either.

To work on reducing the number of ignored errors, we can start by check the suppressed errors in eslint-suppressions.json.

[nzakas]

@mdjermanovic I think this is the same use case as when running on the CLI. Because the CLI automatically picks up the eslint-suppressions.json file, it's necessary to edit, move, or delete the file if your task is to clean up suppressed violations.

[wagenet]

The fact that suppressed lints show up in the IDE is a feature not a bug for me. It encourages developers to fix the issues when they're working in those files and not copy the bad patterns. However, I would like to be able to see the suppressed violations in a different color.

[sushichan044]

@wagenet
For example, do you mean displaying Diagnostics at the Suggestion level for suppressed warnings?
This would probably be possible by returning information about suppressed violations from the API and writing corresponding processing on the IDE extension side.

[wagenet]

I'm not sure what IDE integrations generally allow, but for me the ideal situation is to have the IDE integrations show both suppressed and not-suppressed violations in the IDE, but to distinguish between the two, most likely by color. I'm somewhat agnostic as to how this is achieved.

[sushichan044]

@wagenet
If that's the case, then we're probably thinking the same thing. Sorry for the unclear explanation.

[joshhunt]

👍 +1 to everything @wagenet said - We've been discussing switching to eslint bulk suppressions, and we like that errors show in the IDE. But perhaps a different appearance in IDE to distinguish them would be useful.

[nzakas]

For some context: the ESLint team does not maintain any ESLint IDE integrations. These are maintained by their respective IDE teams. The only thing we can control is whether or not the suppressions information is available to the IDEs and it's up to the IDEs to determine how to use that information.

[nzakas]

@mdjermanovic what do you think of #133 (comment)?

[nzakas]

@sushichan044 please take a moment to sign the CLA (see first comment).

[nzakas]

Overall this looks like a good plan to me. Nice work.

[nzakas]

@mdjermanovic I think this is the same use case as when running on the CLI. Because the CLI automatically picks up the eslint-suppressions.json file, it's necessary to edit, move, or delete the file if your task is to clean up suppressed violations.

[softius]

I think we will need to maintain the existing order of processing. In this case, applySuppressions will be invoked before the call to outputFixes, suppress and prune. This will lead to different behavior from the existing implementation. For example, if the CLI is invoked with --prune-suppressions you will still get errors about unmatched suppressions, since the pruning will happen after.

[sushichan044]

I am currently organizing the issues related to the order of processing. Please give me a little time.

[sushichan044]

@softius cc @nzakas
That's a fair point. However, if we were to make the API's processing order match the CLI's, it would likely require breaking changes to the API.
Therefore, I think it's more realistic to adjust the CLI's processing order to align with the API implementation in this RFC.

[nzakas]

@sushichan044 I don't think changing the CLI is the right choice if it leads to a broken experience as @softius described.

[sushichan044]

@nzakas @softius
After considering it, I realized we can improve IDE support without making changes that would break the CLI's behavior, I think we can implement it to match the current CLI operation.

[nzakas]

That sounds like a good plan to me. 👍

[softius]

Maybe it would be better to pass the whole suppressions config and move the whole logic into the classes, similar to the cache? 🤔

[nzakas]

Can you expand on that suggestion? I'm not clear on what it is you're imagining.

[softius]

This is a suggestion in regards to my concern described here at #133 (comment)

Since we are discussing to move the suppression check from CLI to ESLint classes, we would need to bring over the entire code block to maintain the correct order of execution. In that case we would need to pass over all the suppressions args and let the ESLint classes handle the logic including updating the files (similar to cache).

Having said that, I realised that even with this approach we will be doing the suppressions logic before the automated fixes, which means that once again we are changing the execution order.

[nzakas]

I think we need to take another look to see if we can effectively keep the current behavior while allowing API consumers to filter the results through suppressions.

To that end, it seems like the only reasonable approach would be to add an option to the ESLint constructor to indicate whether it should or shouldn't apply suppressions to results when lintText() or lintFiles() is called? That way, the CLI can tell ESLint not to apply suppressions and maintain its current behavior, and any API consumers who want to apply suppressions can do so. (Question remains: what is the default? Should ESLint apply suppressions automatically or not?)

[sushichan044]

@nzakas
Thank you!
I am very busy right now, so my reply will be delayed until around next Monday.

[nzakas]

@sushichan044 no problem, thanks for letting us know.

[nzakas]

@spence-s that's out of scope for this RFC. Please open a separate issue to discuss.

[nzakas]

@sushichan044 it looks like the remaining work here it to ensure the functionality works the same as with the CLI. Are you still working on that point?

[wagenet]

I would like to have the ability to see all violations with a suppressed property on each of them so I can distinguish instead of just having the suppressions skipped.

[wagenet]

The fact that suppressions are only applied to the CLI is actually a really big feature for me. I want to see the violations in the IDE, but not have them cause my CI to fail. The only change I'd like is the ability for my IDE to show suppressed violations in a different color. Hiding suppressed violations in the IDE would be a regression for me.

[rtablada]

I would definitely agree with @wagenet on this. Having suppressions show in IDE is a big improvement IMO over disables. This allows lints to be started, suppressed and team members to know this is a work item.

Having IDEs and other tools being able to distinguish between suppressed and not though is also something important.
For someone opening a file only to know by running CLI what is new work that needs fixing vs suppressed that is nice to tidy up is a big win for this workflow.

[sushichan044]

About displaying diagnostic information for suppressed violations:
#133 (comment)

[nzakas]

@sushichan044 just checking in -- are you still planning on working on this?

[sushichan044]

@nzakas

Sorry, my academic research has kept me very busy and my response is delayed.

What I’m currently thinking is to have SuppressionService.applySuppressions return not only the LintResult[] after bulk suppression is applied, but also a separate set of suppressed LintResult[]. Then, the lintText / lintFiles APIs would return that information as well.
This approach would simplify the implementation, but it might place more burden on the ecosystem using the API.

Anyway, I’ll make time to reconsider this by the coming weekend.

[sushichan044]

To match CLI behavior, we’d need to extend LintResult for bulk-suppression related data and add a new API like ESLint.outputFixes.

[sushichan044]

As for changes that impose less burden on the ecosystem, I'm also considering the following ideas.
These can be implemented sufficiently with the existing logic, but if API users make mistakes in their implementation, it may result in confusing behavior.

In lintText() and lintFiles():
• When fix is false: apply bulk suppression directly to LintResult[].messages.
• When fix is true: apply bulk suppression to LintResult[].messages entries that cannot be speculatively auto-fixed, on the assumption that outputFixes() will be called in later processing.

note:
In this case, considering the use cases, it is necessary to add some property to LintResult that stores the LintMessage[] removed by bulk suppression.
Alternatively, it may be acceptable to use the existing LintResult.suppressedMessages, as long as we make it possible to distinguish between eslint-disable and bulk suppression.

[nzakas]

Thanks for the update, and no worries -- we don't have any time constraints here. :)

Alternatively, it may be acceptable to use the existing LintResult.suppressedMessages, as long as we make it possible to distinguish between eslint-disable and bulk suppression.

We don't currently distinguish between the two types of suppressions, so I'm not sure I understand why the distinction is necessary here. Can you explain your thinking a bit more?

We do want to be careful about maintaining existing public interfaces like lintText() and lintFiles(), as these are used quite heavily in the ecosystem.