Skip to content

Introduce have_reported_error matcher #2849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
5e6efcc
first pass at have_reported_error matcher
skatkov Jun 4, 2025
5dd2c5c
Adding a feature
skatkov Jun 7, 2025
ce1a162
simplify matcher/readmer.md
skatkov Jun 14, 2025
550c25e
Remove mocks out of tests
skatkov Jun 14, 2025
e074991
test failure messages better
skatkov Jun 14, 2025
1fe7e91
rewrite specs
skatkov Jun 20, 2025
6b41d2b
should require a block matcher
skatkov Jun 22, 2025
3859b04
Create ErrorEvent struct to store errors
skatkov Jun 22, 2025
8352668
simplify matcher
skatkov Jun 22, 2025
030c27a
disable matcher chaining
skatkov Jun 22, 2025
2b9bd02
improved argument error message in case block is not passed to matcher
skatkov Jun 22, 2025
6135d5d
further clean-up
skatkov Jun 22, 2025
8335664
Apply suggestions from code review
skatkov Jun 25, 2025
a14f5c7
clean-up features
skatkov Jun 27, 2025
af58d83
remove a require from test
skatkov Jul 1, 2025
7259a25
rename .with to .with_context
skatkov Jul 1, 2025
1b6deb6
have_reported_error to accept class name and a message
skatkov Jul 1, 2025
c77fca5
switch to error_report matcher type of api
skatkov Jul 1, 2025
f5e427a
refactor
skatkov Jul 1, 2025
7b3abc5
Using UndefinedValue as a default attribute
skatkov Jul 1, 2025
2488a71
Look through all errors, don't just go with last error
skatkov Jul 1, 2025
c1816e2
warn about nil being passed
skatkov Jul 9, 2025
e1ab316
Use ErrorCollector from Rails instead of self-written version'
Aug 4, 2025
8674529
Use defined UNDEFINED value instead of defining our own
Aug 4, 2025
8b89342
Update features/matchers/README.md
pirj Aug 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions features/matchers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,28 @@ expect(response).to render_template(template_name)
# and it is not persisted
expect(assigns(:widget)).to be_a_new(Widget)
```

### error reporting

```ruby
# passes when any error is reported
expect { Rails.error.report(StandardError.new) }.to have_reported_error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this file is supposed to contain just the outline of the matchers defined here (and at a glance it’s rather incomplete). What do you think of keeping just one example of usage of this matcher? The rest will be taken into the same documentation from the feature file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very verbose for us... These lines take up half this file when our primary documentation is the feature file itself...


# passes when specific error class is reported
expect { Rails.error.report(MyError.new) }.to have_reported_error(MyError)

# passes when specific error class with exact message is reported
expect { Rails.error.report(MyError.new("message")) }.to have_reported_error(MyError, "message")

# passes when specific error class with message matching pattern is reported
expect { Rails.error.report(MyError.new("test message")) }.to have_reported_error(MyError, /test/)

# passes when any error with exact message is reported
expect { Rails.error.report(StandardError.new("exact message")) }.to have_reported_error("exact message")

# passes when any error with message matching pattern is reported
expect { Rails.error.report(StandardError.new("test message")) }.to have_reported_error(/test/)

# passes when error is reported with specific context attributes
expect { Rails.error.report(StandardError.new, context: { user_id: 123 }) }.to have_reported_error.with_context(user_id: 123)
```
199 changes: 199 additions & 0 deletions features/matchers/have_reported_error_matcher.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
Feature: `have_reported_error` matcher

The `have_reported_error` matcher is used to check if an error was reported
to Rails error reporting system (`Rails.error`). It can match against error
classes, messages, and attributes.

The matcher supports several matching strategies:
* Any error reported
* A specific error class
* A specific error class with message
* Error message patterns using regular expressions
* Message-only matching (any class)
* Error attributes using `.with_context()`

The matcher is available in all spec types where Rails error reporting is used.

Background:
Given a file named "app/models/user.rb" with:
"""ruby
class User < ApplicationRecord
class ValidationError < StandardError; end
def self.process_data
Rails.error.report(StandardError.new("Processing failed"))
end

def self.process_with_context
Rails.error.report(ArgumentError.new("Invalid input"), context: { context: "user_processing", severity: :error })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nested context here, too. Rename inner to topic: or :section?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The severity: here is inside the context:, is this correct?

end

def self.process_custom_error
Rails.error.report(ValidationError.new("Email is invalid"))
end
end
"""

Scenario: Checking for any error being reported
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports errors" do
expect {
User.process_data
}.to have_reported_error
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Checking for message-only matching
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports error with exact message (any class)" do
expect {
User.process_data
}.to have_reported_error("Processing failed")
end

it "reports error with message pattern (any class)" do
expect {
User.process_custom_error
}.to have_reported_error(/Email/)
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Checking for a specific error class
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports a StandardError" do
expect {
User.process_data
}.to have_reported_error(StandardError)
end

it "reports an ArgumentError" do
expect {
User.process_with_context
}.to have_reported_error(ArgumentError)
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Checking for specific error class with message
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports error with specific message" do
expect {
User.process_data
}.to have_reported_error(StandardError, "Processing failed")
end

it "reports ArgumentError with specific message" do
expect {
User.process_with_context
}.to have_reported_error(ArgumentError, "Invalid input")
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Checking error messages using regular expressions
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports errors with a message matching a pattern (any class)" do
expect {
User.process_data
}.to have_reported_error(/Processing/)
end

it "reports specific class with message matching a pattern" do
expect {
User.process_data
}.to have_reported_error(StandardError, /Processing/)
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Constraining error matches to their attributes using `with_context`
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports error with specific context" do
expect {
User.process_with_context
}.to have_reported_error.with_context(context: "user_processing")
end

it "reports error with multiple attributes" do
expect {
User.process_with_context
}.to have_reported_error(ArgumentError).with_context(context: "user_processing", severity: :error)
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Checking custom error classes
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "reports a ValidationError" do
expect {
User.process_custom_error
}.to have_reported_error(User::ValidationError)
end

it "reports ValidationError with specific message" do
expect {
User.process_custom_error
}.to have_reported_error(User::ValidationError, "Email is invalid")
end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass

Scenario: Using negation - expecting no errors
Given a file named "spec/models/user_spec.rb" with:
"""ruby
require "rails_helper"

RSpec.describe User do
it "does not report any errors for safe operations" do
expect {
# Safe operation that doesn't report errors
"safe code"
}.not_to have_reported_error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the negated form should warn if qualifiers were passed to avoid mistakenly letting unmatched reports through:

          expect {
            User.process_custom_error
          }.not_to have_reported_error("Email is TYPO")

Meaning that if the matcher is used in the negated form, no class, message, or attributes qualifiers are acceptable.

See https://github.com/rspec/rspec/blob/95fbf88255de04e4ba8288172292bda494c4f187/rspec-expectations/lib/rspec/matchers/built_in/raise_error.rb#L162

end
end
"""
When I run `rspec spec/models/user_spec.rb`
Then the examples should all pass
1 change: 1 addition & 0 deletions lib/rspec/rails/matchers.rb
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ module Matchers
require 'rspec/rails/matchers/relation_match_array'
require 'rspec/rails/matchers/be_valid'
require 'rspec/rails/matchers/have_http_status'
require 'rspec/rails/matchers/have_reported_error'
require 'rspec/rails/matchers/send_email'

if RSpec::Rails::FeatureCheck.has_active_job?
Expand Down
Loading
Loading