Skip to content

Map SNS message attributes to SQS listener headers - #1699

Open
azunox wants to merge 4 commits into
awspring:mainfrom
azunox:fix/sns-attributes-in-sqs-headers
Open

azunox wants to merge 4 commits into
awspring:mainfrom
azunox:fix/sns-attributes-in-sqs-headers

Conversation

@azunox

@azunox azunox commented Sep 4, 2026

Copy link
Copy Markdown

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

📜 Description

Maps SNS MessageAttributes embedded in a non-raw SNS notification envelope to Spring Message headers when the notification is received through SQS.

The change:

  • Detects SNS notification envelopes before extracting attributes.
  • Maps String, Number, and Binary SNS attribute values to their corresponding header value types.
  • Preserves existing SQS message attributes when an SNS attribute has the same name.
  • Leaves regular SQS messages and non-SNS JSON payloads unchanged.
  • Avoids parsing message bodies that do not contain MessageAttributes.

💡 Motivation and Context

When raw message delivery is disabled for an SNS-to-SQS subscription, SNS message attributes are stored inside the SNS JSON envelope rather than as native SQS message attributes.

SqsHeaderMapper previously mapped only native SQS message attributes. Consequently, attributes published through SnsTemplate were unavailable from Message#getHeaders() or an @Headers argument in an @SqsListener.

This change promotes the nested SNS attributes to Spring Message headers while retaining native SQS attributes as the authoritative value in case of a name collision.

Closes #1111

💚 How did you test it?

Added focused SqsHeaderMapperTests covering:

  • String, Number, and Binary SNS message attribute mapping.
  • Precedence of native SQS message attributes over nested SNS attributes.
  • Rejection of MessageAttributes found in non-SNS JSON payloads.

Executed:

./mvnw -pl spring-cloud-aws-sqs -am -Dtest=SqsHeaderMapperTests -Dsurefire.failIfNoSpecifiedTests=false test

Result: 33 tests run, 0 failures, 0 errors, 0 skipped. The full repository and LocalStack integration test suites were not executed.

📝 Checklist

  • I reviewed submitted code
  • I added tests to verify changes
  • I updated reference documentation to reflect the change
  • All tests passing
  • No breaking changes

🔮 Next steps

Run the complete CI and integration test suites through the repository pull request workflow.

@github-actions github-actions Bot added the component: sqs SQS integration related issue label Sep 4, 2026

@MatejNedic MatejNedic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @azunox ,

Thanks on PR! I see few things I personally would do differently.

@tomazfernandes what do you think?

private static final TypeReference<Map<String, SnsNotification.MessageAttribute>> SNS_MESSAGE_ATTRIBUTES_TYPE = new TypeReference<>() {
};

private final JsonMapper jsonMapper = new JsonMapper();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JsonMapper should be the same one that is passed in autoconfiguration.

accessor.copyHeadersIfAbsent(getMessageAttributesAsHeaders(source));
accessor.copyHeadersIfAbsent(createDefaultHeaders(source));
accessor.copyHeadersIfAbsent(createAdditionalHeaders(source));
accessor.copyHeadersIfAbsent(getSnsMessageAttributesAsHeaders(source));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs on every SQS message, whether it's an SNS notification or not. SqsHeaderMapper is the default mapper for all SQS traffic, so this adds an SNS-envelope check (and a JSON parse on match) to the hot path for everyone, even deployments that never receive SNS messages.

Custom SnsAwareSqsHeaderMapper should be implemented and wired via existing hooks such as AbstractMessagingMessageConverter.setHeaderMapper and AbstractMessagingMessageConverter.configureHeaderMapper.

Container options can be used then to configure this custom mapper for Queues which will have messages sent via SnsTemplate. This makes it cheaper for performance.

@azunox

azunox commented Sep 4, 2026

Copy link
Copy Markdown
Author

Hello! Thanks for the review, @MatejNedic

I agree that SNS-specific parsing should not run for all SQS messages.

I propose moving the logic into an opt-in SnsAwareSqsHeaderMapper that receives the application's configured JsonMapper, with focused tests and a configuration example for queues receiving SNS notifications.

does this approach look right to you before I update the PR?

@tomazfernandes

Copy link
Copy Markdown
Contributor

Hey @azunox, thanks for the PR, and thanks @MatejNedic for the suggestions.

I'll take a closer look shortly.

@tomazfernandes

Copy link
Copy Markdown
Contributor

Hey @azunox and @MatejNedic, agreed on the new Mapper opt-in direction.

This can be a Jackson 3-only feature since Jackson 2 support is deprecated, let's have that explicit in the docs.

Address review comment 3937155533 by moving SNS envelope parsing out of the default SQS mapper and covering explicit converter configuration.

Signed-off-by: 김우성 <azunox9@gmail.com>
Address review comment 3937126374 with constructor injection, parser configuration coverage, and a dedicated listener factory example.

Signed-off-by: 김우성 <azunox9@gmail.com>
Address Tomaz Fernandes comment 5751349643 by documenting the Jackson 3-only feature and deprecated Jackson 2 support.

Signed-off-by: 김우성 <azunox9@gmail.com>
@github-actions github-actions Bot added the type: documentation Documentation or Samples related issue label Sep 20, 2026
@azunox

azunox commented Sep 20, 2026 •

Copy link
Copy Markdown
Author

Thanks for confirming the direction, @tomazfernandes, and thanks for the suggestions, @MatejNedic.
I've updated the PR accordingly:

  • Moved SNS envelope parsing into an opt-in SnsAwareSqsHeaderMapper. The default SqsHeaderMapper no longer checks or parses message bodies for SNS attributes.
  • Made the new mapper accept the application's configured Jackson 3 JsonMapper through its constructor, rather than creating its own instance.
  • Added a dedicated listener factory example using setHeaderMapper and container options, passing the same JsonMapper to both the message converter and the header mapper. Only listeners selecting that factory opt in to SNS attribute mapping.
  • Explicitly documented that this is a Jackson 3-only feature and that no Jackson 2 implementation is provided, as Jackson 2 support is deprecated.

The changes are split into three commits: b60fc7e9 (opt-in mapper), c214eca0 (configured JsonMapper and configuration example), and 82144bf1 (Jackson 3 documentation).

Focused tests cover attribute type conversion, SQS header precedence, preservation of additional headers and message IDs, non-SNS and malformed bodies, opt-in converter configuration, and use of the supplied mapper's parsing configuration.

Validation:

./mvnw -B -pl spring-cloud-aws-sqs -am \
  -Dtest=SqsHeaderMapperTests,SnsAwareSqsHeaderMapperTests \
  -Dsurefire.failIfNoSpecifiedTests=false test

All 44 tests passed, with no failures, errors, or skipped tests. The build's Spotless checks also passed. The full repository and LocalStack integration test suites were not run.

Please let me know if you'd like any further adjustments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component: sqs SQS integration related issue type: documentation Documentation or Samples related issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing SNS attributes when sending a message via SnsTemplate and receiving via @SqsListener

3 participants