Description

• Druid's transformSpec filtering operates after row deserialization and data ingestion
• Add a feature for pre-ingestion kafka header-based filtering in Supervisor

Motivation

Please provide the following for the desired feature or change:

• A detailed description of the intended use case, if applicable
  • We are currently ingesting from a Kafka topic and filtering out the messages using transformSpec. There are a lot of thrown away events which requires extra indexers.
• Rationale for why the desired feature/change would be beneficial
  • This feature will allow header based filtering and avoid additional compute requirements.