FIP: Add Message Type Filtering to Hub API Endpoints

[TimDaub]

Abstract

Add optional message_type query parameter to Hub API endpoints (castsByFid, reactionsByFid, etc.) to allow filtering between ADD and REMOVE message types. This reduces bandwidth, improves performance, and enables more efficient pagination for applications that only need to process one message type.

Problem Statement

Currently, Hub API endpoints like castsByFid and reactionsByFid return all message types (both MESSAGE_TYPE_*_ADD and MESSAGE_TYPE_*_REMOVE) without the ability to filter. According to documentation, castsByFid should "return CastAdds for an Fid in reverse chron order," but in practice, it returns both ADD and REMOVE messages.

This creates significant inefficiencies:

1. Unnecessary data transfer: Applications that only need ADD messages (e.g., displaying a user's casts) must download and filter out all REMOVE tombstones client-side.

2. Pagination overhead: For users who have deleted most of their content, applications must paginate through potentially thousands of REMOVE messages to find remaining ADD messages. For example, a user with 2,300 deletion tombstones requires 23+ API requests (at 100 items per page) before reaching any actual content.

3. Processing waste: Every application must implement identical client-side filtering logic, duplicating work that could be done once at the Hub level.

Proposed Solution

Add an optional message_type query parameter to message retrieval endpoints:

GET /v1/castsByFid?fid={fid}&message_type=MESSAGE_TYPE_CAST_ADD
GET /v1/reactionsByFid?fid={fid}&reaction_type=Like&message_type=MESSAGE_TYPE_REACTION_ADD

Parameter Specification

Parameter	Type	Values	Default	Description
message_type	enum	MESSAGE_TYPE_*_ADD, MESSAGE_TYPE_*_REMOVE	null (no filter)	Optional filter for message type

When unspecified, endpoints return all messages (current behavior). When specified, only messages of that type are returned.

Use Cases

1. Content Display

Applications displaying a user's posts need only ADD messages. Filtering removes unnecessary REMOVE tombstones from the response.

2. Deletion Tools

Applications helping users delete old content need only ADD messages to identify what can be deleted. Currently, they must paginate through all REMOVE messages (previously deleted content) to find remaining ADD messages.

3. Analytics & Indexing

Services indexing Farcaster content can fetch only ADD messages initially, then separately process REMOVE messages for state reconciliation.

4. Bandwidth-Constrained Clients

Mobile and embedded clients benefit from smaller response sizes when they only need one message type.

Rationale

Why not filter client-side?

• Client-side filtering still requires downloading all data, wasting bandwidth
• Pagination becomes inefficient when most messages are filtered out
• Every application reimplements the same filtering logic

Why not use a different endpoint?

• Creating separate endpoints (e.g., /castAddsByFid, /castRemovesByFid) increases API surface area
• A query parameter maintains backward compatibility while adding flexibility

Backward compatibility:

• When message_type is omitted, behavior is unchanged (all messages returned)
• Existing applications continue to work without modification
• New applications can opt-in to filtering as needed

Example: Current vs Proposed

Current Behavior (User with deleted content)

Request 1: /v1/castsByFid?fid=5309&pageSize=100
Response: 100 MESSAGE_TYPE_CAST_REMOVE messages

Request 2: /v1/castsByFid?fid=5309&pageSize=100&pageToken=...
Response: 100 MESSAGE_TYPE_CAST_REMOVE messages

... (21 more requests) ...

Request 23: /v1/castsByFid?fid=5309&pageSize=100&pageToken=...
Response: 100 MESSAGE_TYPE_CAST_REMOVE messages

Total: 23 API calls, 2.3MB transferred (for a user with 2,300 deletions)

Proposed Behavior

Request 1: /v1/castsByFid?fid=5309&pageSize=100&message_type=MESSAGE_TYPE_CAST_ADD
Response: 0 messages (all casts deleted)

Total: 1 API call, ~100 bytes transferred

References

• Snapchain Hub API Documentation
• Related issue: Applications must paginate through deletion tombstones to find actual content

[vrypan]

How will pageSize and pageToken work in this case? My understanding is that right now, the node fetches keys sequentially from a kv-store. If this is the case, won't filtering the results complicate pagination?

I keep hearing from people who would like a lighter node, one that syncs selectively (specific FIDs for example), or on-demand. I would like hubs to do the bare minimum in order to a) keep implementation complexity down and b) keep hw requirements down.

If it is trivial to implement, that's a good idea. If it adds any complexity, clients who need it, should implement it on their side. And if it's a common need, there could be a proxy that implements these type of filtering and data-processing.

[TimDaub]

Hey, thanks for the feedback. I have no idea about the implementation details but I understand the problem you mention with the kv store. I mean if the architecture there is too constraint to make it happen, fine. But I'd like to at least push/challenge a little bit and ask the developers to find a solution internally. I intentionally choose to leave out implementation details because that isn't my area of expertise. Happy to add recommendations if implementers have suggestions here.
I'm all for elegance etc. so I see the concern. Anyways, not really familiar with the code base.

In any case this is a real problem that I have at the moment running deletecasts.com and it means I have to scan all messages of a profil which is quite resource-wasting.

[vrypan]

I'm not sure of the implementation details either. Can the devs do something?