Skip to content

fix(serde): ProtobufFileSerde deserializer fails to resolve types in google.protobuf.Any fields #1730

@andyvl

Description

@andyvl

Problem

ProtobufFileSerde fails to deserialize messages that contain google.protobuf.Any fields holding non-well-known types. The UI falls back to the Fallback serde and the following error is logged:

com.google.protobuf.InvalidProtocolBufferException: Cannot find type for url: type.googleapis.com/<SomeMessage>

Root Cause

deserializer() delegates to ProtobufSchemaUtils.toJson(), which internally calls JsonFormat.printer().print(message) with no TypeRegistry. Without a TypeRegistry, the Protobuf JSON printer cannot resolve the type_url inside an Any field to its concrete descriptor — even when that descriptor was successfully loaded from the protobufFilesDir.

By contrast, serializer() already correctly builds a TypeRegistry from descriptorPaths.keySet() and passes it to JsonFormat.parser().usingTypeRegistry(...). The deserializer is simply missing the same treatment.

// serializer() – correct
TypeRegistry typeRegistry = TypeRegistry.newBuilder()
    .add(descriptorPaths.keySet())
    .build();
JsonFormat.parser().usingTypeRegistry(typeRegistry).merge(input, builder);

// deserializer() – broken: no TypeRegistry passed
byte[] jsonFromProto = ProtobufSchemaUtils.toJson(protoMsg);  // Any fields unresolvable

Steps to Reproduce

  1. Configure ProtobufFileSerde with a protobufFilesDir containing a root message that has a google.protobuf.Any field (e.g. Notification.body).
  2. Ensure the proto files directory also contains the definition of the type embedded inside the Any field (e.g. MyCustomMessage).
  3. Consume a message from the UI — the value serde shows Fallback and the log contains Cannot find type for url: type.googleapis.com/MyCustomMessage.

Expected Behaviour

The nested message inside the Any field is resolved using the loaded descriptors and rendered as JSON.

Proposed Fix

Build a TypeRegistry in deserializer() (same as serializer() already does) and use JsonFormat.printer().usingTypeRegistry(typeRegistry).print(protoMsg) instead of ProtobufSchemaUtils.toJson(protoMsg).

TypeRegistry typeRegistry = TypeRegistry.newBuilder().add(descriptorPaths.keySet()).build();
// ...
var result = JsonFormat.printer().usingTypeRegistry(typeRegistry).print(protoMsg);

No new imports are needed — both TypeRegistry and JsonFormat are already present in the file.

Environment

  • kafbat-ui: v1.4.2 (also present in main as of 2026-03-18)
  • protobuf-java: 3.25.5

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions