Metadata v8

[patrykwegrzyn]

Redpanda compatibility

[mcollina]

There seems to be a CI problem?

[patrykwegrzyn]

Yeah its a weird one, i'll have a look i tested locally against 3.7

[patrykwegrzyn]

I can see some issues with base tests when force v8.
v8 don't have topic id , any ways to get negotiated api versions from the client ? We could make this version specific.
Open to suggestions

[mcollina]

Actually it seems it was just a flake.

@ShogunPanda can you take a look?

[ShogunPanda]

@patrykwegrzyn Thanks for your contribution!
The code looks good to me, but I just have a little issue: so far I tried not to leave any "hole" in the supported API versions. Could you please implement Metadata v9, v10, v11 as well? I guess it will be mostly copying files over and make little modifications.
Thanks!

[ShogunPanda]

Nit: can you please add { and } here?

[patrykwegrzyn]

I can see some issues with base tests when force v8. v8 don't have topic id , any ways to get negotiated api versions from the client ? We could make this version specific. Open to suggestions

Any thoughts on this? this test will break on lower versions , can we move it to version specific test with something like
metadataV8.async(connection, null, false, true) ?

[ShogunPanda]

We already negotiate the version from the broker, see https://github.com/platformatic/kafka/blob/main/src/clients/base/base.ts#L362 and https://github.com/platformatic/kafka/blob/main/src/clients/base/base.ts#L552
If it poses an issue when testing Redpanda, we can just skip it.

[patrykwegrzyn]

We already negotiate the version from the broker, see https://github.com/platformatic/kafka/blob/main/src/clients/base/base.ts#L362 and https://github.com/platformatic/kafka/blob/main/src/clients/base/base.ts#L552 If it poses an issue when testing Redpanda, we can just skip it.

Cheers for this!!
Exactly what I needed.
My usual flow is Kafka first, then Redpanda, forcing the client up to max API version im working on (v8 here). Totally missed the base-case issue without forcing it, cause the client was happily locking onto v12.
Hope this make sense.

Will you be ok with

  const getApi = promisify(client[kGetApi].bind(client))
  const api = await getApi('Metadata')
  const testTopic = `test-topic-${randomUUID()}`

  // Fetch metadata with autocreate
  const metadata = await client.metadata({
    topics: [testTopic],
    autocreateTopics: true
  })
  const topicMetadata = metadata.topics.get(testTopic)!
  if (api.version >= 10) {
    strictEqual(typeof topicMetadata.id, 'string')
  }

and make strictEqual(typeof topicMetadata.id, 'string') check only on v10+ versions ?

[ShogunPanda]

Yup, go ahead!

[patrykwegrzyn]

sweet ill submit Metadata v9, v10, v11 after holiday