Skip to content

AIMessageChunk incorrectly merged tool_calls #8276

@huhaixiao

Description

@huhaixiao

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain.js documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain.js rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code

aiMessageChunk.concat(aiMessageChunk)

Root cause demo

function _mergeLists(left, right) {
  debugger;
  if (left === undefined && right === undefined) {
    return undefined;
  } else if (left === undefined || right === undefined) {
    return left || right;
  } else {
    const merged = [...left];
    for (const item of right) {
      console.log("Merging item:", item);
      if (
        typeof item === "object" &&
        "index" in item &&
        typeof item.index === "number"
      ) {
        const toMerge = merged.findIndex(
          (leftItem) => leftItem.index === item.index
        );
        if (toMerge !== -1) {
          merged[toMerge] = _mergeDicts(merged[toMerge], item);
        } else {
          merged.push(item);
        }
      } else if (
        typeof item === "object" &&
        "text" in item &&
        item.text === ""
      ) {
        // No-op - skip empty text blocks
        continue;
      } else {
        merged.push(item);
      }
    }
    return merged;
  }
}

function _mergeDicts(
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  left,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  right
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
) {
  debugger;
  const merged = { ...left };
  for (const [key, value] of Object.entries(right)) {
    if (merged[key] == null) {
      merged[key] = value;
    } else if (value == null) {
      continue;
    } else if (
      typeof merged[key] !== typeof value ||
      Array.isArray(merged[key]) !== Array.isArray(value)
    ) {
      throw new Error(
        `field[${key}] already exists in the message chunk, but with a different type.`
      );
    } else if (typeof merged[key] === "string") {
      if (key === "type") {
        // Do not merge 'type' fields
        continue;
      }
      merged[key] += value;
    } else if (typeof merged[key] === "object" && !Array.isArray(merged[key])) {
      merged[key] = _mergeDicts(merged[key], value);
    } else if (Array.isArray(merged[key])) {
      merged[key] = _mergeLists(merged[key], value);
    } else if (merged[key] === value) {
      continue;
    } else {
      console.warn(
        `field[${key}] already exists in this message chunk and value has unsupported type.`
      );
    }
  }
  return merged;
}

var left = {
  tool_calls: [
    {
      id: "call_50556714",
      function: "[Object]",
      index: 0,
      type: "function",
    },
  ],
};

var right = {
  tool_calls: [
    {
      id: "call_79113425",
      function: "[Object]",
      index: 0,
      type: "function",
    },
  ],
};
var merged = _mergeDicts(left, right);

console.log("Merged result:", merged);

/**
Merged result: {
  tool_calls: [
    {
      id: 'call_50556714call_79113425',
      function: '[Object][Object]',
      index: 0,
      type: 'function'
    }
  ]
}
 */

Error Message and Stack Trace (if applicable)

No response

Description

  • I’m trying to concat AIMessageChunks from a same stream to get a gathered AIMessage
  • But the the GatheredAIMessageChunk’s tool related properties are messed.

System Info

This doesn’t matter , I believe my code tells why

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions