Add instrumentation for Langchain #25
Conversation
Summary by CodeRabbit
WalkthroughThe changes introduce a new module for LangChain instrumentation, enabling automatic tracing and logging of LLM calls, chains, agents, tools, and retrievers within the Maxim SDK. Public exports are updated to include the instrumentation function, and the README changelog is updated to reflect these additions in versions 3.8.4 and 3.8.5. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant LangChainClass
participant MaximInstrumenter
participant MaximSDK
User->>LangChainClass: Call method (e.g., invoke)
activate LangChainClass
LangChainClass->>MaximInstrumenter: Wrapped method intercepts call
MaximInstrumenter->>MaximSDK: Start trace/span/generation
MaximInstrumenter->>LangChainClass: Call original method
LangChainClass-->>MaximInstrumenter: Return result or error
MaximInstrumenter->>MaximSDK: Log result or error, end trace/span
MaximInstrumenter-->>User: Return result or raise error
Poem
✨ Finishing Touches
🧪 Generate Unit Tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (3)
README.md(1 hunks)maxim/logger/langchain/__init__.py(2 hunks)maxim/logger/langchain/instrumenter.py(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
maxim/logger/langchain/__init__.py (1)
maxim/logger/langchain/instrumenter.py (1)
instrument_langchain(41-41)
🪛 Pylint (3.3.7)
maxim/logger/langchain/__init__.py
[error] 2-2: Cannot import 'instrumenter' due to 'expected an indented block after function definition on line 41 (maxim.logger.langchain.instrumenter, line 42)'
(E0001)
maxim/logger/langchain/instrumenter.py
[error] 42-42: Parsing failed: 'expected an indented block after function definition on line 41 (maxim.logger.langchain.instrumenter, line 42)'
(E0001)
🪛 Ruff (0.11.9)
maxim/logger/langchain/instrumenter.py
42-42: SyntaxError: Expected an indented block after function definition
50-50: SyntaxError: Unexpected indentation
70-70: Trailing comma missing
(COM812)
92-92: Trailing comma missing
(COM812)
96-96: Trailing comma missing
(COM812)
120-120: Trailing comma missing
(COM812)
132-132: Trailing comma missing
(COM812)
141-141: Trailing comma missing
(COM812)
163-163: Trailing comma missing
(COM812)
167-167: Trailing comma missing
(COM812)
191-191: Trailing comma missing
(COM812)
203-203: Trailing comma missing
(COM812)
213-213: Trailing comma missing
(COM812)
235-235: Trailing comma missing
(COM812)
259-259: Trailing comma missing
(COM812)
275-275: Trailing comma missing
(COM812)
294-294: Trailing comma missing
(COM812)
310-310: Trailing comma missing
(COM812)
🔇 Additional comments (2)
README.md (1)
65-72: LGTM!The changelog entries correctly document the new Langchain instrumentation features introduced in versions 3.8.4 and 3.8.5.
maxim/logger/langchain/instrumenter.py (1)
385-427: Well-designed instrumentation implementation!The patching strategy is comprehensive and handles:
- Both sync and async method variants
- Optional dependencies gracefully
- Multiple LangChain components (LLMs, chains, agents, tools, retrievers)
The use of lambda with default argument capture on line 401 is a clever way to handle the async flag.
| @@ -1,4 +1,5 @@ | |||
| from .tracer import MaximLangchainTracer | |||
| from .instrumenter import instrument_langchain | |||
There was a problem hiding this comment.
Fix the syntax error in instrumenter.py before importing.
The import will fail due to a syntax error in instrumenter.py at line 42. The docstring needs proper indentation.
Also applies to: 36-36
🧰 Tools
🪛 Pylint (3.3.7)
[error] 2-2: Cannot import 'instrumenter' due to 'expected an indented block after function definition on line 41 (maxim.logger.langchain.instrumenter, line 42)'
(E0001)
🤖 Prompt for AI Agents
In maxim/logger/langchain/instrumenter.py at line 42 and also line 36, fix the
syntax error caused by improper indentation of the docstring. Ensure the
docstring is correctly indented according to Python standards so that the file
can be imported without syntax errors in __init__.py at line 2.
| def _wrap_llm(method: Callable, async_fn: bool) -> Callable: | ||
| if async_fn: | ||
| async def async_wrapper(self, *args, **kwargs): | ||
| messages = args[0] if args else kwargs.get("messages") | ||
| metadata = kwargs.get("config", {}).get("metadata") or kwargs.get( | ||
| "metadata" | ||
| ) | ||
| maxim_meta = metadata.get("maxim") if isinstance(metadata, dict) else None | ||
| trace_id = maxim_meta.get("trace_id") if maxim_meta else None | ||
| gen_name = maxim_meta.get("generation_name") if maxim_meta else None | ||
|
|
||
| is_local = trace_id is None and current_trace() is None | ||
| final_trace = trace_id or (current_trace().id if current_trace() else str(uuid4())) | ||
| trace = current_trace() or logger.trace(TraceConfig(id=final_trace)) | ||
| generation = None | ||
| try: | ||
| model, params = parse_langchain_model_parameters(**kwargs) | ||
| provider = parse_langchain_provider({"name": self.__class__.__name__}) | ||
| parsed_messages = parse_langchain_messages(messages) if messages is not None else [] | ||
| generation = trace.generation( | ||
| GenerationConfig( | ||
| id=str(uuid4()), | ||
| provider=provider, | ||
| model=model, | ||
| name=gen_name, | ||
| model_parameters=params, | ||
| messages=parsed_messages, | ||
| ) | ||
| ) | ||
| except Exception as e: # pragma: no cover - best effort | ||
| scribe().warning( | ||
| f"[MaximSDK] Failed preparing langchain generation: {e}" | ||
| ) | ||
|
|
||
| try: | ||
| result = await method(self, *args, **kwargs) | ||
| except Exception as err: | ||
| if generation is not None: | ||
| try: | ||
| generation.error(parse_langchain_llm_error(err)) | ||
| except Exception: | ||
| pass | ||
| if is_local: | ||
| trace.end() | ||
| raise | ||
|
|
||
| try: | ||
| if generation is not None: | ||
| from langchain_core.messages import BaseMessage | ||
| from langchain_core.outputs import LLMResult | ||
|
|
||
| if isinstance(result, LLMResult): | ||
| generation.result(parse_langchain_llm_result(result)) | ||
| elif isinstance(result, BaseMessage): | ||
| generation.result( | ||
| parse_base_message_to_maxim_generation(result) | ||
| ) | ||
| else: | ||
| generation.result(result) | ||
| if is_local: | ||
| if isinstance(result, str): | ||
| trace.set_output(result) | ||
| elif hasattr(result, "content"): | ||
| trace.set_output(getattr(result, "content", "")) | ||
| trace.end() | ||
| except Exception as e: # pragma: no cover - best effort | ||
| scribe().warning( | ||
| f"[MaximSDK] Failed logging langchain generation: {e}" | ||
| ) | ||
| return result | ||
|
|
||
| return async_wrapper | ||
| else: | ||
| def sync_wrapper(self, *args, **kwargs): | ||
| messages = args[0] if args else kwargs.get("messages") | ||
| metadata = kwargs.get("config", {}).get("metadata") or kwargs.get( | ||
| "metadata" | ||
| ) | ||
| maxim_meta = metadata.get("maxim") if isinstance(metadata, dict) else None | ||
| trace_id = maxim_meta.get("trace_id") if maxim_meta else None | ||
| gen_name = maxim_meta.get("generation_name") if maxim_meta else None | ||
|
|
||
| is_local = trace_id is None and current_trace() is None | ||
| final_trace = trace_id or (current_trace().id if current_trace() else str(uuid4())) | ||
| trace = current_trace() or logger.trace(TraceConfig(id=final_trace)) | ||
| generation = None | ||
| try: | ||
| model, params = parse_langchain_model_parameters(**kwargs) | ||
| provider = parse_langchain_provider({"name": self.__class__.__name__}) | ||
| parsed_messages = parse_langchain_messages(messages) if messages is not None else [] | ||
| generation = trace.generation( | ||
| GenerationConfig( | ||
| id=str(uuid4()), | ||
| provider=provider, | ||
| model=model, | ||
| name=gen_name, | ||
| model_parameters=params, | ||
| messages=parsed_messages, | ||
| ) | ||
| ) | ||
| except Exception as e: # pragma: no cover - best effort | ||
| scribe().warning( | ||
| f"[MaximSDK] Failed preparing langchain generation: {e}" | ||
| ) | ||
|
|
||
| try: | ||
| result = method(self, *args, **kwargs) | ||
| except Exception as err: | ||
| if generation is not None: | ||
| try: | ||
| generation.error(parse_langchain_llm_error(err)) | ||
| except Exception: | ||
| pass | ||
| if is_local: | ||
| trace.end() | ||
| raise | ||
|
|
||
| try: | ||
| if generation is not None: | ||
| from langchain_core.messages import BaseMessage | ||
| from langchain_core.outputs import LLMResult | ||
|
|
||
| if isinstance(result, LLMResult): | ||
| generation.result(parse_langchain_llm_result(result)) | ||
| elif isinstance(result, BaseMessage): | ||
| generation.result( | ||
| parse_base_message_to_maxim_generation(result) | ||
| ) | ||
| else: | ||
| generation.result(result) | ||
| if is_local: | ||
| if isinstance(result, str): | ||
| trace.set_output(result) | ||
| elif hasattr(result, "content"): | ||
| trace.set_output(getattr(result, "content", "")) | ||
| trace.end() | ||
| except Exception as e: # pragma: no cover - best effort | ||
| scribe().warning( | ||
| f"[MaximSDK] Failed logging langchain generation: {e}" | ||
| ) | ||
| return result | ||
|
|
||
| return sync_wrapper | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Consider adding trailing commas for better diff readability.
While not critical, adding trailing commas to multi-line function calls improves maintainability and creates cleaner diffs when parameters are added or removed.
Example locations that could benefit from trailing commas:
- Line 70: After
"metadata" - Line 92: After the closing
) - Line 141: After
"metadata" - Line 163: After the closing
)
🧰 Tools
🪛 Ruff (0.11.9)
70-70: Trailing comma missing
(COM812)
92-92: Trailing comma missing
(COM812)
96-96: Trailing comma missing
(COM812)
120-120: Trailing comma missing
(COM812)
132-132: Trailing comma missing
(COM812)
141-141: Trailing comma missing
(COM812)
163-163: Trailing comma missing
(COM812)
167-167: Trailing comma missing
(COM812)
191-191: Trailing comma missing
(COM812)
203-203: Trailing comma missing
(COM812)
🤖 Prompt for AI Agents
In maxim/logger/langchain/instrumenter.py between lines 65 and 208, add trailing
commas to multi-line function calls and dictionary literals, specifically after
"metadata" on lines 70 and 141, and after the closing parentheses on lines 92
and 163. This involves placing a comma at the end of these lines to improve diff
readability and maintainability without changing functionality.
| def instrument_langchain(logger: Logger) -> None: | ||
| """Instrument LangChain classes for automatic tracing. | ||
| The helper monkey-patches core LangChain classes so that LLM calls are | ||
| recorded as Maxim generations and chain or agent executions become spans. | ||
| If no trace is active, a temporary trace is created around the call so logs | ||
| are always captured. | ||
| """ |
There was a problem hiding this comment.
Fix the critical syntax error in the docstring.
The docstring is not properly indented, causing a syntax error that prevents the module from being imported.
Apply this fix:
def instrument_langchain(logger: Logger) -> None:
-"""Instrument LangChain classes for automatic tracing.
-
- The helper monkey-patches core LangChain classes so that LLM calls are
- recorded as Maxim generations and chain or agent executions become spans.
- If no trace is active, a temporary trace is created around the call so logs
- are always captured.
- """
+ """Instrument LangChain classes for automatic tracing.
+
+ The helper monkey-patches core LangChain classes so that LLM calls are
+ recorded as Maxim generations and chain or agent executions become spans.
+ If no trace is active, a temporary trace is created around the call so logs
+ are always captured.
+ """📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def instrument_langchain(logger: Logger) -> None: | |
| """Instrument LangChain classes for automatic tracing. | |
| The helper monkey-patches core LangChain classes so that LLM calls are | |
| recorded as Maxim generations and chain or agent executions become spans. | |
| If no trace is active, a temporary trace is created around the call so logs | |
| are always captured. | |
| """ | |
| def instrument_langchain(logger: Logger) -> None: | |
| """Instrument LangChain classes for automatic tracing. | |
| The helper monkey-patches core LangChain classes so that LLM calls are | |
| recorded as Maxim generations and chain or agent executions become spans. | |
| If no trace is active, a temporary trace is created around the call so logs | |
| are always captured. | |
| """ |
🧰 Tools
🪛 Ruff (0.11.9)
42-42: SyntaxError: Expected an indented block after function definition
🪛 Pylint (3.3.7)
[error] 42-42: Parsing failed: 'expected an indented block after function definition on line 41 (maxim.logger.langchain.instrumenter, line 42)'
(E0001)
🤖 Prompt for AI Agents
In maxim/logger/langchain/instrumenter.py around lines 41 to 48, the docstring
for the function instrument_langchain is not properly indented, causing a syntax
error. Fix this by indenting all lines of the docstring to align with the
function body, ensuring the triple quotes and all text inside the docstring are
consistently indented one level inside the function definition.
Summary
Testing
python3 -m pytest -q(fails: ModuleNotFoundError: No module named 'filetype')https://chatgpt.com/codex/tasks/task_e_6849518ba724832088860946f02f7372