You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(langchain): rewrite custom LLM provider guide with BaseChatModel support
Complete rewrite of the custom LLM provider documentation with:
- Separate comprehensive guides for BaseLLM (text completion) and
BaseChatModel (chat)
- Correct method signatures (_call vs _generate)
- Proper async implementations
- Clear registration instructions (register_llm_provider vs
register_chat_provider)
- Working code examples with correct langchain-core imports
- Important notes on choosing the right base class
This addresses the gap where users were not properly guided on
implementing
custom chat models and were being directed to the wrong interface.
To register a custom LLM provider, you need to create a class that inherits from `BaseLanguageModel` and register it using `register_llm_provider`.
40
+
NeMo Guardrails supports two types of custom LLM providers:
41
+
1. **Text Completion Models** (`BaseLLM`) - For models that work with string prompts
42
+
2. **Chat Models** (`BaseChatModel`) - For models that work with message-based conversations
41
43
42
-
It is important to implement the following methods:
44
+
### Custom Text Completion LLM (BaseLLM)
43
45
44
-
**Required**:
46
+
To register a custom text completion LLM provider, create a class that inherits from `BaseLLM` and register it using `register_llm_provider`.
45
47
46
-
- `_call`
47
-
- `_llm_type`
48
+
**Required methods:**
49
+
- `_call`- Synchronous text completion
50
+
- `_llm_type`- Returns the LLM type identifier
48
51
49
-
**Optional**:
50
-
51
-
- `_acall`
52
-
- `_astream`
53
-
- `_stream`
54
-
- `_identifying_params`
55
-
56
-
In other words, to create your custom LLM provider, you need to implement the following interface methods: `_call`, `_llm_type`, and optionally `_acall`, `_astream`, `_stream`, and `_identifying_params`. Here's how you can do it:
52
+
**Optional methods:**
53
+
- `_acall`- Asynchronous text completion (recommended)
54
+
- `_stream`- Streaming text completion
55
+
- `_astream`- Async streaming text completion
56
+
- `_identifying_params`- Returns parameters for model identification
57
57
58
58
```python
59
59
from typing import Any, Iterator, List, Optional
60
60
61
-
from langchain.base_language import BaseLanguageModel
62
61
from langchain_core.callbacks.manager import (
63
-
CallbackManagerForLLMRun,
64
62
AsyncCallbackManagerForLLMRun,
63
+
CallbackManagerForLLMRun,
65
64
)
65
+
from langchain_core.language_models import BaseLLM
66
66
from langchain_core.outputs import GenerationChunk
67
67
68
68
from nemoguardrails.llm.providers import register_llm_provider
0 commit comments