Skip to content

Bedrock embeddings #392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## Next

### Added
- Added support for Amazon Bedrock embeddings via `BedrockEmbedding` class.
- Users can now leverage Bedrock-hosted Embedding models for vector generation.
- Added unit test and conducted Unit test

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually add one entry for each added feature, so it is better to group the first two in one item. The last one can be skipped.

## 1.9.0

### Fixed
Expand Down
2 changes: 2 additions & 0 deletions src/neo4j_graphrag/embeddings/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from .openai import AzureOpenAIEmbeddings, OpenAIEmbeddings
from .sentence_transformers import SentenceTransformerEmbeddings
from .vertexai import VertexAIEmbeddings
from .bedrockembeddings import BedrockEmbeddings

__all__ = [
"Embedder",
Expand All @@ -29,4 +30,5 @@
"VertexAIEmbeddings",
"MistralAIEmbeddings",
"CohereEmbeddings",
"BedrockEmbeddings"
]
81 changes: 81 additions & 0 deletions src/neo4j_graphrag/embeddings/bedrockembeddings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
from __future__ import annotations

from abc import ABC, abstractmethod
from typing import List
import boto3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this further involves 1) adding this package in the list of optional dependencies in the .toml file, 2) wrapping the import in a try-except block to avoid crashing the whole application if the module is not installed (please check how similar issue is handled in the other embedders e.g., here)

import json
import time
from neo4j_graphrag.embeddings.base import Embedder
from neo4j_graphrag.exceptions import EmbeddingsGenerationError


class BedrockEmbeddings(Embedder):
"""
Embedder implementation using Amazon Bedrock's Titan Text Embedding model.

This class integrates with AWS Bedrock via `boto3` and uses the Titan Embedding
model (`amazon.titan-embed-text-v2:0`) to generate 1536-dimensional vector
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please mention here that amazon.titan-embed-text-v2:0 is the default model used as other models could be accepted as well.

representations for input text.

Example:
>>> embedder = BedrockEmbeddings()
>>> embedding = embedder.embed_query("Neo4j integrates well with Bedrock.")
>>> len(embedding)
1536

Notes:
- Embeddings returned are 1536-dimensional vectors.
- A short sleep delay is applied to avoid throttling.
- This class uses the default AWS credentials chain supported by `boto3`.

AWS Authentication:
The following authentication methods are supported through boto3:

- Environment variables: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` (if needed)
- AWS credentials/config files (e.g., `~/.aws/credentials`)
- IAM roles (if running on EC2, Lambda, SageMaker, etc.)
- AWS CLI named profile via `AWS_PROFILE` environment variable
"""

def __init__(
self,
model_id: str = 'amazon.titan-embed-text-v2:0',
region: str = 'us-east-1'
):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be also good to allow other optional parameters in form of **kwargs (same as in the other embedder classes). same applies to the embed_query function below

"""
Initialize the BedrockEmbeddings instance.

Args:
model_id (str): Identifier for the Bedrock Titan embedding model.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe for the wrapper we can name it model instead of model_id (for consistency with the other embedding wrappers?)

Default is 'amazon.titan-embed-text-v2:0'.
region (str): AWS region where the Bedrock service is hosted.
Default is 'us-east-1'.
"""
self.model_id = model_id
self.bedrock = boto3.client('bedrock-runtime', region_name=region)

def embed_query(self, text: str) -> List[float]:
"""
Generate a vector embedding for the input text using Amazon Bedrock.

Args:
text (str): The input text string to be embedded.

Returns:
List[float]: A 1536-dimensional list representing the text embedding.

Raises:
EmbeddingsGenerationError: If an error occurs during the embedding process.
"""
try:
response = self.bedrock.invoke_model(
modelId=self.model_id,
contentType='application/json',
accept='application/json',
body=json.dumps({"inputText": text})
)
body = json.loads(response['body'].read())
time.sleep(0.05) # To prevent throttling
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather remove this from here as we have a RateLimitHandler interface that could be handling these kind of issues (for now it is only applied on LLM calls for answer generation, but we have a plan to apply it to the embedders as well. The user may also want to customise the behaviour so fixing the behaviour here won't help.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure. I will remove the
time.sleep(0.05)

Thanks

return body['embedding']
except Exception as e:
raise EmbeddingsGenerationError(f"Issue Generating Embeddings: {e}")
48 changes: 48 additions & 0 deletions tests/unit/embeddings/test_bedrockembedding.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
from unittest.mock import patch, MagicMock
import pytest
import json

from neo4j_graphrag.embeddings.bedrockembeddings import BedrockEmbeddings
from neo4j_graphrag.exceptions import EmbeddingsGenerationError


@patch("neo4j_graphrag.embeddings.bedrockembeddings.boto3.client")
def test_bedrock_embedder_happy_path(mock_boto_client):
# Mock AWS response with valid embedding
fake_embedding = [0.1] * 1024
fake_response = {
"embedding": fake_embedding
}

# Mock the .read() to return the fake response as JSON bytes
mock_body = MagicMock()
mock_body.read.return_value = json.dumps(fake_response).encode("utf-8")

# Mock the bedrock client
mock_bedrock_client = MagicMock()
mock_bedrock_client.invoke_model.return_value = {"body": mock_body}
mock_boto_client.return_value = mock_bedrock_client

# Instantiate the embedder and run embed_query
embedder = BedrockEmbeddings()
result = embedder.embed_query("Hello, Bedrock!")

# Assertions
assert isinstance(result, list)
assert len(result) == 1024
assert result == fake_embedding


@patch("neo4j_graphrag.embeddings.bedrockembeddings.boto3.client")
def test_bedrock_embedder_error_path(mock_boto_client):
# Simulate AWS client raising an exception
mock_bedrock_client = MagicMock()
mock_bedrock_client.invoke_model.side_effect = Exception("AWS error")
mock_boto_client.return_value = mock_bedrock_client

embedder = BedrockEmbeddings()

with pytest.raises(EmbeddingsGenerationError) as exc_info:
embedder.embed_query("This will fail.")

assert "Issue Generating Embeddings" in str(exc_info.value)
Loading