Skip to content

Commit 6f722b4

Browse files
willemcdejonghwillemcdejongh
committed
updates
1 parent 48edb93 commit 6f722b4

File tree

3 files changed

+201
-173
lines changed

3 files changed

+201
-173
lines changed

‎agent-os/customize/os/filter_knowledge.mdx‎

Lines changed: 0 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -272,87 +272,6 @@ def validate_and_send_filter(filter_expr, message):
272272
return None
273273
```
274274

275-
## Troubleshooting
276-
277-
### Filter Not Working
278-
279-
<AccordionGroup>
280-
<Accordion title="Verify metadata keys exist">
281-
Check that the keys you're filtering on actually exist in your knowledge base:
282-
283-
```python
284-
# Add content with explicit metadata
285-
knowledge.add_content(
286-
path="doc.pdf",
287-
metadata={"status": "published", "category": "tech"}
288-
)
289-
290-
# Now filter will work
291-
filter = EQ("status", "published")
292-
```
293-
</Accordion>
294-
295-
<Accordion title="Check filter serialization">
296-
Print the serialized filter to verify structure:
297-
298-
```python
299-
filter_expr = EQ("status", "published") & GT("views", 100)
300-
print(json.dumps(filter_expr.to_dict(), indent=2))
301-
```
302-
</Accordion>
303-
</AccordionGroup>
304-
305-
### Complex Filters Failing
306-
307-
<AccordionGroup>
308-
<Accordion title="Break down into smaller filters">
309-
Test each condition individually:
310-
311-
```python
312-
# Test each part separately
313-
filter1 = EQ("status", "published") # Test
314-
filter2 = GT("date", "2024-01-01") # Test
315-
filter3 = IN("region", ["US", "EU"]) # Test
316-
317-
# Then combine
318-
combined = AND(filter1, filter2, filter3)
319-
```
320-
</Accordion>
321-
322-
<Accordion title="Verify JSON validity">
323-
Use a JSON validator on the serialized output:
324-
325-
```python
326-
import json
327-
328-
try:
329-
filter_dict = filter_expr.to_dict()
330-
json_str = json.dumps(filter_dict)
331-
json.loads(json_str) # Verify it parses
332-
print("✓ Valid JSON")
333-
except json.JSONDecodeError as e:
334-
print(f"✗ Invalid JSON: {e}")
335-
```
336-
</Accordion>
337-
338-
<Accordion title="Check operator precedence">
339-
Make sure nested logic is clear and well-structured:
340-
341-
```python
342-
# Clear nested structure
343-
filter = OR(
344-
AND(EQ("a", 1), EQ("b", 2)),
345-
EQ("c", 3)
346-
)
347-
348-
# Break down complex filters for readability
349-
condition1 = AND(EQ("a", 1), EQ("b", 2))
350-
condition2 = EQ("c", 3)
351-
filter = OR(condition1, condition2)
352-
```
353-
</Accordion>
354-
</AccordionGroup>
355-
356275
## Complete Examples
357276

358277
### Example 1: Simple Dict Filter

‎concepts/knowledge/core-concepts/advanced-filtering.mdx‎

Lines changed: 89 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,14 @@
11
---
22
title: Advanced Filtering
3-
description: Understand how to add filter expressions to your Knowledge search requests.
4-
keywords: [search, retrieval, rag, agentic rag, agentic search, search, filters, metadata]
3+
description: Use filter expressions (EQ, AND, OR, NOT) for complex logical filtering of knowledge base searches.
4+
keywords: [search, retrieval, rag, agentic rag, agentic search, search, filters, metadata, filter expressions]
55
---
66

7-
When agents search through knowledge bases, sometimes you need more control than just "find similar content." Maybe you want to search only within specific documents, exclude outdated information, or focus on content from particular sources. That's where advanced filtering comes in—it lets you precisely target which content gets retrieved.
7+
When basic dictionary filters aren't enough, filter expressions give you powerful logical control over knowledge searches. Use them to combine multiple conditions with AND/OR logic, exclude content with NOT, or perform comparisons like "greater than" and "less than".
88

9-
## How Knowledge Filtering Works
9+
For basic filtering with dictionary format, see [Search & Retrieval](/concepts/knowledge/core-concepts/search-retrieval).
1010

11-
Think of filtering like adding smart constraints to a library search. Instead of searching through every book, you can tell the librarian: "Only look in the science section, published after 2020, but exclude textbooks." Knowledge filtering works the same way—you specify criteria based on the metadata attached to your content.
12-
13-
<Steps>
14-
<Step title="Metadata Assignment">
15-
When you add content, attach metadata like department, document type, date, or any custom attributes.
16-
</Step>
17-
<Step title="Filter Construction">
18-
Build filter expressions using comparison and logical operators to define your search criteria.
19-
</Step>
20-
<Step title="Targeted Search">
21-
The knowledge base only searches through content that matches your filter conditions.
22-
</Step>
23-
<Step title="Contextual Results">
24-
You get precisely the information you need from exactly the right sources.
25-
</Step>
26-
</Steps>
27-
28-
## Available Filter Expressions
11+
## Filter Expression Operators
2912

3013
Agno provides a rich set of filter expressions that can be combined to create sophisticated search criteria:
3114

@@ -314,74 +297,7 @@ async def progressive_search(agent, query, base_filters=None):
314297
return broad_results
315298
```
316299

317-
## Working with Metadata
318-
319-
### Designing Effective Metadata
320-
321-
Good filtering starts with thoughtful metadata design:
322-
323-
```python
324-
# ✅ Rich, searchable metadata
325-
good_metadata = {
326-
"document_type": "policy",
327-
"department": "hr",
328-
"category": "benefits",
329-
"audience": "all_employees",
330-
"last_updated": "2024-01-15",
331-
"version": "2.1",
332-
"tags": ["health_insurance", "401k", "vacation"],
333-
"sensitivity": "internal"
334-
}
335-
336-
# ❌ Sparse, hard to filter metadata
337-
poor_metadata = {
338-
"type": "doc",
339-
"id": "12345"
340-
}
341-
```
342-
343-
### Dynamic Metadata Assignment
344-
345-
Add metadata programmatically based on content:
346-
347-
```python
348-
def assign_metadata(file_path: str) -> dict:
349-
"""Generate metadata based on file characteristics."""
350-
metadata = {}
351-
352-
# Extract from filename
353-
if "policy" in file_path.lower():
354-
metadata["document_type"] = "policy"
355-
elif "guide" in file_path.lower():
356-
metadata["document_type"] = "guide"
357-
358-
# Extract department from path
359-
if "/hr/" in file_path:
360-
metadata["department"] = "hr"
361-
elif "/engineering/" in file_path:
362-
metadata["department"] = "engineering"
363-
364-
# Add timestamp
365-
metadata["indexed_at"] = datetime.now().isoformat()
366-
367-
return metadata
368-
369-
# Use when adding content
370-
for file_path in document_files:
371-
knowledge.add_content(
372-
path=file_path,
373-
metadata=assign_metadata(file_path)
374-
)
375-
```
376-
377-
## Best Practices for Advanced Filtering
378-
379-
### Metadata Strategy
380-
381-
- **Be Consistent**: Use standardized values (e.g., always "hr" not sometimes "HR" or "human_resources")
382-
- **Think Hierarchically**: Use nested categories when appropriate (`department.team`, `location.region`)
383-
- **Include Temporal Data**: Add dates, versions, or other time-based metadata for lifecycle management
384-
- **Add Semantic Tags**: Include searchable tags or keywords that might not appear in the content
300+
## Best Practices for Filter Expressions
385301

386302
### Filter Design
387303

@@ -418,6 +334,89 @@ def safe_filter_search(agent, query, filters):
418334
return agent.get_relevant_docs_from_knowledge(query=query)
419335
```
420336

337+
## Troubleshooting
338+
339+
### Filter Not Working
340+
341+
<AccordionGroup>
342+
<Accordion title="Verify metadata keys exist">
343+
Check that the keys you're filtering on actually exist in your knowledge base:
344+
345+
```python
346+
# Add content with explicit metadata
347+
knowledge.add_content(
348+
path="doc.pdf",
349+
metadata={"status": "published", "category": "tech"}
350+
)
351+
352+
# Now filter will work
353+
filter_expr = EQ("status", "published")
354+
```
355+
</Accordion>
356+
357+
<Accordion title="Check filter structure">
358+
Print the filter to verify it's constructed correctly:
359+
360+
```python
361+
from agno.filters import EQ, GT, AND
362+
363+
filter_expr = AND(EQ("status", "published"), GT("views", 100))
364+
print(filter_expr.to_dict())
365+
```
366+
</Accordion>
367+
</AccordionGroup>
368+
369+
### Complex Filters Failing
370+
371+
<AccordionGroup>
372+
<Accordion title="Break down into smaller filters">
373+
Test each condition individually:
374+
375+
```python
376+
# Test each part separately
377+
filter1 = EQ("status", "published") # Test
378+
filter2 = GT("date", "2024-01-01") # Test
379+
filter3 = IN("region", ["US", "EU"]) # Test
380+
381+
# Then combine
382+
combined = AND(filter1, filter2, filter3)
383+
```
384+
</Accordion>
385+
386+
<Accordion title="Verify filter structure">
387+
Check that nested logic is correctly structured:
388+
389+
```python
390+
import json
391+
392+
try:
393+
filter_dict = filter_expr.to_dict()
394+
json_str = json.dumps(filter_dict)
395+
json.loads(json_str) # Verify it parses
396+
print("Valid filter structure")
397+
except (TypeError, ValueError) as e:
398+
print(f"Invalid filter: {e}")
399+
```
400+
</Accordion>
401+
402+
<Accordion title="Check operator precedence">
403+
Make sure nested logic is clear and well-structured:
404+
405+
```python
406+
# Clear nested structure
407+
filter_expr = OR(
408+
AND(EQ("a", 1), EQ("b", 2)),
409+
EQ("c", 3)
410+
)
411+
412+
# Break down complex filters for readability
413+
condition1 = AND(EQ("a", 1), EQ("b", 2))
414+
condition2 = EQ("c", 3)
415+
filter_expr = OR(condition1, condition2)
416+
```
417+
</Accordion>
418+
</AccordionGroup>
419+
421420
### Vector Database Support
422421

423422
Advanced filter expressions (using `FilterExpr` like `EQ()`, `AND()`, etc.) have varying support across vector databases:

0 commit comments

Comments
 (0)