-
Notifications
You must be signed in to change notification settings - Fork 3
feat: complete refactor to remove traceloop and speakeasy #154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 250 commits
e7ec296
94ad362
94f557c
54885a1
5a77ae3
54106f1
7b86dc1
6e0443e
d0ce2cb
e2836c8
3d6990f
72f5c7f
8e75dc5
3a90ffc
b869e64
53ff0f6
5e972e9
bf83be7
7770ef6
dd42d72
ec8dedd
1c77efb
b23523c
59b4485
e4917b6
6b82d0a
c1a37ee
cea0087
14ba86a
e73328f
c6db922
94b9ac0
d7ff0eb
0af91d6
f07e075
4f392c5
df3acc9
132ed84
ecb09ae
c1f7c4c
6aae4aa
6d8d2e5
49880e6
357840b
be0d5b0
82d892c
ccdbdcb
a59b61c
cd64b97
30bc4d4
940898e
e2079c8
4717499
584b431
15f2812
af15613
5b21d50
ebc3b7e
7233ca5
f999922
260dc8c
4941124
73bc981
bad9aac
05773a4
e8c1e67
c712d0b
3e54993
dcbe798
a9d31a9
7ac2d5b
74c4cc7
ef4df82
b948239
32bb618
3431eea
452145d
6414f2a
30afa7f
cb5e8f3
4338b9f
5bec259
a2e7aab
f5e6dbd
f934778
13f348b
c567f40
0c12425
d1e9da3
78a45ea
94b0f61
29d4886
e66b029
d82bc48
ac68fda
c163e6f
dbe4cc5
e6c1679
fc0643c
c85b62d
89ddc01
7be3b55
d89eb6c
4def1a0
7bdfdbb
a4cab4b
1853982
35fa07e
8f17736
f3241a4
5c1d1d0
d2f6618
79971f8
a739bf9
5e1c52c
3e836de
a308c5f
54e2fa1
50f6d71
84615ff
6576070
38fbb0f
caae699
07eb8d3
baec8e0
83306e5
631aa99
1d2aa06
1544fd9
ca01973
4f08a39
79e1db3
02e49ca
a2d68f6
54b0040
4474e14
4858c4c
e53aa34
a81be94
8ce07d0
f06505b
6538ff0
7022423
dda5d1b
59d73da
b22441c
43e507e
cb0225f
4329371
1a5f842
5cd3397
42878e7
c2c1a5c
2ebe473
3309a4f
84ca1d4
d996fee
2b2c8e7
0fb0818
33995d3
7e253fc
7f46723
232ebd4
a10d5eb
7b4c05b
5026b70
6b25eef
897ac72
8954847
77693c2
cf115c7
9535ca6
2feaa41
96b32b6
14d531d
2f58085
de8c987
ad6e6d7
29e9323
345c4a9
89bd29e
876c0bc
8275e43
f1f1797
b805696
d78560f
73abbdc
846b835
e5772d9
95dc31c
ea0e5e5
5698a0f
3c5f286
8b1536a
60c4b17
1f3fe19
27065f8
e328907
7c0da70
c15c3fd
956df36
84616e7
312cb85
71f9e77
ea9594c
3b034cf
416a3a2
d908d68
35b5e07
6e2ed49
555de19
e547a86
531c1bb
4291199
9605c79
9630737
c659fd0
13588f1
283198e
b6c02fa
0088068
9e23d99
5aec9a0
05bfe4f
30ef480
ddf1963
3819e0b
7c57bb4
e8e9492
af55619
24b0e7b
7082af1
a437d0c
25f010e
35b6420
f391125
bb881b4
f20c281
303f1ff
fcbd0a3
268533b
475ee17
b183714
fbee48e
2b9506a
42faba0
3b45e42
fded2f5
9733de0
923d420
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Claude Code Configuration - HoneyHive Python SDK | ||
|
|
||
| ## Project Context | ||
| This is the HoneyHive Python SDK (complete-refactor branch) - a comprehensive observability and evaluation platform for LLM applications. | ||
|
|
||
| ## Agent OS Integration | ||
| The project uses Agent OS for structured development. Key directories: | ||
| - Standards: `.agent-os/standards/` - Global coding standards | ||
| - Product: `.agent-os/product/` - Product documentation | ||
| - Specs: `.agent-os/specs/` - Feature specifications | ||
|
|
||
| ## Critical Project Rules | ||
|
|
||
| ### 🔴 MUST FOLLOW | ||
| 1. **ALWAYS use tox for testing** - Never run pytest directly | ||
| ```bash | ||
| tox -e py311 # Python 3.11 tests | ||
| tox -e unit # Unit tests only | ||
| ``` | ||
|
|
||
| 2. **Type hints are MANDATORY** - All functions must have type hints | ||
| 3. **No code in `__init__.py`** - Only imports allowed | ||
| 4. **Use Black formatting** - Line length 88 | ||
| 5. **Multi-instance tracers** - No singleton pattern | ||
|
|
||
| ### Key Patterns | ||
| - Unified `@trace` decorator works for both sync/async | ||
| - HTTP tracing disabled by default for performance | ||
| - Graceful degradation - never crash host application | ||
| - Environment variables: HH_*, HTTP_*, EXPERIMENT_* | ||
|
|
||
| ## Quick Commands | ||
|
|
||
| ### Testing | ||
| ```bash | ||
| tox -e py311 # Test on Python 3.11 | ||
| tox -e unit # Run unit tests | ||
| tox -e integration # Run integration tests | ||
| tox -e lint # Run linting | ||
| ``` | ||
|
|
||
| ### Common Patterns | ||
| ```python | ||
| # Initialize tracer | ||
| from honeyhive import HoneyHiveTracer | ||
|
|
||
| tracer = HoneyHiveTracer.init( | ||
| api_key="hh_api_...", | ||
| project="my-project" | ||
| ) | ||
|
|
||
| # Use decorators | ||
| @trace(event_type="llm_call") | ||
| async def my_function(): | ||
| return await process() | ||
| ``` | ||
|
|
||
| ## Development Workflow | ||
| 1. Check `.agent-os/product/roadmap.md` for current priorities | ||
| 2. Create specs in `.agent-os/specs/` for new features | ||
| 3. Follow standards in `.agent-os/standards/` | ||
| 4. Update `.agent-os/product/decisions.md` for architectural choices | ||
|
|
||
| ## References | ||
| - Product Overview: `.agent-os/product/overview.md` | ||
| - Code Style: `.agent-os/standards/code-style.md` | ||
| - Best Practices: `.agent-os/standards/best-practices.md` | ||
| - Technical Decisions: `.agent-os/product/decisions.md` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| { | ||
| "mcpServers": { | ||
| "praxis-os": { | ||
| "command": "${workspaceFolder}/.praxis-os/venv/bin/python", | ||
| "args": [ | ||
| "-m", | ||
| "ouroboros", | ||
| "--transport", | ||
| "dual", | ||
| "--log-level", | ||
| "INFO" | ||
| ], | ||
| "env": { | ||
| "PROJECT_ROOT": "${workspaceFolder}", | ||
| "PYTHONPATH": "${workspaceFolder}/.praxis-os", | ||
| "PYTHONUNBUFFERED": "1" | ||
| }, | ||
| "autoApprove": [ | ||
| "pos_search_project", | ||
| "pos_workflow", | ||
| "pos_browser", | ||
| "pos_filesystem", | ||
| "current_date", | ||
| "get_server_info" | ||
| ] | ||
| } | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| { | ||
| "mcpServers": { | ||
| "python-sdk": { | ||
| "command": "/Users/josh/src/github.com/honeyhiveai/python-sdk/.praxis-os/venv/bin/python", | ||
| "args": [ | ||
| "-m", | ||
| "ouroboros", | ||
| "--transport", | ||
| "dual", | ||
| "--log-level", | ||
| "DEBUG" | ||
| ], | ||
| "env": { | ||
| "PYTHONPATH": "/Users/josh/src/github.com/honeyhiveai/python-sdk/.praxis-os" | ||
| }, | ||
| "autoApprove": [ | ||
| "pos_search_project", | ||
| "pos_workflow", | ||
| "pos_browser", | ||
| "pos_filesystem", | ||
| "get_server_info", | ||
| "current_date" | ||
| ] | ||
| } | ||
| } | ||
| } | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bug: Temporary Artifacts Don't Belong in Version ControlA backup configuration file |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| # Analyze Product - HoneyHive Python SDK | ||
|
|
||
| When analyzing the existing codebase or adding Agent OS to existing code: | ||
|
|
||
| ## Analysis Process | ||
|
|
||
| ### 1. Understand Current Architecture | ||
| ```python | ||
| # Key directories to analyze | ||
| src/honeyhive/ | ||
| ├── api/ # API client layer | ||
| ├── tracer/ # OpenTelemetry integration | ||
| ├── evaluation/ # Evaluation framework | ||
| ├── models/ # Data models | ||
| └── utils/ # Shared utilities | ||
| ``` | ||
|
|
||
| ### 2. Key Architectural Patterns | ||
|
|
||
| #### Multi-Instance Support | ||
| - Each tracer instance is independent | ||
| - No global singleton pattern | ||
| - Thread-safe operations | ||
|
|
||
| #### Unified Decorators | ||
| ```python | ||
| # Single @trace works for both sync and async | ||
| from honeyhive.models import EventType | ||
|
|
||
| @trace(event_type=EventType.tool) | ||
| def sync_func(): pass | ||
|
|
||
| @trace(event_type=EventType.tool) | ||
| async def async_func(): pass | ||
| ``` | ||
|
|
||
| #### Graceful Degradation | ||
| - SDK never crashes host application | ||
| - Errors logged but handled gracefully | ||
| - Optional returns for non-critical operations | ||
|
|
||
| ### 3. Current Implementation Details | ||
|
|
||
| #### Testing Framework | ||
| - **950+ tests** currently passing (831 unit + 119 integration) | ||
| - **81.14% coverage** achieved (exceeds 80% requirement) | ||
| - **Two-tier testing**: Unit (mocked, fast) vs Integration (real APIs, no mocks) | ||
| - **tox** for test orchestration | ||
| - Python 3.11, 3.12, 3.13 support | ||
| - **NO MOCKS IN INTEGRATION TESTS** - Critical rule established | ||
|
|
||
| #### Configuration | ||
| - Environment variables: HH_*, HTTP_*, EXPERIMENT_* | ||
| - Configuration precedence: Constructor > Env > Defaults | ||
| - HTTP tracing disabled by default | ||
|
|
||
| #### Key Dependencies | ||
| - OpenTelemetry >=1.20.0 | ||
| - httpx >=0.24.0 | ||
| - pydantic >=2.0.0 | ||
| - Python 3.11+ required | ||
|
|
||
| ### 4. Integration Points | ||
|
|
||
| #### Provider Integrations | ||
| - OpenAI / Azure OpenAI | ||
| - Anthropic Claude | ||
| - Google Gemini | ||
| - AWS Bedrock | ||
| - 15+ more providers | ||
|
|
||
| #### Framework Support | ||
| - LangChain / LangGraph | ||
| - LlamaIndex | ||
| - CrewAI | ||
| - LiteLLM | ||
|
|
||
| ### 5. When Analyzing Existing Code | ||
|
|
||
| #### Check for: | ||
| - Existing test patterns | ||
| - Configuration mechanisms | ||
| - Error handling approaches | ||
| - Performance optimizations | ||
| - Security practices | ||
|
|
||
| #### Document in Agent OS: | ||
| - Update `.agent-os/product/features.md` with discovered features | ||
| - Add to `.agent-os/product/decisions.md` for architectural choices | ||
| - Create specs in `.agent-os/specs/` for improvements | ||
|
|
||
| ## Critical Patterns to Maintain | ||
|
|
||
| 1. **NO MOCKS IN INTEGRATION TESTS** - Integration tests must use real systems | ||
| 2. **Always use tox** for testing - Never pytest directly | ||
| 3. **Type hints mandatory** on all functions with docstrings | ||
| 4. **No code in __init__.py** files - Only imports | ||
| 5. **Multi-instance support** required - No singleton pattern | ||
| 6. **Graceful degradation** essential - Never crash host app | ||
| 7. **EventType enums only** - Never string literals in documentation | ||
| 8. **80% test coverage** minimum (project-wide) | ||
| 9. **Test count reporting** - Always report total tests correctly (unit + integration) | ||
|
|
||
| ## Standards to Follow | ||
| Always reference: | ||
| - **Best Practices**: `.agent-os/standards/best-practices.md` (includes Agent OS spec standards) | ||
| - **Technology Stack**: `.agent-os/standards/tech-stack.md` for technology choices | ||
| - **Code Style**: `.agent-os/standards/code-style.md` for coding standards | ||
|
|
||
| ## References | ||
| - **Product Documentation**: | ||
| - Overview: `.agent-os/product/overview.md` | ||
| - Features: `.agent-os/product/features.md` | ||
| - Roadmap: `.agent-os/product/roadmap.md` | ||
| - Decisions: `.agent-os/product/decisions.md` | ||
| - **Standards Documentation**: | ||
| - Best Practices: `.agent-os/standards/best-practices.md` | ||
| - Tech Stack: `.agent-os/standards/tech-stack.md` | ||
| - Code Style: `.agent-os/standards/code-style.md` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| # Create Spec - HoneyHive Python SDK | ||
|
|
||
| When creating specifications for new features, follow the Agent OS specification standards. | ||
|
|
||
| ## 🚨 CRITICAL: Follow Agent OS Standards | ||
|
|
||
| **All specification creation MUST follow the standards defined in:** | ||
| - **`.agent-os/standards/best-practices.md`** - Complete Agent OS specification standards (starting at "📋 Agent OS Specification Standards") | ||
|
|
||
| **Key Requirements**: | ||
| - **File Structure**: Follow the mandatory 5-file structure (srd.md, specs.md, tasks.md, README.md, implementation.md) | ||
| - **Content Standards**: Each file has specific required sections and format requirements | ||
| - **Task Format**: Follow checkbox specifications defined in `.cursor/rules/execute-tasks.mdc` | ||
| - **Date Standards**: Use current system date for all spec creation | ||
|
|
||
| ## Spec Creation Protocol | ||
|
|
||
| **MANDATORY**: When creating new Agent OS specs, AI assistants MUST: | ||
|
|
||
| ### 1. Get Current Date | ||
| ```bash | ||
| CURRENT_DATE=$(date +"%Y-%m-%d") | ||
| echo "Today is: $CURRENT_DATE" | ||
| ``` | ||
|
|
||
| ### 2. Create Directory with Proper Naming | ||
| ```bash | ||
| SPEC_NAME="your-spec-name" | ||
| SPEC_DIR=".agent-os/specs/${CURRENT_DATE}-${SPEC_NAME}" | ||
| mkdir -p "$SPEC_DIR" | ||
| ``` | ||
|
|
||
| ### 3. Create ALL Required Files | ||
| ```bash | ||
| # Create mandatory files | ||
| touch "$SPEC_DIR/srd.md" | ||
| touch "$SPEC_DIR/specs.md" | ||
| touch "$SPEC_DIR/tasks.md" | ||
|
|
||
| # Create recommended files | ||
| touch "$SPEC_DIR/README.md" | ||
|
|
||
| # Create optional files (if needed) | ||
| touch "$SPEC_DIR/implementation.md" | ||
| ``` | ||
|
|
||
| ### 4. Use Proper Headers in Each File | ||
| ```markdown | ||
| # Spec Name - File Type | ||
|
|
||
| **Date**: 2025-09-06 | ||
| **Status**: Draft/Active/Completed | ||
| **Priority**: High/Medium/Low | ||
| ``` | ||
|
|
||
| ## Validation Commands | ||
|
|
||
| **Use the validation commands defined in `.agent-os/standards/best-practices.md`** | ||
|
|
||
| **Quick Validation**: | ||
| ```bash | ||
| # Get current date for spec creation | ||
| CURRENT_DATE=$(date +"%Y-%m-%d") | ||
| echo "Today is: $CURRENT_DATE" | ||
|
|
||
| # Verify spec follows Agent OS standards | ||
| # (Complete validation commands are in .agent-os/standards/best-practices.md) | ||
| ``` | ||
|
|
||
| ## Standards to Follow | ||
| - **Agent OS Standards**: `.agent-os/standards/best-practices.md` | ||
| - **Technology Stack**: `.agent-os/standards/tech-stack.md` | ||
| - **Code Style**: `.agent-os/standards/code-style.md` | ||
|
|
||
| ## Critical Rules for HoneyHive SDK | ||
| 1. **NO MOCKS IN INTEGRATION TESTS** - Integration tests must use real systems | ||
| 2. **All functions must have type hints** and docstrings | ||
| 3. **Minimum 80% test coverage** (project-wide) | ||
| 4. **Use tox for ALL testing** - Never pytest directly | ||
| 5. **Graceful degradation required** - Never crash host app | ||
| 6. **Use EventType enums** - Never string literals in documentation | ||
| 7. **Test count reporting** - Always report total tests correctly (unit + integration) | ||
|
|
||
| ## Common Violations to Prevent | ||
|
|
||
| **❌ WRONG**: | ||
| - Not consulting `.agent-os/standards/best-practices.md` before creating specs | ||
| - Duplicating standards content instead of referencing it | ||
| - Ignoring existing Agent OS specification structure | ||
| - **Task format errors**: Using checkboxes on section headers or wrong checkbox format | ||
|
|
||
| **✅ CORRECT**: | ||
| - **Always reference Agent OS standards first**: Read `.agent-os/standards/best-practices.md` | ||
| - **Follow established patterns**: Use existing specs as templates | ||
| - **Proper task format**: Follow checkbox specifications in `.cursor/rules/execute-tasks.mdc` | ||
| - **Leverage standards system**: Reference, don't duplicate |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Documentation uses string literal instead of EventType enum (Bugbot Rules)
The example code uses a string literal
"llm_call"for theevent_typeparameter, but project rules mandate usingEventTypeenums in all documentation. This violates the critical rule stated in.cursor/rules/execute-tasks.mdc: "EventType enums only - Never string literals in documentation". The correct pattern should importEventTypefromhoneyhive.modelsand useEventType.modelor appropriate enum value instead.