⚠️ Disclaimer: The iOS code in this project was built with the assistance of Kiro, an agentic AI IDE, to bridge a gap in Swift/iOS expertise. This is a demo project — it is not intended for production use.
Hands-free AI assistant for Meta Ray-Ban smart glasses powered by Amazon Bedrock AgentCore and Strands Agents. Say a wake word, ask anything — the agent responds through the glasses speakers.
Voice-controlled AI agent that runs on Meta Ray-Ban glasses. The agent uses wake word detection, processes natural language queries, and speaks responses directly through the glasses speakers — no need to touch the phone.
The agent has access to web search, IMDb ratings, GitHub repository search, notes (Obsidian), and math. Supports Amazon Bedrock, Anthropic, and OpenAI as model providers. Users authenticate via Amazon Cognito — the API is fully protected, no API keys stored on the device.
The agent uses two layers of memory following STM/LTM principles:
Short-Term Memory (STM) — conversation context within a session, powered by Amazon Bedrock AgentCore Runtime isolated sessions. A new session starts each time the glasses connect; all messages in that connection share context.
Long-Term Memory (LTM) — user facts and preferences that persist across all sessions, powered by AgentCore Memory. The agent learns your name, location, and preferences over time without any explicit action from you.
This app uses Strands Agents, which makes it simple to extend the agent with new capabilities — just add a @tool function in Python. It comes with 10 built-in tools:
📓 Obsidian integration note: This demo saves ideas directly to an Obsidian vault stored in Amazon S3. The vault is synced to the Obsidian desktop/mobile app using the Remotely Save community plugin, which supports Amazon S3 as a backend. The agent writes structured Markdown notes to S3 — when you open Obsidian, the new notes appear automatically.
| Tool | What it does |
|---|---|
tavily |
Web search — current events, news, facts, any internet query |
search_imdb |
Movie and TV show ratings, cast, director, plot from IMDb |
search_github_repos |
Find GitHub repositories by topic, language, or keyword |
search_github_code |
Find code examples on GitHub |
save_to_obsidian |
Save ideas as structured Markdown notes to an S3-backed Obsidian vault |
calculator |
Math and unit conversions |
current_time |
Current date and time |
think |
Complex reasoning before answering |
http_request |
Call public APIs directly |
browser |
Navigate dynamic websites when needed |
- Authentication: Amazon Cognito User Pool — users sign up with email, verify with code, and authenticate via JWT tokens
- Token storage: iOS Keychain — never UserDefaults
- API secrets: AWS Systems Manager (SSM) Parameter Store SecureString — never in CloudFormation or code
- Session: RefreshToken valid for 10 years — session persists across app restarts and phone locks
- API: Protected by Amazon Cognito authorizer — no request reaches the backend without a valid JWT
| Folder | Description |
|---|---|
ray-ban-voice-agent-bedrock/backend/ |
AWS Cloud Development Kit (CDK) stack — Amazon API Gateway, AWS Lambda, AgentCore Runtime, Amazon Cognito, Memory |
ray-ban-voice-agent-bedrock/ios/ |
SwiftUI iOS app — voice commands, wake word, Cognito auth |
ray-ban-voice-agent-bedrock/backend/agent_files/ |
Strands agent with tools |
ray-ban-voice-agent-bedrock/update_ios_config.py |
One-command deploy and iOS config update |
The agent supports three providers. Priority: Anthropic → OpenAI → Amazon Bedrock (default).
| Provider | How to activate | Default model |
|---|---|---|
| Amazon Bedrock | Default, no extra config | anthropic.claude-3-haiku-20240307-v1:0 — change with -c model_id=... |
| Anthropic | Pass anthropic_api_key |
claude-opus-4-6 |
| OpenAI | Pass openai_api_key |
gpt-4o |
To switch providers without redeploying:
source ray-ban-voice-agent-bedrock/backend/.venv/bin/activate
# Use Anthropic
python ray-ban-voice-agent-bedrock/update_ios_config.py --skip-deploy -c anthropic_api_key="sk-ant-..."
# Use OpenAI
python ray-ban-voice-agent-bedrock/update_ios_config.py --skip-deploy -c openai_api_key="sk-..."
# Back to Bedrock: remove the key from AWS SSM Parameter Store consoleSecrets are stored as SSM Parameter Store SecureString — never in CloudFormation or code.
The setup requires two deployments:
First deploy — without iOS-specific values (get the outputs first):
cd ray-ban-voice-agent-bedrock/backend
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cd ..
source backend/.venv/bin/activate
python update_ios_config.py \
-c tavily_api_key="tvly-..." \
-c anthropic_api_key="sk-ant-..." # or openai_api_key, omit for BedrockAfter the first deploy, note the outputs:
UniversalLinkDomain— needed for Meta Developer Center registration (see full guide)
Second deploy — after registering on Meta Developer Center, add iOS values:
python update_ios_config.py \
-c team_id="YOUR_APPLE_TEAM_ID" \
-c bundle_id="com.example.YourApp" \
-c obsidian_bucket="your-s3-bucket" \
-c tavily_api_key="tvly-..." \
-c anthropic_api_key="sk-ant-..."This updates the Lambda with Universal Link config and refreshes AppConfig.swift with the Cognito values needed by the iOS app.
Full setup guide: ray-ban-voice-agent-bedrock/README.md
OpenClaw is one of the most popular open-source AI agents right now — it runs autonomously, connects to the messaging apps you already use (WhatsApp, Telegram, Slack, Signal, Discord), and comes with 56+ built-in skills for web browsing, file management, email, calendar, shell commands, and more.
Since this project runs on Amazon Bedrock AgentCore, you can host your own OpenClaw instance on the same infrastructure. Use this AWS sample repo to deploy OpenClaw on AgentCore Runtime — each user gets their own isolated container with persistent workspace storage:
👉 sample-host-openclaw-on-amazon-bedrock-agentcore
This gives your Ray-Ban glasses access to the full OpenClaw skill ecosystem on top of the tools already built into this agent.
If this project helped you build something awesome, please consider starring the repo — it helps others discover it too!
Contributions are welcome! See CONTRIBUTING for more information.
If you discover a potential security issue in this project, notify AWS/Amazon Security via the vulnerability reporting page. Please do not create a public GitHub issue.
This library is licensed under the MIT-0 License. See the LICENSE file for details.