Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Agent_Based_Samples/customer_assist/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ The datasets ([dataset-1](src/solution_accelerators/customer_assist/data) and [d

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

# Responsible AI Transparency FAQ
Please refer to [Transparency FAQ](TRANSPARENCY-FAQ.md) for responsible AI transparency details of this solution accelerator.

---

<div align="center">
Expand Down
75 changes: 75 additions & 0 deletions Agent_Based_Samples/customer_assist/TRANSPARENCY-FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Customer Assist Solution Accelerator: Responsible AI FAQ

## What is Customer Assist Solution Accelerator?

The Customer Assist Solution Accelerator (also called "Customer Assist") is an enterprise-grade AI-powered solution that demonstrates how intelligent agent orchestration can transform customer service operations. It leverages the Microsoft Semantic Kernel Process Framework and Azure AI Foundry to empower customer service representatives with real-time insights, contextual assistance, and automated workflows.

## What can the Customer Assist Solution Accelerator do?

The solution provides several key capabilities:

### Core Features:

- **Multi-Agent Orchestration**: Codifies business processes using specialized agents (Conversation Processor, Sentiment Analysis, and Post-call Analysis agents) orchestrated with Microsoft Semantic Kernel
- **Multi-Modal Support**: Handles text, images, audio, and documents in real-time with content understanding and structured insight extraction
- **Flexible AI Integration**: Supports any language model ("Bring Your Own LLM") with task allocation to specialized models for cost and performance optimization
- **Real-time Processing**: Provides live sentiment analysis, document verification, and policy retrieval
- **Agent Evaluation & Observability**: Monitors agent behavior and system health with usage, performance, and quality analytics

### Business Capabilities:

- **Enhanced Customer Experience**: Real-time sentiment analysis and personalized interactions
- **Improved Operational Efficiency**: Automated document verification and policy retrieval
- **Reduced Training Requirements**: Enables new representatives to operate with expert-level knowledge from day one
- **Increased First-Contact Resolution**: Addresses customer needs faster and more effectively

## What is Customer Assist Solution Accelerator intended use(s)?

The solution is designed for multiple industry verticals:

- **Banking & Financial Services**: Streamline loan processing, fraud resolution, and financial product guidance
- **Healthcare & Insurance**: Automate onboarding, claims assistance, and ensure compliance during patient/member interactions
- **Retail & eCommerce**: Enhance product discovery, resolve order issues, and drive upsell opportunities
- **Telecom & Utilities**: Guide agents through technical troubleshooting, plan recommendations, and outage communication
- **Travel & Hospitality**: Support booking changes, handle complaints sensitively, and personalize loyalty program engagement

## How was Customer Assist Accelerator evaluated? What metrics are used to measure performance?

The solution incorporates a comprehensive evaluation framework powered by the Azure AI Evaluation SDK, enabling performance evaluations across agents.

### Evaluation Metrics:

- **Intent Resolution**: Measures the agent's ability to accurately identify and scope user intent. Scale: 1–5 (higher is better)
- **Coherence**: Measures how well the language model can produce output that flows smoothly, reads naturally, and resembles human-like language. Scale: 1–5 (higher is better)

We've applied this metric to evaluate the Assist Agent, verifying whether the agent correctly interprets and acts on user intent, and produces relevant & coherent responses.

For safety evaluations, we've used the Azure AI Foundry Red Teaming Agent, which simulates a variety of attack strategies and complexity levels to test the system end-to-end.

### Key safeguards in place:

- **Text and Image Moderators**: All user inputs are first screened by dedicated moderation layers leveraging Azure AI Content Safety Service.
- **Azure OpenAI Content Filters**: Every agent uses Azure OpenAI service with content filters enabled at the model deployment level.

## What are the limitations of Customer Assist Solution Accelerator?

### Performance Dependencies:

- Azure service throttling
- Network latency requirements

## Sensitive Use Guidance

We encourage users to incorporate the below considerations when leveraging the workflow for the scenarios below.

**Banking & Financial Services**: AI-generated financial guidance should include disclaimers and source attribution to the extent possible. Users should be able to trace advice to validated financial documents or policies. Confidence scores for fraud detection or loan eligibility should be interpretable, with thresholds triggering mandatory human review when confidence is low. Agent suggestions should be explainable, with links to relevant regulations or internal policies.

**Healthcare & Insurance**: Claims assistance and onboarding should clearly indicate machine assistance, with all outputs reviewed by authorized personnel. Sensitive health information must be processed under strict data minimization principles, with access controls and retention policies enforced. AI-generated summaries should preserve the original message and tone, especially when dealing with urgent or emotionally charged content.

**Retail & eCommerce**: Product recommendations and upsell prompts should be explainable and grounded in catalog data. AI-generated responses to order issues must be labeled and editable by human customer assistant agents. Observability metrics should track ungroundness rates and user satisfaction to ensure transparency in customer interactions.

**Telecom & Utilities**: Troubleshooting flows should include transparency on decision logic. Plan recommendations should link to terms and conditions and be accompanied by a "Why was this suggested?" feature. All AI-generated actions should be logged and auditable to support regulatory compliance.

**Travel & Hospitality**: Complaint handling and loyalty engagement should be sensitive to tone and context. AI-generated responses should be labeled and editable, with clear indicators of machine assistance. Booking changes and itinerary suggestions should be traceable to source data and reviewed by human agents before finalization.

For key use cases that highlight customer-impacting actions, we emphasize making outputs available to human stakeholders so that they can be reviewed regularly.
7 changes: 0 additions & 7 deletions Agent_Based_Samples/release_manager/scripts/createEnv.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,6 @@ if (-Not (Test-Path -Path "$Path/.venv/pip.ini")) {

# now create the pip.ini file
Write-Host "Creating pip.ini file in directory $Path/.venv and set up the index-url"
$pipIniContent = @"
[global]
index-url=https://pkgs.dev.azure.com/AIP-CXE-Eng/Engineering/_packaging/CustomerEng/pypi/simple/
"@

$pipIniPath = Join-Path -Path "$Path/.venv" -ChildPath "pip.ini"
$pipIniContent | Out-File -FilePath $pipIniPath -Encoding UTF8

Write-Host "Python virtual environment setup completed successfully."

Expand Down
7 changes: 0 additions & 7 deletions Agent_Based_Samples/sales_analyst/scripts/createEnv.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,6 @@ if (-Not (Test-Path -Path "$Path/.venv/pip.ini")) {

# now create the pip.ini file
Write-Host "Creating pip.ini file in directory $Path/.venv and set up the index-url"
$pipIniContent = @"
[global]
index-url=https://pkgs.dev.azure.com/AIP-CXE-Eng/Engineering/_packaging/CustomerEng/pypi/simple/
"@

$pipIniPath = Join-Path -Path "$Path/.venv" -ChildPath "pip.ini"
$pipIniContent | Out-File -FilePath $pipIniPath -Encoding UTF8

Write-Host "Python virtual environment setup completed successfully."

Expand Down