Skip to content

add chart handler #1085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Aug 6, 2025
Merged

add chart handler #1085

merged 15 commits into from
Aug 6, 2025

Conversation

iceljc
Copy link
Collaborator

@iceljc iceljc commented Jul 1, 2025

PR Type

Enhancement


Description

  • Add new ChartHandler plugin for AI chart plotting

  • Replace ProductTemplate with ProgramCodeTemplate message type

  • Update messaging enums and parser for code templates

  • Minor template name refactoring in file handler


Diagram Walkthrough

flowchart LR
  A["User Request"] --> B["ChartHandler Plugin"]
  B --> C["PlotChartFn"]
  C --> D["LLM Processing"]
  D --> E["JavaScript Code Generation"]
  E --> F["ProgramCodeTemplateMessage"]
  F --> G["Chart Rendering"]
Loading

File Walkthrough

Relevant files
Enhancement
14 files
BotSharpMessageParser.cs
Update parser for ProgramCodeTemplate support                       
+8/-4     
RichTypeEnum.cs
Add ProgramCode rich type enum                                                     
+1/-0     
TemplateTypeEnum.cs
Replace Product with ProgramCode template type                     
+1/-1     
ProductTemplateMessage.cs
Remove ProductTemplateMessage class                                           
+0/-19   
ProgramCodeTemplateMessage.cs
Add ProgramCodeTemplateMessage with language support         
+16/-0   
ChartHandlerPlugin.cs
Create main ChartHandler plugin class                                       
+18/-0   
UtilityName.cs
Define ChartPlotter utility name constant                               
+6/-0     
PlotChartFn.cs
Implement chart plotting function with LLM                             
+82/-0   
ChartHandlerUtilityHook.cs
Register chart plotting utility hook                                         
+24/-0   
LlmContextIn.cs
Define input context for plotting requirements                     
+10/-0   
LlmContextOut.cs
Define output context for generated code                                 
+14/-0   
util-chart-plot_chart.fn.liquid
Add chart plotting function template                                         
+1/-0     
util-chart-plot_instruction.liquid
Add detailed chart plotting instructions                                 
+22/-0   
util-chart-plot_chart.json
Define chart plotting function schema                                       
+14/-0   
Miscellaneous
1 files
FileInstructService.SelectFile.cs
Update template name reference                                                     
+1/-1     
Configuration changes
7 files
Using.cs
Add global using statements for plugin                                     
+32/-0   
BotSharp.sln
Add ChartHandler project to solution                                         
+11/-0   
BotSharp.Core.csproj
Update template file references                                                   
+2/-2     
BotSharp.Plugin.ChartHandler.csproj
Create ChartHandler project configuration                               
+30/-0   
BotSharp.Plugin.FileHandler.csproj
Remove unused template file reference                                       
+1/-5     
WebStarter.csproj
Add ChartHandler plugin reference                                               
+2/-1     
appsettings.json
Enable ChartHandler plugin in configuration                           
+1/-0     
Formatting
1 files
LlmContextIn.cs
Remove commented image URL property                                           
+0/-4     
Additional files
1 files
util-file-select_file_instruction.liquid [link]   

Copy link

qodo-merge-pro bot commented Jul 1, 2025

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 Security concerns

Code injection vulnerability:
The GenerateChartFn generates JavaScript code from LLM output and wraps it in a JsCodeTemplateMessage without any validation or sanitization. This JavaScript code could potentially contain malicious scripts that execute in the user's browser, leading to XSS attacks. The LLM-generated code should be validated, sanitized, or executed in a sandboxed environment before being sent to the client.

⚡ Recommended focus areas for review

Hardcoded Model

The LLM model is hardcoded to "gpt-4.1" which may not exist or be available in all environments. This should be configurable or use a fallback mechanism.

var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4.1");
var response = await completion.GetChatCompletions(agent, dialogs);
Security Risk

The function generates JavaScript code from LLM output and returns it directly without validation or sanitization, which could lead to XSS vulnerabilities when executed in the browser.

Message = new JsCodeTemplateMessage
{
    Text = response
}
Limited Scope

The instruction template is hardcoded to only generate pie charts, but the function description suggests it can generate charts "in any format that user requested".

Please take a look at "Requirement" and generate a javascript code that can be used to render a pie chart on a canvas element with id {{ chart_element_id }}.

Copy link

qodo-merge-pro bot commented Jul 1, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix invalid model name

The model name "gpt-4.1" appears to be invalid as OpenAI's GPT-4 models
typically use naming conventions like "gpt-4" or "gpt-4-turbo". This will likely
cause API failures when attempting to generate chart code.

src/Plugins/BotSharp.Plugin.ChartHandler/Functions/GenerateChartFn.cs [69]

-var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4.1");
+var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4");
  • Apply / Chat
Suggestion importance[1-10]: 9

__

Why: The suggestion correctly identifies that "gpt-4.1" is not a standard OpenAI model name, and using it would likely cause the feature to fail at runtime.

High
Add null safety for deserialization

If message.FunctionArgs is null or invalid JSON, the deserialization will fail
and throw an exception. Add null checking and exception handling to prevent
runtime crashes.

src/Plugins/BotSharp.Plugin.ChartHandler/Functions/GenerateChartFn.cs [28-38]

-var args = JsonSerializer.Deserialize<LlmContextIn>(message.FunctionArgs);
+var args = !string.IsNullOrEmpty(message.FunctionArgs) 
+    ? JsonSerializer.Deserialize<LlmContextIn>(message.FunctionArgs) 
+    : null;
 ...
 { "plotting_requirement", args?.PlottingRequirement ?? string.Empty },

[To ensure code accuracy, apply this suggestion manually]

Suggestion importance[1-10]: 7

__

Why: This is a good defensive coding suggestion that prevents a potential ArgumentNullException if message.FunctionArgs is null, improving the code's robustness.

Medium
General
Remove hardcoded chart type restriction
Suggestion Impact:The entire file content was removed, including the hardcoded "pie chart" text that the suggestion aimed to fix

code diff:

-Please take a look at "Requirement" and generate a javascript code that can be used to render a pie chart on a canvas element with id {{ chart_element_id }}.

The instruction hardcodes "pie chart" but the function accepts any plotting
requirement. This limits flexibility and may not match user requests for other
chart types like bar charts or line graphs.

src/Plugins/BotSharp.Plugin.ChartHandler/data/agents/6745151e-6d46-4a02-8de4-1c4f21c7da95/templates/util-chart-plot-instruction.liquid [1]

-Please take a look at "Requirement" and generate a javascript code that can be used to render a pie chart on a canvas element with id {{ chart_element_id }}.
+Please take a look at "Requirement" and generate a javascript code that can be used to render the requested chart on a canvas element with id {{ chart_element_id }}.

[Suggestion processed]

Suggestion importance[1-10]: 8

__

Why: The suggestion correctly points out that hardcoding "pie chart" in the prompt unnecessarily restricts the LLM's output, conflicting with the goal of handling general chart requests.

Medium
  • More

@iceljc iceljc marked this pull request as draft July 1, 2025 04:11
@iceljc iceljc changed the title init add chart handler Jul 1, 2025
@iceljc iceljc marked this pull request as ready for review August 6, 2025 15:21
@iceljc iceljc merged commit 54a90e0 into SciSharp:master Aug 6, 2025
4 checks passed
Copy link

qodo-merge-pro bot commented Aug 6, 2025

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 Security concerns

Code injection:
The ChartHandler generates and executes JavaScript code based on user input without proper sanitization. The LLM-generated JavaScript code in ProgramCodeTemplateMessage could potentially contain malicious scripts that execute in the client browser, leading to XSS vulnerabilities. The plotting_requirement parameter is passed directly to the LLM without validation, allowing users to potentially inject malicious instructions.

⚡ Recommended focus areas for review

Hardcoded Model

The function hardcodes the OpenAI model "gpt-4.1" which may not exist or be available in all environments. This could cause runtime failures and should be configurable.

var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4.1");
var response = await completion.GetChatCompletions(agent, dialogs);
Error Handling

The catch block only logs warnings but returns the error message as chart content, which could expose internal error details to users and may not render properly as JavaScript code.

catch (Exception ex)
{
    var error = $"Error when plotting chart. {ex.Message}";
    _logger.LogWarning(ex, error);
    return error;
}
Missing Imports

The class uses JsonPropertyName attribute but doesn't include the necessary using statement for System.Text.Json.Serialization, which could cause compilation errors.

[JsonPropertyName("rich_type")]
public string RichType => RichTypeEnum.ProgramCode;

[JsonPropertyName("text")]
public string Text { get; set; } = string.Empty;

[JsonPropertyName("template_type")]
public virtual string TemplateType { get; set; } = TemplateTypeEnum.ProgramCode;

[JsonPropertyName("language")]
public string Language { get; set; } = string.Empty;

Copy link

qodo-merge-pro bot commented Aug 6, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix invalid model name

The model name "gpt-4.1" appears to be invalid. OpenAI's GPT-4 models typically
use names like "gpt-4" or "gpt-4-turbo". This will likely cause API calls to
fail with an invalid model error.

src/Plugins/BotSharp.Plugin.ChartHandler/Functions/PlotChartFn.cs [71]

-var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4.1");
+var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4");
  • Apply / Chat
Suggestion importance[1-10]: 9

__

Why: The suggestion correctly identifies that gpt-4.1 is not a valid OpenAI model name, which would cause the new chart plotting feature to fail.

High
Add null safety check

The deserialization could fail if message.FunctionArgs is null or contains
invalid JSON, causing a runtime exception. Add null checking and error handling
to prevent crashes.

src/Plugins/BotSharp.Plugin.ChartHandler/Functions/PlotChartFn.cs [28]

-var args = JsonSerializer.Deserialize<LlmContextIn>(message.FunctionArgs);
+var args = !string.IsNullOrEmpty(message.FunctionArgs) 
+    ? JsonSerializer.Deserialize<LlmContextIn>(message.FunctionArgs) 
+    : new LlmContextIn();
  • Apply / Chat
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly points out a potential ArgumentNullException if message.FunctionArgs is null, and the proposed change makes the deserialization process more robust by handling this case gracefully.

Medium
High-level
Hardcoded LLM provider and model

The PlotChartFn hardcodes "openai" provider and "gpt-4.1" model in the
GetChatCompletion method. This creates tight coupling and prevents configuration
flexibility. Consider using dependency injection or configuration to make the
LLM provider and model configurable.

Examples:

src/Plugins/BotSharp.Plugin.ChartHandler/Functions/PlotChartFn.cs [71]
var completion = CompletionProvider.GetChatCompletion(_services, provider: "openai", model: "gpt-4.1");

Solution Walkthrough:

Before:

private async Task<string> GetChatCompletion(...)
{
    // ...
    var completion = CompletionProvider.GetChatCompletion(_services, 
        provider: "openai", 
        model: "gpt-4.1");
    var response = await completion.GetChatCompletions(agent, dialogs);
    return response.Content;
}

After:

private async Task<string> GetChatCompletion(...)
{
    var settings = _services.GetRequiredService<ChartHandlerSettings>();
    var llmProviderService = _services.GetRequiredService<ILlmProviderService>();
    var completion = CompletionProvider.GetChatCompletion(_services, 
        provider: settings.Provider, 
        model: settings.Model);
    var response = await completion.GetChatCompletions(agent, dialogs);
    return response.Content;
}
Suggestion importance[1-10]: 8

__

Why: The suggestion correctly identifies hardcoded LLM provider and model names in PlotChartFn, which limits flexibility and contradicts the platform's provider-agnostic design.

Medium
General
Make template type readonly

The TemplateType property should be read-only like other template message
classes to maintain consistency. Making it settable could lead to inconsistent
template type values.

src/Infrastructure/BotSharp.Abstraction/Messaging/Models/RichContent/Template/ProgramCodeTemplateMessage.cs [11-12]

 [JsonPropertyName("template_type")]
-public virtual string TemplateType { get; set; } = TemplateTypeEnum.ProgramCode;
+public string TemplateType => TemplateTypeEnum.ProgramCode;
  • Apply / Chat
Suggestion importance[1-10]: 6

__

Why: This is a good suggestion for improving code quality and consistency, as making the TemplateType property read-only prevents accidental modification and aligns with the design of other similar classes in the codebase.

Low
  • More

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant