[DRAFT] Implement Vision Capabilities for Azure OpenAI Integration

[DmitryKatson]

Summary

This PR adds support for vision-enabled AI models to our Azure OpenAI integration, allowing Business Central to send both text and images to AI models and receive detailed responses about image content.

Technical Implementation

We've extended the AOAI Chat Messages codeunit with methods to handle various image sources:

• Image URLs
• Image streams (InStream)
• TenantMedia records
• Temp Blobs
• MediaSet fields (supporting multiple images)

All methods convert the images to the format required by the Azure OpenAI API - either direct URLs or base64-encoded data URLs with appropriate MIME types.

Key Features

• Support for all common image formats (PNG, JPEG, GIF, WEBP)
• Detail level parameter to control image analysis depth (auto, low, high)
• Multiple images support in a single request (with 10-image limit enforced)
• Automatic handling of MediaSet fields with multiple images
• Proper error handling for unsupported file types and empty images

Usage Examples

Example 1: Using an image URL

var
    AOAIChatMessages: Codeunit "AOAI Chat Messages";
    UserText: Text;
    ImageUrl: Text;
begin
    UserText := 'What can you tell me about this product?';
    ImageUrl := 'https://example.com/product-image.jpg';
    
    // Add a message with both text and image URL
    AOAIChatMessages.AddUserMessage(UserText, ImageUrl, Enum::"AOAI Image Detail Level"::high);
    
    // Send to AI model and process response...
end;

Example 2: Using an image from a Record MediaSet field

var
    AOAIChatMessages: Codeunit "AOAI Chat Messages";
    Item: Record Item;
    MediaSetId: Guid;
    UserText: Text;
begin
    Item.Get('ITEM0001');
    MediaSetId := Item.Picture.MediaId;
    UserText := 'Analyze this product image and suggest improvements:';
    
    // Add a message with text and image(s) from MediaSet
    AOAIChatMessages.AddUserMessage(UserText, MediaSetId, Enum::"AOAI Image Detail Level"::auto);
    
    // Send to AI model and process response...
end;

Example 3: Using an image stream (e.g., from file upload)

var
    AOAIChatMessages: Codeunit "AOAI Chat Messages";
    FileManagement: Codeunit "File Management";
    ImageStream: InStream;
    UserText: Text;
    FileName: Text;
    FileExtension: Text;
begin
    UploadImageFile();
    UserText := 'What does this image show?';
    
    // Add a message with text and image from stream
    AOAIChatMessages.AddUserMessage(UserText, ImageStream, FileExtension, Enum::"AOAI Image Detail Level"::low);
    
    // Send to AI model and process response...
end;

local procedure UploadImageFile()
    var
        ImportImageLbl: Label 'Import Image';
        ImageFileTypeFilterLbl: Label 'Image Files (*.jpg;*.jpeg;*.png)|*.jpg;*.jpeg;*.png';
        FileManagement: Codeunit "File Management";
    begin
        if UploadIntoStream(ImportImageLbl, '', ImageFileTypeFilterLbl, FileName, ImageStream) then
            FileExtension := FileManagement.GetExtension(FileName);
    end;

Example 4: Using multiple images from a MediaSet

var
    AOAIChatMessages: Codeunit "AOAI Chat Messages";
    SalesHeader: Record "Sales Header";
    MediaSetId: Guid;
    UserText: Text;
begin
    SalesHeader.Get(SalesHeader."Document Type"::Order, '1001');
    MediaSetId := SalesHeader.Attachments.MediaId;
    UserText := 'Compare these product images and tell me the differences:';
    
    // Add a message with text and multiple images from MediaSet
    AOAIChatMessages.AddUserMessage(UserText, MediaSetId, Enum::"AOAI Image Detail Level"::high);
    
    // The API will process up to 10 images maximum
    // Send to AI model and process response...
end;

Testing

Comprehensive test coverage has been added with 11 test cases that validate:

• Proper JSON message structure for all input types
• URL and data URL encoding
• MediaSet handling with multiple images
• 10-image limit enforcement
• Error handling for unsupported file types
• End-to-end communication flow

Notes

• This implementation requires access to vision-enabled models like GPT-4o, GPT-4o-mini
• When using high detail level, token consumption will increase significantly

Work Item(s)

Fixes #3633

Fixes AB#578341

[DmitryKatson]

Guys, @JesperSchulz any traction here?

[JesperSchulz]

Guys, @JesperSchulz any traction here?

Not really, no. This needs product management resources, which we don't have at the moment ☹️ @pborring accepted to be the PM on this slice, but I don't think he's had much opportunity to drive this forward. Let me check what the latest status is, and whether we think it's realistic to tackle this for 2026 wave 1.

[DmitryKatson]

Guys, @JesperSchulz any traction here?

Not really, no. This needs product management resources, which we don't have at the moment ☹️ @pborring accepted to be the PM on this slice, but I don't think he's had much opportunity to drive this forward. Let me check what the latest status is, and whether we think it's realistic to tackle this for 2026 wave 1.

Very sad:( tbh i don't understand what's the issue with that, as this is a wrapper around official Azure OpenAI API. Many partners need that. Let's make a bit of magic and include this in this fall wave ;)

[JesperSchulz]

Guys, @JesperSchulz any traction here?

Not really, no. This needs product management resources, which we don't have at the moment ☹️ @pborring accepted to be the PM on this slice, but I don't think he's had much opportunity to drive this forward. Let me check what the latest status is, and whether we think it's realistic to tackle this for 2026 wave 1.

Very sad:( tbh i don't understand what's the issue with that, as this is a wrapper around official Azure OpenAI API. Many partners need that. Let's make a bit of magic and include this in this fall wave ;)

2025 wave 2 is completely out of the question, but we could easily release in a minor release, as soon as 2025 wave 2 update 1 in November (not sure if that is realistic either though). I get that many partners need this - we'd like it too. But we need to build things properly.

And I understand how you don't see the issue, as we haven't shared them with you. But there are a lot of things which must fall into place before we can move forward with this. Maybe @pborring wants to elaborate 🙂 We can also have a call soon, where I can explain. Let's touch basis on this one soon!

[bc4all]

@JesperSchulz, @pborring is there an option that we will come back to this PR? Or this is already solved? Would it be great to be able to handle attachments.