docs: ai api updates (#8997)

* updates AI API reference

* adds model providers and meta

* fills out env vars

* adds some details

---------

Co-authored-by: Michael Irvine <[email protected]>

Author: mjirv

docs/pages/product/apis-integrations/ai-api.mdx

@@ -138,3 +138,83 @@ One way of handling this is to pass the error message back into the AI API; it m
 #### 3. Continue wait
 
 When using `"runQuery": true`, you might sometimes receive a query result containing `{ "error": "Continue wait" }`. If this happens, you should use `/load` ([described above](#2-load)) instead of `runQuery` to run the query, and handle retries as described in the [REST API documentation](/product/apis-integrations/rest-api#continue-wait).
+
+## Advanced Usage
+
+<InfoBox>
+    The advanced features discussed here are available on Cube version 1.1.7 and above.
+</InfoBox>
+
+### Custom prompts
+
+You can prompt the AI API with custom instructions. For example, you may want it to always
+respond in a particular language, or to refer to itself by a name matching your brand.
+Custom prompts also allow you to give the model more context on your company and data model,
+for example if it should usually prefer a particular view.
+
+To use a custom prompt, set the `CUBE_CLOUD_AI_API_PROMPT` environment variable in your deployment.
+
+<InfoBox>
+  Custom prompts add to, rather than overwrite, the AI API's existing prompting, so you
+  do not need to re-write instructions around how to generate the query itself.
+</InfoBox>
+
+### Meta tags
+
+The AI API can read [meta tags](/reference/data-model/view#meta) on your dimensions, measures, 
+segments, and views.
+
+Use the `ai` meta tag to give context that is specific to AI and goes beyond what is 
+included in the description. This can have any keys that you want. For example, you can use it
+to give the AI context on possible values in a categorical dimension:
+```yaml
+      - name: status
+        sql: status
+        type: string
+        meta:
+          ai:
+            values:
+              - shipped
+              - processing
+              - completed
+```
+
+### Other LLM providers
+
+<InfoBox>
+  These environment variables also apply to the [AI Assistant](/product/workspace/ai-assistant),
+  if it is enabled on your deployment.
+</InfoBox>
+
+If desired, you may "bring your own" LLM model by providing a model and API credentials
+for a supported model provider. Do this by setting environment variables in your Cube
+deployment. See below for required variables by provider (required unless noted):
+
+#### AWS Bedrock
+
+<WarningBox>
+  The AI API currently supports only Anthropic Claude models on AWS Bedrock. Other
+  models may work but are not fully supported.
+</WarningBox>
+
+- `CUBE_BEDROCK_MODEL_ID` - A supported [AWS Bedrock chat model](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html), for example `anthropic.claude-3-5-sonnet-20241022-v2:0`
+- `CUBE_BEDROCK_ACCESS_KEY` - An access key for an IAM user with `InvokeModelWithResponseStream` permissions on the desired region/model.
+- `CUBE_BEDROCK_ACCESS_SECRET` - The corresponding access secret
+- `CUBE_BEDROCK_REGION_ID` - A supported AWS Bedrock region, for example `us-west-2`
+
+#### GCP Vertex
+
+<WarningBox>
+  The AI API currently supports only Anthropic Claude models on GCP Vertex. Other
+  models may work but are not fully supported.
+</WarningBox>
+
+- `CUBE_VERTEX_MODEL_ID` - A supported GCP Vertex chat model, for example `claude-3-5-sonnet@20240620`
+- `CUBE_VERTEX_PROJECT_ID` - The GCP project the model is deployed in
+- `CUBE_VERTEX_REGION` - The GCP region the model is deployed in, for example `us-east5`
+- `CUBE_VERTEX_CREDENTIALS` - The private key for a service account with permissions to run the chosen model
+
+#### OpenAI
+
+- `OPENAI_MODEL` - An OpenAI chat model ID, for example `gpt-4o`
+- `OPENAI_API_KEY` - An OpenAI API key (we recommend creating a service account for the AI API)

docs/pages/product/workspace/ai-assistant.mdx

@@ -58,6 +58,49 @@ The query will automatically run in the sidebar and can be opened in the [Playgr
 
 <Screenshot src="https://ucarecdn.com/4249ff1e-fae1-42c8-ad3a-b9e406ea2022/Screenshot20240624at34327PM.png" />
 
+## Advanced Usage
+
+<InfoBox>
+    The advanced features discussed here are available on Cube version 1.1.7 and above.
+</InfoBox>
+
+### Custom prompts
+
+You can prompt the AI Assistant with custom instructions. For example, you may want it to always
+respond in a particular language, or to refer to itself by a name matching your brand.
+Custom prompts also allow you to give the model more context on your company and data model,
+for example if it should usually prefer a particular view.
+
+To use a custom prompt, set the `CUBE_CLOUD_AI_ASSISTANT_PROMPT` environment variable in your deployment.
+
+<InfoBox>
+  Custom prompts add to, rather than overwrite, the AI Assistant's existing prompting.
+</InfoBox>
+
+### Meta tags
+
+The AI Assistant can read [meta tags](/reference/data-model/view#meta) on your dimensions, measures, 
+segments, and views.
+
+Use the `ai` meta tag to give context that is specific to AI and goes beyond what is 
+included in the description. This can have any keys that you want. For example, you can use it
+to give the AI context on possible values in a categorical dimension:
+```yaml
+      - name: status
+        sql: status
+        type: string
+        meta:
+          ai:
+            values:
+              - shipped
+              - processing
+              - completed
+```
+
+### Other LLM providers
+
+See the [AI API's documentation][ref-ai-api-providers] for information on how to "bring your own" LLM. 
+
 ## FAQ and limitations
 
 ### 1. What language model(s) does the AI Assistant use?
@@ -83,3 +126,4 @@ The query will automatically run in the sidebar and can be opened in the [Playgr
 [ref-catalog]: /product/workspace/semantic-catalog
 [ref-playground]: /product/workspace/playground
 [ref-catalog-downstream]: /product/workspace/semantic-catalog#connecting-downstream-tools
+[ref-ai-api-providers]: /product/apis-integrations/ai-api#other-llm-providers

docs/pages/reference/ai-api.mdx

@@ -11,11 +11,21 @@ Generate a Cube query that can be used to answer a user's question, and (optiona
 | `messages` | ✅ Yes   | An array of messages in the format: `{ "role": "user" \| "assistant", "content": "string" }`    |
 | `views`    |          | An array of view names (used to limit the views that the AI API can use to generate its answer) |
 | `runQuery` |          | Boolean (true or false) whether to run the query and return its results                         |
+| `options`  |          | An object in the format `{ "chart": true \| false }`
 
 Response
 
 - `message` - A message from the AI assistant describing the query, how it was chosen, why it could not generate the requested query, etc.
 - `cube_query` - A Cube [Query](/product/apis-integrations/rest-api/query-format) that could be used to answer the given question
+- `chart` - If the `chart` option is set to `true`, an object containing a chart spec for the generated query in the following format:
+    ```json
+    {
+        "type": "bar" | "line" | "pie" | "table" | "area" | "scatter",
+        "x": string,
+        "y": string[],
+        "pivot": string // optional; the field to pivot by, if any
+    }
+    ```
 
 ### Examples
 
@@ -28,7 +38,7 @@ curl \
  -X POST  \
  -H "Content-Type: application/json" \
  -H "Authorization: EXAMPLE-API-TOKEN" \
- --data '{ "messages": [{ "role": "user", "content": "What cities have the highest aov this year?" }]}' \
+ --data '{ "messages": [{ "role": "user", "content": "What cities have the highest aov this year?", "views": ["orders_view"] }]}' \
  https://YOUR_CUBE_API/cubejs-api/v1/ai/query/completions
 ```