Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .vscode/cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -1071,7 +1071,10 @@
"percentilesw",
"plotly",
"nbformat",
"nbconvert"
"nbconvert",
"rollupby",
"milli",
"resourceid"
]
},
{
Expand Down
13 changes: 13 additions & 0 deletions sdk/monitor/azure-monitor-querymetrics/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Release History

## 1.0.0 (Unreleased)

### Features Added

- Initial release. This version includes the core functionality for querying metrics from Azure Monitor using the `MetricsClient`, which was originally introduced in the `azure-monitor-query` package.

### Breaking Changes

### Bugs Fixed

### Other Changes
21 changes: 21 additions & 0 deletions sdk/monitor/azure-monitor-querymetrics/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Copyright (c) Microsoft Corporation.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
7 changes: 7 additions & 0 deletions sdk/monitor/azure-monitor-querymetrics/MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
include *.md
include LICENSE
include azure/monitor/querymetrics/py.typed
recursive-include tests *.py
recursive-include samples *.py *.md
include azure/__init__.py
include azure/monitor/__init__.py
250 changes: 250 additions & 0 deletions sdk/monitor/azure-monitor-querymetrics/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
# Azure Monitor Query Metrics client library for Python

The Azure Monitor Query Metrics client library is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s metrics data platform:

- [Metrics](https://learn.microsoft.com/azure/azure-monitor/essentials/data-platform-metrics) - Collects numeric data from monitored resources into a time series database. Metrics are numerical values that are collected at regular intervals and describe some aspect of a system at a particular time. Metrics are lightweight and capable of supporting near real-time scenarios, making them useful for alerting and fast detection of issues.

**Resources:**

<!-- TODO: Add PyPI, Conda, Ref Docs, Samples links-->
- [Source code][source]
- [Service documentation][azure_monitor_overview]
- [Change log][changelog]

## Getting started

### Prerequisites

- Python 3.9 or later
- An [Azure subscription][azure_subscription]
- Authorization to read metrics data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription containing the resources to be queried.
- An Azure resource of any kind (Storage Account, Key Vault, Cosmos DB, etc.).

### Install the package

Install the Azure Monitor Query Metrics client library for Python with [pip][pip]:

```bash
pip install azure-monitor-querymetrics
```

### Create the client

An authenticated client is required to query Metrics. The library includes both synchronous and asynchronous forms of the client. To authenticate, create an instance of a token credential. Use that instance when creating a `MetricsClient`. The following examples use `DefaultAzureCredential` from the [azure-identity](https://pypi.org/project/azure-identity/) package.

#### Synchronous client

Consider the following example, which creates a synchronous client for Metrics querying:

```python
from azure.identity import DefaultAzureCredential
from azure.monitor.querymetrics import MetricsClient

credential = DefaultAzureCredential()
metrics_client = MetricsClient("https://<regional endpoint>", credential)
```

#### Asynchronous client

The asynchronous form of the client API is found in the `.aio`-suffixed namespace. For example:

```python
from azure.identity.aio import DefaultAzureCredential
from azure.monitor.querymetrics.aio import MetricsClient

credential = DefaultAzureCredential()
async_metrics_client = MetricsClient("https://<regional endpoint>", credential)
```

To use the asynchronous clients, you must also install an async transport, such as [aiohttp](https://pypi.org/project/aiohttp/).

```sh
pip install aiohttp
```

#### Configure client for Azure sovereign cloud

By default, the client is configured to use the Azure public cloud. To use a sovereign cloud, provide the correct `audience` argument when creating the `MetricsClient`. For example:

```python
from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
from azure.monitor.querymetrics import MetricsClient

# Authority can also be set via the AZURE_AUTHORITY_HOST environment variable.
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)

metrics_client = MetricsClient(
"https://usgovvirginia.metrics.monitor.azure.us", credential, audience="https://metrics.monitor.azure.us"
)
```

### Execute the query

For examples of Metrics queries, see the [Examples](#examples) section.

## Key concepts

### Metrics data structure

Each set of metric values is a time series with the following characteristics:

- The time the value was collected
- The resource associated with the value
- A namespace that acts like a category for the metric
- A metric name
- The value itself
- Some metrics have multiple dimensions as described in multi-dimensional metrics.

## Examples

- [Metrics query](#metrics-query)
- [Handle metrics query response](#handle-metrics-query-response)

### Metrics query

To query metrics for one or more Azure resources, use the `query_resources` method of `MetricsClient`. This method requires a regional endpoint when creating the client. For example, "https://westus3.metrics.monitor.azure.com".

Each Azure resource must reside in:

- The same region as the endpoint specified when creating the client.
- The same Azure subscription.

The resource IDs must be that of the resources for which metrics are being queried. It's normally of the format `/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/topics/<resource-name>`.

To find the resource ID/URI:

1. Navigate to your resource's page in the Azure portal.
1. Select the **JSON View** link in the **Overview** section.
1. Copy the value in the **Resource ID** text box at the top of the JSON view.

Furthermore:

- The user must be authorized to read monitoring data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription to be queried.
- The metric namespace containing the metrics to be queried must be provided. For a list of metric namespaces, see [Supported metrics and log categories by resource type][metric_namespaces].

```python
from datetime import timedelta
import os

from azure.core.exceptions import HttpResponseError
from azure.identity import DefaultAzureCredential
from azure.monitor.querymetrics import MetricsClient, MetricAggregationType

endpoint = "https://westus3.metrics.monitor.azure.com"
credential = DefaultAzureCredential()
client = MetricsClient(endpoint, credential)

resource_ids = [
"/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/storageAccounts/<resource-name-1>",
"/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/storageAccounts/<resource-name-2>"
]

response = client.query_resources(
resource_ids=resource_ids,
metric_namespace="Microsoft.Storage/storageAccounts",
metric_names=["UsedCapacity"],
timespan=timedelta(hours=2),
granularity=timedelta(minutes=5),
aggregations=[MetricAggregationType.AVERAGE],
)

for metrics_query_result in response:
for metric in metrics_query_result.metrics:
print(f"Metric: {metric.name}")
for time_series in metric.timeseries:
for metric_value in time_series.data:
if metric_value.average is not None:
print(f"Average: {metric_value.average}")
```

#### Handle metrics query response

The metrics query API returns a list of `MetricsQueryResult` objects. The `MetricsQueryResult` object contains properties such as a list of `Metric`-typed objects, `granularity`, `namespace`, and `timespan`. The `Metric` objects list can be accessed using the `metrics` param. Each `Metric` object in this list contains a list of `TimeSeriesElement` objects. Each `TimeSeriesElement` object contains `data` and `metadata_values` properties. In visual form, the object hierarchy of the response resembles the following structure:

```
MetricsQueryResult
|---granularity
|---timespan
|---cost
|---namespace
|---resource_region
|---metrics (list of `Metric` objects)
|---id
|---type
|---name
|---unit
|---timeseries (list of `TimeSeriesElement` objects)
|---metadata_values
|---data (list of data points)
```

**Note:** Each `MetricsQueryResult` is returned in the same order as the corresponding resource in the `resource_ids` parameter. If multiple different metrics are queried, the metrics are returned in the order of the `metric_names` sent.

**Example of handling response**

```python
import os
from azure.monitor.querymetrics import MetricsClient, MetricAggregationType
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
client = MetricsClient("https://<regional endpoint>", credential)

metrics_uri = os.environ['METRICS_RESOURCE_URI']
response = client.query_resource(
metrics_uri,
metric_names=["PublishSuccessCount"],
aggregations=[MetricAggregationType.AVERAGE]
)

for metrics_query_result in response:
for metric in metrics_query_result.metrics:
print(f"Metric: {metric.name}")
for time_series in metric.timeseries:
for metric_value in time_series.data:
if metric_value.average is not None:
print(f"Average: {metric_value.average}")
```

## Troubleshooting

See our [troubleshooting guide][troubleshooting_guide] for details on how to diagnose various failure scenarios.

## Next steps

To learn more about Azure Monitor, see the [Azure Monitor service documentation][azure_monitor_overview].

### Samples

The following code samples show common scenarios with the Azure Monitor Query Metrics client library.

#### Metrics query samples

To be added.

## Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [[email protected]][coc_contact] with any additional questions or comments.

<!-- LINKS -->

[azure_core_exceptions]: https://aka.ms/azsdk/python/core/docs#module-azure.core.exceptions
[azure_core_ref_docs]: https://aka.ms/azsdk/python/core/docs
[azure_monitor_overview]: https://learn.microsoft.com/azure/azure-monitor/
[azure_subscription]: https://azure.microsoft.com/free/python/
[changelog]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics/CHANGELOG.md
[metric_namespaces]: https://learn.microsoft.com/azure/azure-monitor/reference/supported-metrics/metrics-index#supported-metrics-and-log-categories-by-resource-type
[pip]: https://pypi.org/project/pip/
[python_logging]: https://docs.python.org/3/library/logging.html
[samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics/samples
[source]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-querymetrics/
[troubleshooting_guide]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-querymetrics/TROUBLESHOOTING.md

[cla]: https://cla.microsoft.com
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
[coc_contact]: mailto:[email protected]
89 changes: 89 additions & 0 deletions sdk/monitor/azure-monitor-querymetrics/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Troubleshooting Azure Monitor Query Metrics client library issues

This troubleshooting guide contains instructions to diagnose frequently encountered issues while using the Azure Monitor Query Metrics client library for Python.

## Table of contents

* [General Troubleshooting](#general-troubleshooting)
* [Enable client logging](#enable-client-logging)
* [Troubleshooting authentication issues with metrics query requests](#authentication-errors)
* [Troubleshooting running async APIs](#errors-with-running-async-apis)
* [Troubleshooting Metrics Query](#troubleshooting-metrics-query)
* [Troubleshooting insufficient access error](#troubleshooting-insufficient-access-error-for-metrics-query)
* [Troubleshooting unsupported granularity for metrics query](#troubleshooting-unsupported-granularity-for-metrics-query)
* [Additional azure-core configurations](#additional-azure-core-configurations)


## General Troubleshooting

Monitor query raises exceptions described in [`azure-core`](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md)

### Enable client logging

To troubleshoot issues with Azure Monitor Query Metrics library, it is important to first enable logging to monitor the behavior of the application. The errors and warnings in the logs generally provide useful insights into what went wrong and sometimes include corrective actions to fix issues.

This library uses the standard [logging](https://docs.python.org/3/library/logging.html) library for logging. Basic information about HTTP sessions, such as URLs and headers, is logged at the INFO level.
Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with the logging_enable argument:

```python
import logging
from azure.monitor.querymetrics import MetricsClient

# Create a logger for the 'azure.monitor.querymetrics' SDK
logger = logging.getLogger('azure.monitor.querymetrics')
logger.setLevel(logging.DEBUG)

# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)

client = MetricsClient(credential, logging_enable=True)
```

Similarly, logging_enable can enable detailed logging for a single operation, even when it isn't enabled for the client:

```python
client.query_workspace(logging_enable=True)
```

### Authentication errors

Azure Monitor Query Metrics supports Azure Active Directory authentication. The MetricsClient has methods to set the `credential`. To provide a valid credential, you can use
`azure-identity` dependency. For more details on getting started, refer to
the [README](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics#create-the-client)
of Azure Monitor Query Metrics library. You can also refer to
the [Azure Identity documentation](https://learn.microsoft.com/python/api/overview/azure/identity-readme)
for more details on the various types of credential supported in `azure-identity`.

For more help on troubleshooting authentication errors please see the Azure Identity client library [troubleshooting guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/TROUBLESHOOTING.md).

### Errors with running async APIs

The async transport is designed to be opt-in. [AioHttp](https://pypi.org/project/aiohttp/) is one of the supported implementations of async transport. It is not installed by default. You need to install it separately as follows:

```
pip install aiohttp
```

## Troubleshooting Metrics Query

### Troubleshooting insufficient access error for metrics query

If you encounter 401 authorization errors while querying metrics using `MetricsClient`, please ensure you are authorized to read monitoring data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription to be queried.

### Troubleshooting unsupported granularity for metrics query

If you notice the following exception, this is due to an invalid time granularity in the metrics query request. Your
query might have set the `granularity` keyword argument to an unsupported duration.

```text
"{"code":"BadRequest","message":"Invalid time grain duration: PT10M, supported ones are: 00:01:00,00:05:00,00:15:00,00:30:00,01:00:00,06:00:00,12:00:00,1.00:00:00"}"
```

As documented in the error message, the supported granularity for metrics queries are 1 minute, 5 minutes, 15 minutes,
30 minutes, 1 hour, 6 hours, 12 hours and 1 day.

## Additional azure-core configurations

When calling the methods, some properties including `retry_mode`, `timeout`, `connection_verify` can be configured by passing in as keyword arguments. See
[configurations](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md#configurations) for list of all such properties.
Loading
Loading