Skip to content

Commit f9c9fe8

Browse files
pvaneckpvaneck
committed
[Monitor] Add Metrics Query library
This adds the TypeSpec-based azure-monitor-querymetrics package. Signed-off-by: Paul Van Eck <[email protected]>
1 parent 3e01d23 commit f9c9fe8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+6731
-0
lines changed

‎sdk/monitor/azure-monitor-querymetrics/CHANGELOG.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Release History
2+
3+
## 1.0.0 (Unreleased)
4+
5+
### Features Added
6+
7+
- Initial release. This version includes the core functionality for querying metrics from Azure Monitor using the `MetricsClient`, which was originally introduced in the `azure-monitor-query` package.
8+
9+
### Breaking Changes
10+
11+
### Bugs Fixed
12+
13+
### Other Changes

‎sdk/monitor/azure-monitor-querymetrics/LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
Copyright (c) Microsoft Corporation.
2+
3+
MIT License
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎sdk/monitor/azure-monitor-querymetrics/MANIFEST.in

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
include *.md
2+
include LICENSE
3+
include azure/monitor/querymetrics/py.typed
4+
recursive-include tests *.py
5+
recursive-include samples *.py *.md
6+
include azure/__init__.py
7+
include azure/monitor/__init__.py
8+
include azure/monitor/querymetrics/__init__.py

‎sdk/monitor/azure-monitor-querymetrics/README.md

Lines changed: 252 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,252 @@
1+
# Azure Monitor Query Metrics client library for Python
2+
3+
The Azure Monitor Query Metrics client library is used to execute read-only queries against [Azure Monitor][azure_monitor_overview]'s metrics data platform:
4+
5+
- [Metrics](https://learn.microsoft.com/azure/azure-monitor/essentials/data-platform-metrics) - Collects numeric data from monitored resources into a time series database. Metrics are numerical values that are collected at regular intervals and describe some aspect of a system at a particular time. Metrics are lightweight and capable of supporting near real-time scenarios, making them useful for alerting and fast detection of issues.
6+
7+
**Resources:**
8+
9+
<!-- TODO: Add PyPI, Conda, Ref Docs, Samples links-->
10+
- [Source code][source]
11+
- [Service documentation][azure_monitor_overview]
12+
- [Change log][changelog]
13+
14+
## Getting started
15+
16+
### Prerequisites
17+
18+
- Python 3.9 or later
19+
- An [Azure subscription][azure_subscription]
20+
- Authorization to read metrics data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription containing the resources to be queried.
21+
- An Azure resource of any kind (Storage Account, Key Vault, Cosmos DB, etc.).
22+
23+
### Install the package
24+
25+
Install the Azure Monitor Query Metrics client library for Python with [pip][pip]:
26+
27+
```bash
28+
pip install azure-monitor-querymetrics
29+
```
30+
31+
### Create the client
32+
33+
An authenticated client is required to query Metrics. The library includes both synchronous and asynchronous forms of the client. To authenticate, create an instance of a token credential. Use that instance when creating a `MetricsClient`. The following examples use `DefaultAzureCredential` from the [azure-identity](https://pypi.org/project/azure-identity/) package.
34+
35+
#### Synchronous client
36+
37+
Consider the following example, which creates a synchronous client for Metrics querying:
38+
39+
```python
40+
from azure.identity import DefaultAzureCredential
41+
from azure.monitor.querymetrics import MetricsClient
42+
43+
credential = DefaultAzureCredential()
44+
metrics_client = MetricsClient("https://<regional endpoint>", credential)
45+
```
46+
47+
#### Asynchronous client
48+
49+
The asynchronous form of the client API is found in the `.aio`-suffixed namespace. For example:
50+
51+
```python
52+
from azure.identity.aio import DefaultAzureCredential
53+
from azure.monitor.querymetrics.aio import MetricsClient
54+
55+
credential = DefaultAzureCredential()
56+
async_metrics_client = MetricsClient("https://<regional endpoint>", credential)
57+
```
58+
59+
To use the asynchronous clients, you must also install an async transport, such as [aiohttp](https://pypi.org/project/aiohttp/).
60+
61+
```sh
62+
pip install aiohttp
63+
```
64+
65+
#### Configure client for Azure sovereign cloud
66+
67+
By default, the client is configured to use the Azure public cloud. To use a sovereign cloud, provide the correct `audience` argument when creating the `MetricsClient`. For example:
68+
69+
```python
70+
from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
71+
from azure.monitor.querymetrics import MetricsClient
72+
73+
# Authority can also be set via the AZURE_AUTHORITY_HOST environment variable.
74+
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)
75+
76+
metrics_client = MetricsClient(
77+
"https://usgovvirginia.metrics.monitor.azure.us", credential, audience="https://metrics.monitor.azure.us"
78+
)
79+
```
80+
81+
### Execute the query
82+
83+
For examples of Metrics queries, see the [Examples](#examples) section.
84+
85+
## Key concepts
86+
87+
### Metrics data structure
88+
89+
Each set of metric values is a time series with the following characteristics:
90+
91+
- The time the value was collected
92+
- The resource associated with the value
93+
- A namespace that acts like a category for the metric
94+
- A metric name
95+
- The value itself
96+
- Some metrics have multiple dimensions as described in multi-dimensional metrics.
97+
98+
## Examples
99+
100+
- [Metrics query](#metrics-query)
101+
- [Handle metrics query response](#handle-metrics-query-response)
102+
- [Example of handling response](#example-of-handling-response)
103+
- [Query metrics for multiple resources](#query-metrics-for-multiple-resources)
104+
105+
### Metrics query
106+
107+
To query metrics for one or more Azure resources, use the `query_resources` method of `MetricsClient`. This method requires a regional endpoint when creating the client. For example, "https://westus3.metrics.monitor.azure.com".
108+
109+
Each Azure resource must reside in:
110+
111+
- The same region as the endpoint specified when creating the client.
112+
- The same Azure subscription.
113+
114+
The resource IDs must be that of the resources for which metrics are being queried. It's normally of the format `/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/topics/<resource-name>`.
115+
116+
To find the resource ID/URI:
117+
118+
1. Navigate to your resource's page in the Azure portal.
119+
1. Select the **JSON View** link in the **Overview** section.
120+
1. Copy the value in the **Resource ID** text box at the top of the JSON view.
121+
122+
Furthermore:
123+
124+
- The user must be authorized to read monitoring data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription to be queried.
125+
- The metric namespace containing the metrics to be queried must be provided. For a list of metric namespaces, see [Supported metrics and log categories by resource type][metric_namespaces].
126+
127+
```python
128+
from datetime import timedelta
129+
import os
130+
131+
from azure.core.exceptions import HttpResponseError
132+
from azure.identity import DefaultAzureCredential
133+
from azure.monitor.querymetrics import MetricsClient, MetricAggregationType
134+
135+
endpoint = "https://westus3.metrics.monitor.azure.com"
136+
credential = DefaultAzureCredential()
137+
client = MetricsClient(endpoint, credential)
138+
139+
resource_ids = [
140+
"/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/storageAccounts/<resource-name-1>",
141+
"/subscriptions/<id>/resourceGroups/<rg-name>/providers/<source>/storageAccounts/<resource-name-2>"
142+
]
143+
144+
response = client.query_resources(
145+
resource_ids=resource_ids,
146+
metric_namespace="Microsoft.Storage/storageAccounts",
147+
metric_names=["UsedCapacity"],
148+
timespan=timedelta(hours=2),
149+
granularity=timedelta(minutes=5),
150+
aggregations=[MetricAggregationType.AVERAGE],
151+
)
152+
153+
for metrics_query_result in response:
154+
for metric in metrics_query_result.metrics:
155+
print(f"Metric: {metric.name}")
156+
for time_series in metric.timeseries:
157+
for metric_value in time_series.data:
158+
if metric_value.average is not None:
159+
print(f"Average: {metric_value.average}")
160+
```
161+
162+
#### Handle metrics query response
163+
164+
The metrics query API returns a list of `MetricsQueryResult` objects. The `MetricsQueryResult` object contains properties such as a list of `Metric`-typed objects, `granularity`, `namespace`, and `timespan`. The `Metric` objects list can be accessed using the `metrics` param. Each `Metric` object in this list contains a list of `TimeSeriesElement` objects. Each `TimeSeriesElement` object contains `data` and `metadata_values` properties. In visual form, the object hierarchy of the response resembles the following structure:
165+
166+
```
167+
MetricsQueryResult
168+
|---granularity
169+
|---timespan
170+
|---cost
171+
|---namespace
172+
|---resource_region
173+
|---metrics (list of `Metric` objects)
174+
|---id
175+
|---type
176+
|---name
177+
|---unit
178+
|---timeseries (list of `TimeSeriesElement` objects)
179+
|---metadata_values
180+
|---data (list of data points)
181+
```
182+
183+
**Note:** Each `MetricsQueryResult` is returned in the same order as the corresponding resource in the `resource_ids` parameter. If multiple different metrics are queried, the metrics are returned in the order of the `metric_names` sent.
184+
185+
**Example of handling response**
186+
187+
```python
188+
import os
189+
from azure.monitor.querymetrics import MetricsClient, MetricAggregationType
190+
from azure.identity import DefaultAzureCredential
191+
192+
credential = DefaultAzureCredential()
193+
client = MetricsClient("https://<regional endpoint>", credential)
194+
195+
metrics_uri = os.environ['METRICS_RESOURCE_URI']
196+
response = client.query_resource(
197+
metrics_uri,
198+
metric_names=["PublishSuccessCount"],
199+
aggregations=[MetricAggregationType.AVERAGE]
200+
)
201+
202+
for metrics_query_result in response:
203+
for metric in metrics_query_result.metrics:
204+
print(f"Metric: {metric.name}")
205+
for time_series in metric.timeseries:
206+
for metric_value in time_series.data:
207+
if metric_value.average is not None:
208+
print(f"Average: {metric_value.average}")
209+
```
210+
211+
## Troubleshooting
212+
213+
See our [troubleshooting guide][troubleshooting_guide] for details on how to diagnose various failure scenarios.
214+
215+
## Next steps
216+
217+
To learn more about Azure Monitor, see the [Azure Monitor service documentation][azure_monitor_overview].
218+
219+
### Samples
220+
221+
The following code samples show common scenarios with the Azure Monitor Query Metrics client library.
222+
223+
#### Metrics query samples
224+
225+
To be added.
226+
227+
## Contributing
228+
229+
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].
230+
231+
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repositories using our CLA.
232+
233+
This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [[email protected]][coc_contact] with any additional questions or comments.
234+
235+
<!-- LINKS -->
236+
237+
[azure_core_exceptions]: https://aka.ms/azsdk/python/core/docs#module-azure.core.exceptions
238+
[azure_core_ref_docs]: https://aka.ms/azsdk/python/core/docs
239+
[azure_monitor_overview]: https://learn.microsoft.com/azure/azure-monitor/
240+
[azure_subscription]: https://azure.microsoft.com/free/python/
241+
[changelog]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics/CHANGELOG.md
242+
[metric_namespaces]: https://learn.microsoft.com/azure/azure-monitor/reference/supported-metrics/metrics-index#supported-metrics-and-log-categories-by-resource-type
243+
[pip]: https://pypi.org/project/pip/
244+
[python_logging]: https://docs.python.org/3/library/logging.html
245+
[samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics/samples
246+
[source]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-querymetrics/
247+
[troubleshooting_guide]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/monitor/azure-monitor-querymetrics/TROUBLESHOOTING.md
248+
249+
[cla]: https://cla.microsoft.com
250+
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
251+
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
252+
[coc_contact]: mailto:[email protected]

‎sdk/monitor/azure-monitor-querymetrics/TROUBLESHOOTING.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Troubleshooting Azure Monitor Query Metrics client library issues
2+
3+
This troubleshooting guide contains instructions to diagnose frequently encountered issues while using the Azure Monitor Query Metrics client library for Python.
4+
5+
## Table of contents
6+
7+
* [General Troubleshooting](#general-troubleshooting)
8+
* [Enable client logging](#enable-client-logging)
9+
* [Troubleshooting authentication issues with metrics query requests](#authentication-errors)
10+
* [Troubleshooting running async APIs](#errors-with-running-async-apis)
11+
* [Troubleshooting Metrics Query](#troubleshooting-metrics-query)
12+
* [Troubleshooting insufficient access error](#troubleshooting-insufficient-access-error-for-metrics-query)
13+
* [Troubleshooting unsupported granularity for metrics query](#troubleshooting-unsupported-granularity-for-metrics-query)
14+
* [Additional azure-core configurations](#additional-azure-core-configurations)
15+
16+
17+
## General Troubleshooting
18+
19+
Monitor query raises exceptions described in [`azure-core`](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md)
20+
21+
### Enable client logging
22+
23+
To troubleshoot issues with Azure Monitor Query Metrics library, it is important to first enable logging to monitor the behavior of the application. The errors and warnings in the logs generally provide useful insights into what went wrong and sometimes include corrective actions to fix issues.
24+
25+
This library uses the standard [logging](https://docs.python.org/3/library/logging.html) library for logging. Basic information about HTTP sessions, such as URLs and headers, is logged at the INFO level.
26+
Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with the logging_enable argument:
27+
28+
```python
29+
import logging
30+
from azure.monitor.querymetrics import MetricsClient
31+
32+
# Create a logger for the 'azure.monitor.querymetrics' SDK
33+
logger = logging.getLogger('azure.monitor.querymetrics')
34+
logger.setLevel(logging.DEBUG)
35+
36+
# Configure a console output
37+
handler = logging.StreamHandler(stream=sys.stdout)
38+
logger.addHandler(handler)
39+
40+
client = MetricsClient(credential, logging_enable=True)
41+
```
42+
43+
Similarly, logging_enable can enable detailed logging for a single operation, even when it isn't enabled for the client:
44+
45+
```python
46+
client.query_workspace(logging_enable=True)
47+
```
48+
49+
### Authentication errors
50+
51+
Azure Monitor Query Metrics supports Azure Active Directory authentication. The MetricsClient has methods to set the `credential`. To provide a valid credential, you can use
52+
`azure-identity` dependency. For more details on getting started, refer to
53+
the [README](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-querymetrics#create-the-client)
54+
of Azure Monitor Query Metrics library. You can also refer to
55+
the [Azure Identity documentation](https://learn.microsoft.com/python/api/overview/azure/identity-readme)
56+
for more details on the various types of credential supported in `azure-identity`.
57+
58+
For more help on troubleshooting authentication errors please see the Azure Identity client library [troubleshooting guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/TROUBLESHOOTING.md).
59+
60+
### Errors with running async APIs
61+
62+
The async transport is designed to be opt-in. [AioHttp](https://pypi.org/project/aiohttp/) is one of the supported implementations of async transport. It is not installed by default. You need to install it separately as follows:
63+
64+
```
65+
pip install aiohttp
66+
```
67+
68+
## Troubleshooting Metrics Query
69+
70+
### Troubleshooting insufficient access error for metrics query
71+
72+
If you encounter 401 authorization errors while querying metrics using `MetricsClient`, please ensure you are authorized to read monitoring data at the Azure subscription level. For example, the [Monitoring Reader role](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles/monitor#monitoring-reader) on the subscription to be queried.
73+
74+
### Troubleshooting unsupported granularity for metrics query
75+
76+
If you notice the following exception, this is due to an invalid time granularity in the metrics query request. Your
77+
query might have set the `granularity` keyword argument to an unsupported duration.
78+
79+
```text
80+
"{"code":"BadRequest","message":"Invalid time grain duration: PT10M, supported ones are: 00:01:00,00:05:00,00:15:00,00:30:00,01:00:00,06:00:00,12:00:00,1.00:00:00"}"
81+
```
82+
83+
As documented in the error message, the supported granularity for metrics queries are 1 minute, 5 minutes, 15 minutes,
84+
30 minutes, 1 hour, 6 hours, 12 hours and 1 day.
85+
86+
## Additional azure-core configurations
87+
88+
When calling the methods, some properties including `retry_mode`, `timeout`, `connection_verify` can be configured by passing in as keyword arguments. See
89+
[configurations](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md#configurations) for list of all such properties.

‎sdk/monitor/azure-monitor-querymetrics/_metadata.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"apiVersion": "2024-02-01"
3+
}

0 commit comments

Comments
 (0)