You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/tutorials/features.rst
+4-18Lines changed: 4 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -137,19 +137,6 @@ For more detailed information, check `torch.compile for GPU <features/torch_comp
137
137
138
138
features/torch_compile_gpu
139
139
140
-
Legacy Profiler Tool (Prototype)
141
-
--------------------------------
142
-
143
-
The legacy profiler tool is an extension of PyTorch* legacy profiler for profiling operators' overhead on XPU devices. With this tool, you can get the information in many fields of the run models or code scripts. Build Intel® Extension for PyTorch* with profiler support as default and enable this tool by adding a `with` statement before the code segment.
144
-
145
-
For more detailed information, check `Legacy Profiler Tool <features/profiler_legacy.md>`_.
146
-
147
-
.. toctree::
148
-
:hidden:
149
-
:maxdepth:1
150
-
151
-
features/profiler_legacy
152
-
153
140
Simple Trace Tool (Prototype)
154
141
-----------------------------
155
142
@@ -191,14 +178,13 @@ For more detailed information, check `Compute Engine <features/compute_engine.md
191
178
features/compute_engine
192
179
193
180
181
+
``IPEX_LOGGING`` (Prototype feature for debug)
182
+
----------------------------------------------
194
183
195
184
196
-
IPEX LOG (Prototype)
197
-
--------------------
198
-
199
-
IPEX_LOGGING provides the capacity to log IPEX internal information. If you would like to use torch-style log, that is, the log/verbose is introduced by torch, and refer to cuda code, pls still use torch macro to show the log. For example, TORCH_CHECK, TORCH_ERROR. If the log is IPEX specific, or is going to trace IPEX execution, pls use IPEX_LOGGING. For some part of usage are still discussed with habana side, if has change some feature will update here.
185
+
``IPEX_LOGGING`` provides the capability to log verbose information from Intel® Extension for PyTorch\* . Please use ``IPEX_LOGGING`` to get the log information or trace the execution from Intel® Extension for PyTorch\*. Please continue using PyTorch\* macros such as ``TORCH_CHECK``, ``TORCH_ERROR``, etc. to get the log information from PyTorch\*.
200
186
201
-
For more detailed information, check `IPEX LOG<features/ipex_log.md>`_.
187
+
For more detailed information, check `IPEX_LOGGING<features/ipex_log.md>`_.
IPEX_LOGGING provides the capacity to log IPEX internal information. If you would like to use torch-style log, that is, the log/verbose is introduced by torch, and refer to cuda code, pls still use torch macro to show the log. For example, TORCH_CHECK, TORCH_ERROR. If the log is IPEX specific, or is going to trace IPEX execution, pls use IPEX_LOGGING. For some part of usage are still discussed with habana side, if has change some feature will update here.
6
+
`IPEX_LOGGING` provides the capability to log verbose information from Intel® Extension for PyTorch\* . Please use `IPEX_LOGGING`to get the log information or trace the execution from Intel® Extension for PyTorch\*. Please continue using PyTorch\* macros such as `TORCH_CHECK`, `TORCH_ERROR`, etc. to get the log information from PyTorch\*.
14
7
15
-
## Feature for IPEX Log
16
-
### Log level
17
-
Currently supported log level and usage are as follow, default using log level is `WARN`:
8
+
## `IPEX_LOGGING` Definition
9
+
### Log Level
10
+
The supported log levels are defined as follows, default log level is `DISABLED`:
18
11
19
12
| log level | number | usage |
20
13
| :----: | :----: | :----: |
21
-
| TRACE | 0 | reserve it for further usage extension|
22
-
| DEBUG | 1 | We would like to insert DEBUG inside each host function, when log level is debug, we can get the whole calling stack |
23
-
| INFO | 2 | Record calls to other library functions and environment variable settings, such as onemkl calling and set verbose level|
24
-
| WARN | 3 | On the second attempt of the program, such as memory reallocation |
25
-
| ERR | 4 | Found error in try catch |
26
-
| CRITICAL | 5 | reserve it for further usage extension |
14
+
| DISABLED | -1 | Disable the logging |
15
+
| TRACE | 0 | Reserve for further usage |
16
+
| DEBUG | 1 | Provide the whole calling stack info |
17
+
| INFO | 2 | Record calling info to other library functions and environment variable settings |
18
+
| WARN | 3 | Warn the second attempt of an action, such as memory reallocation |
19
+
| ERR | 4 | Report error in try catch |
20
+
| CRITICAL | 5 | Reserve for further usage |
27
21
28
-
### Log component
29
-
Log component is for specify which part of IPEX does this log belongs to, currently we have seprate IPEX into four parts, shown as table below.
22
+
### Log Component
23
+
Log component is used to specify which part from Intel® Extension for PyTorch\*does this log information belong to. The supported log components are defined as follows:
All the usage are defined in `utils/LogUtils.h`. Currently Intel® Extension for PyTorch\* supports:
37
35
38
-
For `SYNGRAPH` you can also add log sub componment which is no restriction on categories.
36
+
### Simple Log
37
+
You can use `IPEX_XXX_LOG`, XXX represents the log level as mentioned above. There are four parameters defined for simple log:
38
+
- Log component, representing which part of Intel® Extension for PyTorch\* does this log belong to.
39
+
- Log sub component, input an empty string("") for general usages. For `SYNGRAPH` you can add any log sub componment.
40
+
- Log message template format string.
41
+
- Log name.
39
42
43
+
Below is an example for using simple log inside abs kernel:
40
44
41
-
## How to add log in IPEX
42
-
All the usage are inside file `utils/LogUtils.h`. IPEX Log support two types of log usage, the first one is simple log, you can use IPEX_XXX_LOG, XXX represents the log level, including six log level mentioned above. There are for params for simple log, the first one is log component, representing which part of IPEX does this log belongs to. The second on is log sub component, for most of the IPEX usage, just input a empty string("") here. For the third param, it is an log message template, you can use it as python format string, or you can also refer to fmt lib https://github.com/fmtlib/fmt. Here is an example for simple log, for add a log inside abs kernel:
43
45
```c++
44
46
45
47
IPEX_INFO_LOG("OPS", "", "Add a log for inside ops {}", "abs");
46
48
47
49
```
50
+
### Event Log
51
+
Event log is used for recording a whole event, such as an operator calculation. The whole event is identified by an unique `event_id`. You can also mark each step by using `step_id`. Use `IPEX_XXX_EVENT_END()` to complete the logging of the whole event.
48
52
49
-
For the second log level is event log, which is used for recording a whole event, such as a ops calculation. A whole event is identify through a event_id, for the whole event this event_id should be the same, but cannot be duplicate with other event_id, or it will met an undefined behaviour. You can also mark each step by using a step_id, there are no limitation for step_id. For the end of the whole event, should use IPEX_XXX_EVENT_END(), XXX is the log level mention aboved.
53
+
Below is an example for using eventlog:
50
54
51
-
Below is an example for ipex event log:
52
55
```c++
53
56
IPEX_EVENT_END("OPS", "", "record_avg_pool", "start", "Here record the time start with arg:{}", arg);
IPEX privide some enviornment setting used for log output settings, currently, we support below five settings.
64
+
Intel® Extension for PyTorch\* provides five enviornment variables for configuring log output:
62
65
63
-
1.IPEX_LOGGING_LEVEL, accept int or string, default is 3 for `WARN`. Currently you can choose seven different log level within ipex, including -1 `DISABLED` under this setting, all the usage related with IPEX_LOGGING will be disabled. Another six log levels are we mentioned above.
64
-
2.IPEX_LOG_COMPONENT, accept a string, sepreated by `/` first part is log component and the second part is for log sub component, used for state which component and sub log component you would like to log, default is "ALL". Currently we supoort 5 different log component,, `ALL` is for all the output in IPEX, the other four log component are we mentioned above. You could also specify several log component, sepreating using `,`such as "OPS;MEMORY".For log sub component, it still discussed with habana side, pls don't use it first.
65
-
3.IPEX_LOG_OUTPUT, accept a string. If you are using IPEX_LOG_OUTPUT, than all the logs will log inside the file rather than log into the console, you can use it like export IPEX_LOG_OUTPUT="./ipex.log", all the log will log inside ipex.log in current work folder.
66
-
4.IPEX_LOG_ROTATE_SIZE, accept a int, default =10. Only validate when export IPEX_LOG_OUTPUT, specifing how large file will be used when rotating this log, size is MB.
67
-
5.IPEX_LOG_SPLIT_SIZE, accept a int, default = null. Only validate when export IPEX_LOG_OUTPUT, specifing how large file will be used when split this log, size is MB.
66
+
-`IPEX_LOGGING_LEVEL`, accept integar or string, default is -1 for `DISABLED`.
67
+
-`IPEX_LOG_COMPONENT`, accept string, used for specifying the log componentand sub log component you would like to log, default is "ALL". The log component and sub log component are separated by `/`. You could also specify several log components, such as "OPS;MEMORY".
68
+
-`IPEX_LOG_OUTPUT`, accept string. If you are using `IPEX_LOG_OUTPUT`, than all the logs will recorded inside a file rather than the console. Example: export IPEX_LOG_OUTPUT="./ipex.log".
69
+
-`IPEX_LOG_ROTATE_SIZE`, accept integar, default is 10. Can be used only with `IPEX_LOG_OUTPUT`, for specifing how large file will be used when rotating this log, size is MB.
70
+
-`IPEX_LOG_SPLIT_SIZE`, accept integar, default = null. Can be used only with `IPEX_LOG_OUTPUT`, for specifing how large file will be used when splitting the logs, size is MB.
68
71
69
72
## Usage in python
70
-
1.torch.xpu.set_log_level(log_level) and torch.xpu.get_log_level(), these two functions are used for get and set the log level.
71
-
2.torch.xpu.set_log_output_file_path(log_path) and torch.xpu.get_log_output_file_path(), these two functions are used for get and set the log output file path, once log output file path is set, logs will not be print on the console, will only output in the file.
72
-
3.torch.xpu.set_log_rotate_file_size(file size) and torch.xpu.get_log_rotate_file_size(), these two functions are used for get and set the log rotate file size, only validate when output file path is set.
73
-
4.torch.xpu.set_log_split_file_size(file size) and torch.xpu.get_log_split_file_size(), these two functions are used for get and set the log split file size, only validate when output file path is set.
74
-
5.torch.xpu.set_log_component(log_component), and torch.xpu.get_log_component(), these two functions are used for get and set the log component, log component string are same with enviornment settings.
73
+
-`torch.xpu.set_log_level(log_level)` and `torch.xpu.get_log_level()`, these two functions are used for getting and setting the log level.
74
+
-`torch.xpu.set_log_output_file_path(log_path)` and `torch.xpu.get_log_output_file_path()`, these two functions are used for getting and setting the log output file path, once log output file path is set, logs will be recorded in file only.
75
+
-`torch.xpu.set_log_rotate_file_size(file size)` and `torch.xpu.get_log_rotate_file_size()`, these two functions are used for getting and setting the log rotate file size. Can be used when output file path is set.
76
+
-`torch.xpu.set_log_split_file_size(file size)` and `torch.xpu.get_log_split_file_size()`, these two functions are used for getting and setting the log split file size. Can be used when output file path is set.
77
+
-`torch.xpu.set_log_component(log_component)`, and `torch.xpu.get_log_component()`, these two functions are used for getting and setting the log component. The log component string are the same as defined in enviornment settings.
75
78
76
-
## Use IPEX log for simple trace
77
-
For now, IPEX_SIMPLE_TRACE is depre deprecated, and pls use torch.xpu.set_log_level(0), it will show logs like previous IPEX_SIMPLE_TRACE.
79
+
## Replace `IPEX_SIMPLE_TRACE`
80
+
Use `torch.xpu.set_log_level(0)` to get logs to replace the previous usage in `IPEX_SIMPLE_TRACE`.
78
81
79
-
## Use IPEX log for verbose
80
-
For now, IPEX_VERBOSE is deprecated, pls use torch.xpu.set_log_level(1), it will show logs like previous IPEX_VERBOSE.
82
+
## Replace `IPEX_VERBOSE`
83
+
Use `torch.xpu.set_log_level(1)` to get logs to replace the previous usage in `IPEX_VERBOSE`.
You can examine the sequence of profiled operators, runtime functions andXPU kernels in these trace viewers. Here shows a trace result for ResNet50 run on XPU backend viewed by Perfetto viewer:
You may meet an issue that cannot collect profiling information of XPU kernels and device memory operations due to the failures in creating the tracers, when using Kineto profiler based on oneTrace. If you meet such failures that any tracer or collector could not be successfully created, please try the following workaround.
175
+
176
+
```bash
177
+
export ZE_ENABLE_TRACING_LAYER=1
178
+
```
179
+
180
+
> Note that this environment variable should be setasglobal before running any user level applications.
The legacy profiler tool is an extension of PyTorch\* legacy profiler for profiling operators' overhead on XPU devices. With this tool, users can get the information in many fields of the run models or code scripts. User should build Intel® Extension for PyTorch\* with profiler support as default and enable this tool by a `with` statement before the code segment.
7
-
8
-
## Use Case
9
-
10
-
To use the legacy profiler tool, you need to build Intel® Extension for PyTorch\* from source or install it via prebuilt wheel. You also have various methods to disable this tool.
11
-
12
-
### Build Tool
13
-
14
-
The build option `BUILD_PROFILER` is switched on as default but you can switch it off via setting `BUILD_PROFILER=OFF` while building Intel® Extension for PyTorch\* from source. With `BUILD_PROFILER=OFF`, no profiler code will be compiled and all python scripts using profiler with XPU support will raise a runtime error to user.
15
-
16
-
```bash
17
-
[BUILD_PROFILER=ON] python setup.py install # build from source with profiler tool
18
-
BUILD_PROFILER=OFF python setup.py install # build from source without profiler tool
19
-
```
20
-
21
-
### Use Tool
22
-
23
-
In your model script, write `with` statement to enable the legacy profiler tool ahead of your code snippets, as shown in the following example:
24
-
25
-
```python
26
-
# import all necessary libraries
27
-
import torch
28
-
import intel_extension_for_pytorch
29
-
30
-
# these lines won't be profiled before enabling profiler tool
# enable legacy profiler tool with a `with` statement
34
-
with torch.autograd.profiler_legacy.profile(use_xpu=True) as prof:
35
-
# do what you want to profile here after the `with` statement with proper indent
36
-
output_tensor_1 = torch.nonzero(input_tensor)
37
-
output_tensor_2 = torch.unique(input_tensor)
38
-
39
-
# print the result table formatted by the legacy profiler tool as your wish
40
-
print(prof.key_averages().table())
41
-
```
42
-
43
-
There are a number of useful parameters defined in `torch.autograd.profiler_legacy.profile()`. Many of them are aligned with usages defined in PyTorch\*'s official profiler, such as `record_shapes`, a very useful parameter to control whether to record the shape of input tensors for each operator. To enable legacy profiler on XPU devices, pass `use_xpu=True`. For the usage of more parameters, please refer to [PyTorch\*'s tutorial page](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html).
44
-
45
-
### Disable Tool in Model Script
46
-
47
-
To disable the legacy profiler tool temporarily in your model script, pass `enabled=False` to `torch.autograd.profiler_legacy.profile()`:
48
-
49
-
```python
50
-
with torch.autograd.profiler_legacy.profile(enabled=False, use_xpu=True) as prof:
51
-
# as `enabled` is set to false, the profiler won't work on these lines of code
52
-
output_tensor_1 = torch.nonzero(input_tensor)
53
-
output_tensor_2 = torch.unique(input_tensor)
54
-
55
-
# This print will raise an error to user as the profiler was disabled
56
-
print(prof.key_averages().table())
57
-
```
58
-
59
-
### Results
60
-
61
-
Using the script shown above in **Use Tool** part, you'll see the result table printed out to the console as below:
-`Self CPU %`, `Self CPU`: the time consumed by the operator itself at host excluded its children operator call. The column marked with percentage sign shows the propotion of time to total self cpu time. While an operator calls more than once in a run, the self cpu time may increase in this field.
69
-
-`CPU total %`, `CPU total`: the time consumed by the operator at host included its children operator call. The column marked with percentasge sign shows the propotion of time to total cpu time. While an operator calls more than once in a run, the cpu time may increase in this field.
70
-
-`CPU time avg`: the average time consumed by each once call of the operator at host. This average is calculated on the cpu total time.
71
-
-`Self XPU`, `Self XPU %`: similar to `Self CPU (%)` but shows the time consumption on XPU devices.
72
-
-`XPU total`: similar to `CPU total` but shows the time consumption on XPU devices.
73
-
-`XPU time avg`: similar to `CPU time avg` but shows average time sonsumption on XPU devices. This average is calculated on the XPU total time.
74
-
-`# of Calls`: number of call for each operators in a run.
75
-
76
-
You can print result table in different styles, such as sort all called operators in reverse order via `print(prof.table(sort_by='id'))` like:
For more example results, please refer to [PyTorch\*'s tutorial page](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html).
93
-
6
+
The legacy profiler tool will be deprecated from Intel® Extension for PyTorch* very soon. Please use [Kineto Supported Profiler Tool](./profiler_kineto.md) instead for profiling operators' executing time cost on Intel® GPU devices.
0 commit comments