ORT perf test support for plugin EP

[chilo-ms]

Description

Add support for onnxruntime_perf_test to register plugin EP dll and run plugin EP.

As support for plugin execution providers (EPs) requires additional options and most single-character options have already been used, multi-character options are now necessary to ensure clarity and readability. Therefore, support for Abseil flags is added, which enables multi-character options and provides cross-platform compatibility.

New options:

• --plugin_ep_libs [registration names and libraries] Specifies a list of plugin execution provider (EP) registration names and their corresponding shared libraries to register.
  [Usage]: --plugin_ep_libs "plugin_ep_name_1|plugin_ep_1.dll plugin_ep_name_2|plugin_ep_2.dll ... "

• --plugin_eps [Plugin EPs] Specifies a semicolon-separated list of plugin execution providers (EPs) to use.
  [Usage]: --plugin_eps "plugin_ep_1;plugin_ep_2;... "

• --plugin_ep_options [EP options] Specifies provider options for each EP listed in --plugin_eps. Options (key-value pairs) for each EP are separated by space and EPs are separated by semicolons.
  [Usage]:
  --plugin_ep_options "ep_1_option_1_key|ep_1_option_1_value ...;ep_2_option_1_key|ep_2_option_1_value ...;..." or
  --plugin_ep_options ";ep_2_option_1_key|ep_2_option_1_value ...;..." or
  --plugin_ep_options "ep_1_option_1_key|ep_1_option_1_value ...;;ep_3_option_1_key|ep_3_option_1_value ...;..."

• --list_ep_devices Prints all available device indices and their properties (including metadata). This option makes the program exit early without performing inference.

• --select_ep_devices [list of device indices] A semicolon-separated list of device indices to add to the session and run with.

Usage:

1. Use --plugin_ep_libs and --list_ep_devices to list all the devices.

--list_ep_devices --plugin_ep_libs "TensorRTEp|C:\TensorRTEp.dll example_ep|C:\example_plugin_ep.dll"

It will print the devices info

===== EP device id 0 ======
name: CPUExecutionProvider
vendor: Microsoft
metadata:
  version: 1.23.0

===== EP device id 1 ======
name: example_ep
vendor: Contoso
metadata:
  supported_devices: CrackGriffin 7+
  version: 0.1.0

===== EP device id 2 ======
name: TensorRTEp
vendor: Nvidia
metadata:
  gpu_type: data center
  version: 0.1.0

2. Use --select_ep_devices to select the device by index. And add --plugin_eps to specify the EP name. The EP name should match the name when ep library passes in to create the ep factory.

--plugin_ep_libs "TensorRTEp|C:\TensorRTEp.dll" --select_ep_devices 2 --plugin_eps TensorRTEp -r 1 C:\mul_op\mul_1.onnx

3. Or simply use -e to specify the EP name. ORT perf test will add all the devices created by the plugin EP.
   The EP name should match the name when ep library passes in to create the ep factory.

--plugin_ep_libs "TensorRTEp|C:\TensorRTEp.dll" --plugin_eps TensorRTEp -r 1 C:\mul_op\mul_1.onnx

[yuslepukhin]

🕐

[jywu-msft]

@javier-intel please review and provide requirements for OVEP here. thanks.

[javier-intel]

@javier-intel please review and provide requirements for OVEP here. thanks.

Hi @jywu-msft, to support OVEP ABI changes in perf_test I suggest introducing new options to facilitate device selection:

• -list_devices prints all the index and properties (including metadata) of every entry in env.GetEpDevices()

• -select_devices takes a (semicolon separated?) list of device indices, and use that to further list the devices in this code before calling session_options.AppendExecutionProvider_V2(env, added_ep_devices, provider_options);

[jywu-msft]

@javier-intel please review and provide requirements for OVEP here. thanks.

Hi @jywu-msft, to support OVEP ABI changes in perf_test I suggest introducing new options to facilitate device selection:

• -list_devices prints all the index and properties (including metadata) of every entry in env.GetEpDevices()
• -select_devices takes a (semicolon separated?) list of device indices, and use that to further list the devices in this code before calling session_options.AppendExecutionProvider_V2(env, added_ep_devices, provider_options);

@chilo-ms

[github-actions]

You can commit the suggested changes from lintrunner.

[yuslepukhin]