Skip to content

Commit bdd509f

Browse files
not apply auto batch for CPU plugin
Signed-off-by: HU Yuan2 <[email protected]>
1 parent b9332d2 commit bdd509f

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

src/inference/src/dev/core_impl.cpp

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1094,8 +1094,11 @@ std::shared_ptr<const ov::Model> ov::CoreImpl::apply_auto_batching(const std::sh
10941094
.get_property(ov::supported_properties.name(), parsed._config)
10951095
.as<std::vector<ov::PropertyName>>();
10961096
auto it = std::find(metrics.begin(), metrics.end(), ov::optimal_batch_size.name());
1097-
if (metrics.end() == it)
1097+
if (metrics.end() == it) {
10981098
return model;
1099+
} else if (parsed._deviceName == "CPU") {//not apply auto batch for CPU plugin
1100+
return model;
1101+
}
10991102

11001103
// if applicable, the Auto-Batching is implicitly enabled via the performance hints
11011104
bool bTputInPlg = get_plugin(parsed._deviceName)

0 commit comments

Comments
 (0)