[SYCL][NFCI] Refactor device code split implementation once again (#8833)

AlexeySachkov · web-flow · commit 67da385a5be4 · 2023-04-28T13:55:08.000+02:00
#### Intro This is a refactoring of how we perform device code split in `sycl-post-link`, which is intended to solve several existing issues with the current implementation: 1. increased peak RAM consumption by `sycl-post-link` 2. bad scaling with more and more split "dimensions" being added 3. increased tests maintenance cost due to non-deterministic order (between commits) of output files produced by `sycl-post-link` #### A bit more context about the issues above: (1) Increase peak RAM consumption is caused by the fact that we currently preserve **all** splits in-memory, even though we can process them on-by-one and discard them as soon as we stored them to a disk. This was implemented as a memory consumption optimization in #5021, but it got accidentally reverted in #7302 as an attempt to workaround (2). (2) is pretty much summarized in our source code: https://github.com/intel/llvm/blob/afebb2543ccecb89f83c84b68fba7616bbab89ac/llvm/tools/sycl-post-link/sycl-post-link.cpp#L806-L811 (3) is caused by a bad implementation decision made in #7302: because every split is now identified by a hash, every time you add a new split "dimension"/new feature to an account, it results in different hashes for existing tests. Just look how many unrelated tests had to be updated in #7512, #8056 and #8167 #### Now to the PR itself: It introduces a new infrastructure for categorizing/grouping kernel functions: instead of using hashes, we now build a string description for each kernel function and then group kernels with the same description string together. String description is built by a new entity: it accepts a set of rules, where each rule is a simple function which returns a string for passed `llvm::Function`. Results of all rules are concatenated together and rules are invoked in a stable order of their registration. There is a simple API for building those rules. It provides some predefined rules for the most popular use cases like turning a function attribute or a metadata into a string descriptor for the function. There is also a possibility to pass a custom callback there to implement more complicated logic. #### How does this PR help with issues above? (1) and (2) are fixed in conjunction: `sycl-post-link` was refactored to avoid storing more than one split module at a time and that is possible because the PR unifies per-scope and optional-kernel-features splitters into a single generic splitter. The new API for kernels categorization seems to be flexible enough to provide that infrastructure so merged splitters still look OK code-wise. (3) is caused by using string identifiers instead of hashes as well as by using a data structure which sorts identifiers. #### Any other benefits from this PR? About 50 lines of code less to support :) Extending device code split for more optional features would be even easier than it is now: instead of adding several changes to various places around `UsedOptionalFeatures` structure, it will be just adding a 1-3 lines of code. Please also note that `UsedOptionalFeatures` contains tons of inconsistencies in its implementation, which will all gone with this PR: in `operator==` we don't use hash and instead compare certain fields directly (and we do miss some of them); `generateModuleName` method skips some of optional features and ignores them. Cross-module `device_global` usages checks should now work at all split dimensions (except for ESIMD). #### Any potential downsides? With current `UsedOptionalFeatures` there is a possibility to embed various information (used aspects, `large-grf` flag, etc.) directly during device code split to avoid re-gathering that information later when we generate properties. With the suggested approach, it would be harder to do, because it doesn't seem to naturally fit to the proposed infrastructure: see changes I did around `large-grf` in this PR. However, we have never actually implemented this and re-querying some metadata from function doesn't seem like a bottleneck, so it should really be a very minor and only theoretical downside.
diff --git a/llvm/include/llvm/SYCLLowerIR/SYCLUtils.h b/llvm/include/llvm/SYCLLowerIR/SYCLUtils.h
@@ -22,6 +22,7 @@ namespace llvm {
 namespace sycl {
 namespace utils {
 constexpr char ATTR_SYCL_MODULE_ID[] = "sycl-module-id";
+constexpr char ATTR_SYCL_OPTLEVEL[] = "sycl-optlevel";
 
 using CallGraphNodeAction = ::std::function<void(Function *)>;
 using CallGraphFunctionFilter =
diff --git a/llvm/test/tools/sycl-post-link/assert/property-1.ll b/llvm/test/tools/sycl-post-link/assert/property-1.ll
@@ -12,9 +12,9 @@
 ; RUN: FileCheck %s -input-file=%t_0.prop --implicit-check-not TheKernel2
 ;
 ; RUN: sycl-post-link -split=kernel -symbols -S < %s -o %t.table
-; RUN: FileCheck %s -input-file=%t_0.prop --check-prefixes=CHECK-K1
-; RUN: FileCheck %s -input-file=%t_1.prop --check-prefixes=CHECK-K2
-; RUN: FileCheck %s -input-file=%t_2.prop --check-prefixes=CHECK-K3
+; RUN: FileCheck %s -input-file=%t_0.prop --check-prefixes=CHECK-K3
+; RUN: FileCheck %s -input-file=%t_1.prop --check-prefixes=CHECK-K1
+; RUN: FileCheck %s -input-file=%t_2.prop --check-prefixes=CHECK-K2
 
 ; SYCL source:
 ; void foo() {
diff --git a/llvm/test/tools/sycl-post-link/device-code-split/per-aspect-split-1.ll b/llvm/test/tools/sycl-post-link/device-code-split/per-aspect-split-1.ll
@@ -10,29 +10,29 @@
 ; RUN: sycl-post-link -split=auto -symbols -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t_0.ll --check-prefixes CHECK-M0-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M1-IR \
+; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M1-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M2-IR \
+; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M2-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
 ; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-M0-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M1-SYMS \
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M1-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M2-SYMS \
+; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M2-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
 
 ; RUN: sycl-post-link -split=source -symbols -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t_0.ll --check-prefixes CHECK-M0-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M1-IR \
+; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M1-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M2-IR \
+; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M2-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
 ; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-M0-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M1-SYMS \
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M1-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M2-SYMS \
+; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M2-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
 
 ; RUN: sycl-post-link -split=kernel -symbols -S < %s -o %t.table
diff --git a/llvm/test/tools/sycl-post-link/device-code-split/per-aspect-split-2.ll b/llvm/test/tools/sycl-post-link/device-code-split/per-aspect-split-2.ll
@@ -21,12 +21,12 @@
 ; CHECK-TABLE-NEXT: _2.sym
 ; CHECK-TABLE-EMPTY:
 
-; CHECK-M0-SYMS: kernel0
+; CHECK-M0-SYMS: kernel3
 
-; CHECK-M1-SYMS: kernel3
+; CHECK-M1-SYMS: kernel1
+; CHECK-M1-SYMS: kernel2
 
-; CHECK-M2-SYMS: kernel1
-; CHECK-M2-SYMS: kernel2
+; CHECK-M2-SYMS: kernel0
 
 target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024"
 target triple = "spir64-unknown-linux"
diff --git a/llvm/test/tools/sycl-post-link/device-code-split/per-reqd-wg-size-split-1.ll b/llvm/test/tools/sycl-post-link/device-code-split/per-reqd-wg-size-split-1.ll
@@ -10,29 +10,29 @@
 ; RUN: sycl-post-link -split=auto -symbols -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t_0.ll --check-prefixes CHECK-M0-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M1-IR \
+; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M1-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M2-IR \
+; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M2-IR \
 ; RUN: --implicit-check-not kernel1 --implicit-check-not kernel2
 ; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-M0-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M1-SYMS \
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M1-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel2
-; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M2-SYMS \
+; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M2-SYMS \
 ; RUN: --implicit-check-not kernel1 --implicit-check-not kernel2
 
 ; RUN: sycl-post-link -split=source -symbols -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t_0.ll --check-prefixes CHECK-M0-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M1-IR \
+; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M1-IR \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel2
-; RUN: FileCheck %s -input-file=%t_1.ll --check-prefixes CHECK-M2-IR \
+; RUN: FileCheck %s -input-file=%t_2.ll --check-prefixes CHECK-M2-IR \
 ; RUN: --implicit-check-not kernel1 --implicit-check-not kernel2
 ; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-M0-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel1
-; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M1-SYMS \
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M1-SYMS \
 ; RUN: --implicit-check-not kernel0 --implicit-check-not kernel2
-; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-M2-SYMS \
+; RUN: FileCheck %s -input-file=%t_2.sym --check-prefixes CHECK-M2-SYMS \
 ; RUN: --implicit-check-not kernel1 --implicit-check-not kernel2
 
 ; RUN: sycl-post-link -split=kernel -symbols -S < %s -o %t.table
diff --git a/llvm/test/tools/sycl-post-link/device-code-split/per-reqd-wg-size-split-2.ll b/llvm/test/tools/sycl-post-link/device-code-split/per-reqd-wg-size-split-2.ll
@@ -4,29 +4,29 @@
 ; RUN: sycl-post-link -split=auto -symbols -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t.table --check-prefix CHECK-TABLE
 ;
-; RUN: FileCheck %s -input-file=%t_1.sym --check-prefix CHECK-M0-SYMS \
-; RUN:     --implicit-check-not kernel0 --implicit-check-not kernel1 \
+; RUN: FileCheck %s -input-file=%t_0.sym --check-prefix CHECK-M0-SYMS \
+; RUN:     --implicit-check-not kernel0 --implicit-check-not kernel2
+;
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefix CHECK-M1-SYMS \
+; RUN:     --implicit-check-not kernel1 --implicit-check-not kernel3 \
 ; RUN:     --implicit-check-not kernel2
 ;
 ; RUN: FileCheck %s -input-file=%t_2.sym --check-prefix CHECK-M2-SYMS \
-; RUN:     --implicit-check-not kernel0 --implicit-check-not kernel3
-;
-; RUN: FileCheck %s -input-file=%t_0.sym --check-prefix CHECK-M1-SYMS \
 ; RUN:     --implicit-check-not kernel1 --implicit-check-not kernel2 \
-; RUN:     --implicit-check-not kernel3
+; RUN:     --implicit-check-not kernel0
 
 ; CHECK-TABLE: Code
 ; CHECK-TABLE-NEXT: _0.sym
 ; CHECK-TABLE-NEXT: _1.sym
 ; CHECK-TABLE-NEXT: _2.sym
 ; CHECK-TABLE-EMPTY:
 
-; CHECK-M0-SYMS: kernel3
+; CHECK-M0-SYMS: kernel1
+; CHECK-M0-SYMS: kernel2
 
 ; CHECK-M1-SYMS: kernel0
 
-; CHECK-M2-SYMS: kernel1
-; CHECK-M2-SYMS: kernel2
+; CHECK-M2-SYMS: kernel3
 
 target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024"
 target triple = "spir64-unknown-linux"
diff --git a/llvm/test/tools/sycl-post-link/device-code-split/split-with-kernel-declarations.ll b/llvm/test/tools/sycl-post-link/device-code-split/split-with-kernel-declarations.ll
@@ -8,9 +8,9 @@
 ;
 ; RUN: sycl-post-link -split=kernel -symbols -S < %s -o %t1.table
 ; RUN: FileCheck %s -input-file=%t1.table --check-prefix CHECK-PER-KERNEL-TABLE
-; RUN: FileCheck %s -input-file=%t1_0.sym --check-prefix CHECK-PER-KERNEL-SYM0
-; RUN: FileCheck %s -input-file=%t1_1.sym --check-prefix CHECK-PER-KERNEL-SYM1
-; RUN: FileCheck %s -input-file=%t1_2.sym --check-prefix CHECK-PER-KERNEL-SYM2
+; RUN: FileCheck %s -input-file=%t1_0.sym --check-prefix CHECK-PER-KERNEL-SYM1
+; RUN: FileCheck %s -input-file=%t1_1.sym --check-prefix CHECK-PER-KERNEL-SYM2
+; RUN: FileCheck %s -input-file=%t1_2.sym --check-prefix CHECK-PER-KERNEL-SYM0
 
 ; With per-source split, there should be two device images
 ; CHECK-PER-SOURCE-TABLE: [Code|Properties|Symbols]
diff --git a/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_kernels_in_one_module.ll b/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_kernels_in_one_module.ll
@@ -1,6 +1,6 @@
 ; RUN: sycl-post-link --device-globals --split=source -S < %s -o %t.files.table
-; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD0
-; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD1
+; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD1
+; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD0
 
 ; This test is intended to check that sycl-post-link generates no errors
 ; when a device global variable with the 'device_image_scope' property
diff --git a/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_no_dev_global.ll b/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_no_dev_global.ll
@@ -1,7 +1,7 @@
 ; RUN: sycl-post-link --device-globals --split=source -S < %s -o %t.files.table
-; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD0
-; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD1
-; RUN: FileCheck %s -input-file=%t.files_2.ll --check-prefix CHECK-MOD2
+; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD2
+; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD0
+; RUN: FileCheck %s -input-file=%t.files_2.ll --check-prefix CHECK-MOD1
 
 ; This test is intended to check that sycl-post-link generates no error if the
 ; 'device_image_scope' property is attached to not a device global variable.
diff --git a/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_no_dev_img_scope.ll b/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_no_dev_img_scope.ll
@@ -1,7 +1,7 @@
 ; RUN: sycl-post-link --device-globals --split=source -S < %s -o %t.files.table
-; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD0
-; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD1
-; RUN: FileCheck %s -input-file=%t.files_2.ll --check-prefix CHECK-MOD2
+; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD2
+; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD0
+; RUN: FileCheck %s -input-file=%t.files_2.ll --check-prefix CHECK-MOD1
 
 ; ModuleID = 'llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_no_dev_img_scope.ll'
 source_filename = "llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_no_dev_img_scope.ll"
diff --git a/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_two_vars_ok.ll b/llvm/test/tools/sycl-post-link/device-globals/test_global_variable_many_modules_two_vars_ok.ll
@@ -1,6 +1,6 @@
 ; RUN: sycl-post-link --device-globals --split=source -S < %s -o %t.files.table
-; RUN: FileCheck %s -input-file=%t.files_0.ll --check-prefix CHECK-MOD0
-; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD1
+; RUN: FileCheck %s -input-file=%t.files_1.ll --check-prefix CHECK-MOD0
+; RUN: FileCheck %s -input-file=%t.files_2.ll --check-prefix CHECK-MOD1
 
 ; This test is intended to check that sycl-post-link generates no errors
 ; when each device global variable with the 'device_image_scope' property
diff --git a/llvm/test/tools/sycl-post-link/device-requirements/aspects.ll b/llvm/test/tools/sycl-post-link/device-requirements/aspects.ll
@@ -17,8 +17,8 @@
 ; RUN: FileCheck %s -input-file=%t.files_0.prop --check-prefix CHECK-PROP-AUTO-SPLIT
 
 ; RUN: sycl-post-link -split=kernel < %s -o %t.files.table
-; RUN: FileCheck %s -input-file=%t.files_0.prop --check-prefix CHECK-PROP-KERNEL-SPLIT-0
-; RUN: FileCheck %s -input-file=%t.files_1.prop --check-prefix CHECK-PROP-KERNEL-SPLIT-1
+; RUN: FileCheck %s -input-file=%t.files_0.prop --check-prefix CHECK-PROP-KERNEL-SPLIT-1
+; RUN: FileCheck %s -input-file=%t.files_1.prop --check-prefix CHECK-PROP-KERNEL-SPLIT-0
 
 ; CHECK-PROP-AUTO-SPLIT: [SYCL/device requirements]
 ; CHECK-PROP-AUTO-SPLIT-NEXT: aspects=2|gCAAAAAAAAAAAAAABAAAAYAAAAQCAAAAMAAAAA
diff --git a/llvm/test/tools/sycl-post-link/emit_exported_symbols.ll b/llvm/test/tools/sycl-post-link/emit_exported_symbols.ll
@@ -6,17 +6,17 @@
 ;
 ; Per-module split
 ; RUN: sycl-post-link -symbols -split=source -emit-exported-symbols -S < %s -o %t.per_module.files.table
-; RUN: FileCheck %s -input-file=%t.per_module.files_0.prop -implicit-check-not="NotExported" --check-prefix=CHECK-PERMODULE-0-PROP
-; RUN: FileCheck %s -input-file=%t.per_module.files_1.prop -implicit-check-not="NotExported" --check-prefix=CHECK-KERNELONLY-PROP
+; RUN: FileCheck %s -input-file=%t.per_module.files_0.prop -implicit-check-not="NotExported" --check-prefix=CHECK-KERNELONLY-PROP
+; RUN: FileCheck %s -input-file=%t.per_module.files_1.prop -implicit-check-not="NotExported" --check-prefix=CHECK-PERMODULE-0-PROP
 ; RUN: FileCheck %s -input-file=%t.per_module.files_2.prop -implicit-check-not="NotExported" --check-prefix=CHECK-PERMODULE-2-PROP
 ;
 ; Per-kernel split
 ; RUN: sycl-post-link -symbols -split=kernel -emit-exported-symbols -S < %s -o %t.per_kernel.files.table
-; RUN: FileCheck %s -input-file=%t.per_kernel.files_0.prop --implicit-check-not="NotExported" --check-prefix=CHECK-PERKERNEL-0-PROP
-; RUN: FileCheck %s -input-file=%t.per_kernel.files_1.prop --implicit-check-not="NotExported" --check-prefix=CHECK-PERKERNEL-1-PROP
-; RUN: FileCheck %s -input-file=%t.per_kernel.files_2.prop --implicit-check-not="NotExported" --check-prefix=CHECK-PERKERNEL-2-PROP
-; RUN: FileCheck %s -input-file=%t.per_kernel.files_3.prop --implicit-check-not="NotExported" --check-prefix=CHECK-KERNELONLY-PROP
-; RUN: FileCheck %s -input-file=%t.per_kernel.files_4.prop --implicit-check-not="NotExported" --check-prefix=CHECK-KERNELONLY-PROP
+; RUN: FileCheck %s -input-file=%t.per_kernel.files_0.prop --implicit-check-not="NotExported" --check-prefix=CHECK-KERNELONLY-PROP
+; RUN: FileCheck %s -input-file=%t.per_kernel.files_1.prop --implicit-check-not="NotExported" --check-prefix=CHECK-KERNELONLY-PROP
+; RUN: FileCheck %s -input-file=%t.per_kernel.files_2.prop --implicit-check-not="NotExported" --check-prefix=CHECK-PERKERNEL-0-PROP
+; RUN: FileCheck %s -input-file=%t.per_kernel.files_3.prop --implicit-check-not="NotExported" --check-prefix=CHECK-PERKERNEL-1-PROP
+; RUN: FileCheck %s -input-file=%t.per_kernel.files_4.prop --implicit-check-not="NotExported" --check-prefix=CHECK-PERKERNEL-2-PROP
 
 target triple = "spir64-unknown-unknown"
 
diff --git a/llvm/test/tools/sycl-post-link/sycl-esimd-large-grf.ll b/llvm/test/tools/sycl-post-link/sycl-esimd-large-grf.ll
@@ -9,16 +9,16 @@
 
 ; RUN: sycl-post-link -split=source -symbols -split-esimd -lower-esimd -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t.table
-; RUN: FileCheck %s -input-file=%t_esimd_large_grf_1.ll --check-prefixes CHECK-ESIMD-LargeGRF-IR  --implicit-check-not='RegisterAllocMode'
-; RUN: FileCheck %s -input-file=%t_esimd_large_grf_1.prop --check-prefixes CHECK-ESIMD-LargeGRF-PROP
-; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-SYCL-SYM
-; RUN: FileCheck %s -input-file=%t_esimd_0.sym --check-prefixes CHECK-ESIMD-SYM
-; RUN: FileCheck %s -input-file=%t_esimd_large_grf_1.sym --check-prefixes CHECK-ESIMD-LargeGRF-SYM
+; RUN: FileCheck %s -input-file=%t_esimd_0.ll --check-prefixes CHECK-ESIMD-LargeGRF-IR  --implicit-check-not='RegisterAllocMode'
+; RUN: FileCheck %s -input-file=%t_esimd_0.prop --check-prefixes CHECK-ESIMD-LargeGRF-PROP
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-SYCL-SYM
+; RUN: FileCheck %s -input-file=%t_esimd_1.sym --check-prefixes CHECK-ESIMD-SYM
+; RUN: FileCheck %s -input-file=%t_esimd_0.sym --check-prefixes CHECK-ESIMD-LargeGRF-SYM
 
 ; CHECK: [Code|Properties|Symbols]
-; CHECK: {{.*}}esimd-large-grf.ll.tmp_0.ll|{{.*}}esimd-large-grf.ll.tmp_0.prop|{{.*}}esimd-large-grf.ll.tmp_0.sym
-; CHECK: {{.*}}esimd-large-grf.ll.tmp_esimd_0.ll|{{.*}}esimd-large-grf.ll.tmp_esimd_0.prop|{{.*}}esimd-large-grf.ll.tmp_esimd_0.sym
+; CHECK: {{.*}}_0.ll|{{.*}}_0.prop|{{.*}}_0.sym
 ; CHECK: {{.*}}_1.ll|{{.*}}_1.prop|{{.*}}_1.sym
+; CHECK: {{.*}}_esimd_1.ll|{{.*}}_esimd_1.prop|{{.*}}_esimd_1.sym
 
 ; CHECK-ESIMD-LargeGRF-PROP: isEsimdImage=1|1
 ; CHECK-ESIMD-LargeGRF-PROP: isLargeGRF=1|1
diff --git a/llvm/test/tools/sycl-post-link/sycl-large-grf.ll b/llvm/test/tools/sycl-post-link/sycl-large-grf.ll
@@ -9,13 +9,13 @@
 
 ; RUN: sycl-post-link -split=source -symbols -split-esimd -lower-esimd -S < %s -o %t.table
 ; RUN: FileCheck %s -input-file=%t.table
-; RUN: FileCheck %s -input-file=%t_large_grf_1.ll --check-prefixes CHECK-LARGE-GRF-IR
-; RUN: FileCheck %s -input-file=%t_large_grf_1.prop --check-prefixes CHECK-LARGE-GRF-PROP
-; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-SYCL-SYM
-; RUN: FileCheck %s -input-file=%t_large_grf_1.sym --check-prefixes CHECK-LARGE-GRF-SYM
+; RUN: FileCheck %s -input-file=%t_0.ll --check-prefixes CHECK-LARGE-GRF-IR
+; RUN: FileCheck %s -input-file=%t_0.prop --check-prefixes CHECK-LARGE-GRF-PROP
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-SYCL-SYM
+; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-LARGE-GRF-SYM
 
 ; CHECK: [Code|Properties|Symbols]
-; CHECK: {{.*}}-large-grf.ll.tmp_0.ll|{{.*}}-large-grf.ll.tmp_0.prop|{{.*}}-large-grf.ll.tmp_0.sym
+; CHECK: {{.*}}_0.ll|{{.*}}_0.prop|{{.*}}_0.sym
 ; CHECK: {{.*}}_1.ll|{{.*}}_1.prop|{{.*}}_1.sym
 
 ; CHECK-LARGE-GRF-PROP: isLargeGRF=1|1
diff --git a/llvm/test/tools/sycl-post-link/sycl-opt-level-external-funcs.ll b/llvm/test/tools/sycl-post-link/sycl-opt-level-external-funcs.ll
@@ -0,0 +1,68 @@
+; This test checks, that 'optLevel' property is only emitted based on the module
+; entry points
+;
+; In this test we have functions 'foo' and 'boo' defined in different
+; translation units. They are both entry points and 'foo' calls 'boo'.
+; As a result, we expect two modules:
+; - module with 'foo' (as entry point) and 'bar' (included as dependency) with
+;   'optLevel' set to 1 (taken from 'foo')
+; - module with 'bar' (as entry point) with 'optLevel' set to 2 (taken from
+;   'bar')
+
+; RUN: sycl-post-link -split=source -symbols -S < %s -o %t.table
+; RUN: FileCheck %s -input-file=%t.table
+; RUN: FileCheck %s -input-file=%t_0.prop --check-prefixes CHECK-OPT-LEVEL-PROP-0
+; RUN: FileCheck %s -input-file=%t_1.prop --check-prefixes CHECK-OPT-LEVEL-PROP-1
+; RUN: FileCheck %s -input-file=%t_0.sym --check-prefixes CHECK-SYM-0
+; RUN: FileCheck %s -input-file=%t_1.sym --check-prefixes CHECK-SYM-1
+; RUN: FileCheck %s -input-file=%t_0.ll --check-prefix CHECK-IR-0
+; RUN: FileCheck %s -input-file=%t_1.ll --check-prefix CHECK-IR-1
+
+; CHECK: [Code|Properties|Symbols]
+; CHECK: {{.*}}_0.ll|{{.*}}_0.prop|{{.*}}_0.sym
+; CHECK: {{.*}}_1.ll|{{.*}}_1.prop|{{.*}}_1.sym
+; CHECK-EMPTY:
+
+; CHECK-OPT-LEVEL-PROP-0: optLevel=1|1
+; CHECK-OPT-LEVEL-PROP-1: optLevel=1|2
+; CHECK-SYM-0: _Z3fooii
+; CHECK-SYM-0-EMPTY:
+; CHECK-SYM-1: _Z3barii
+;
+; CHECK-IR-0-DAG: define {{.*}} @_Z3fooii{{.*}} #[[#ATTR0:]]
+; CHECK-IR-0-DAG: define {{.*}} @_Z3barii{{.*}} #[[#ATTR1:]]
+; CHECK-IR-0-DAG: attributes #[[#ATTR0]] = { {{.*}} "sycl-optlevel"="1" }
+; CHECK-IR-0-DAG: attributes #[[#ATTR1]] = { {{.*}} "sycl-optlevel"="2" }
+;
+; CHECK-IR-1: define {{.*}} @_Z3barii{{.*}} #[[#ATTR0:]]
+; CHECK-IR-1: attributes #[[#ATTR0]] = { {{.*}} "sycl-optlevel"="2" }
+
+target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64"
+target triple = "spir64-unknown-unknown"
+
+define dso_local spir_func noundef i32 @_Z3fooii(i32 noundef %a, i32 noundef %b) local_unnamed_addr #0 {
+entry:
+  %call = call i32 @_Z3barii(i32 %a, i32 %b)
+  %sub = sub nsw i32 %a, %call
+  ret i32 %sub
+}
+
+define dso_local spir_func noundef i32 @_Z3barii(i32 noundef %a, i32 noundef %b) #1 {
+entry:
+  %retval = alloca i32, align 4
+  %a.addr = alloca i32, align 4
+  %b.addr = alloca i32, align 4
+  %retval.ascast = addrspacecast i32* %retval to i32 addrspace(4)*
+  %a.addr.ascast = addrspacecast i32* %a.addr to i32 addrspace(4)*
+  %b.addr.ascast = addrspacecast i32* %b.addr to i32 addrspace(4)*
+  store i32 %a, i32 addrspace(4)* %a.addr.ascast, align 4
+  store i32 %b, i32 addrspace(4)* %b.addr.ascast, align 4
+  %0 = load i32, i32 addrspace(4)* %a.addr.ascast, align 4
+  %1 = load i32, i32 addrspace(4)* %b.addr.ascast, align 4
+  %add = add nsw i32 %0, %1
+  ret i32 %add
+}
+
+attributes #0 = { mustprogress nofree norecurse nosync nounwind willreturn memory(none) "sycl-module-id"="test3.cpp" "sycl-optlevel"="1" }
+attributes #1 = { convergent mustprogress noinline norecurse nounwind optnone "sycl-module-id"="test2.cpp" "sycl-optlevel"="2" }
+
diff --git a/llvm/test/tools/sycl-post-link/sycl-opt-level.ll b/llvm/test/tools/sycl-post-link/sycl-opt-level.ll
diff --git a/llvm/tools/sycl-post-link/ModuleSplitter.cpp b/llvm/tools/sycl-post-link/ModuleSplitter.cpp
diff --git a/llvm/tools/sycl-post-link/ModuleSplitter.h b/llvm/tools/sycl-post-link/ModuleSplitter.h
diff --git a/llvm/tools/sycl-post-link/sycl-post-link.cpp b/llvm/tools/sycl-post-link/sycl-post-link.cpp
