-
Notifications
You must be signed in to change notification settings - Fork 15.2k
[mlir][vector] Add alignment attribute to vector operations. #152507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
a97cbc6
158b91c
36949d1
a4d820f
e2ad0f9
92b3886
482ad75
9b451db
3180cd0
47db5b1
a714521
0f81b89
be906aa
46d9010
87958eb
c80adf0
dcf38e2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -1382,6 +1382,11 @@ def Vector_TransferReadOp : | |
An additional `1` broadcast is required. On a GPU this broadcast could be | ||
implemented using a warp-shuffle if loop `j` were mapped to `threadIdx.x`. | ||
|
||
An optional `alignment` attribute allows to specify the byte alignment of the | ||
gather operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
|
||
Syntax | ||
``` | ||
operation ::= ssa-id `=` `vector.transfer_read` ssa-use-list | ||
|
@@ -1714,7 +1719,6 @@ def Vector_LoadOp : Vector_Op<"load", [ | |
load operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
A value of 0 indicates no specific alignment requirement. | ||
}]; | ||
|
||
let arguments = (ins Arg<AnyMemRef, "the reference to load from", | ||
|
@@ -1830,7 +1834,6 @@ def Vector_StoreOp : Vector_Op<"store", [ | |
store operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
A value of 0 indicates no specific alignment requirement. | ||
}]; | ||
|
||
let arguments = (ins | ||
|
@@ -1919,7 +1922,6 @@ def Vector_MaskedLoadOp : | |
load operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
A value of 0 indicates no specific alignment requirement. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you remind me what happens when alignment is not specified? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I originally wanted thought about removing this line since I imagined that the constructors using the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And just to double check - is There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In the original PR (#144344) the default parameter to one of the constructors is indeed zero, but the attribute is optional and the attribute linked to the operation is actually a CArg<"uint64_t", "0">:$alignment), [{
return build($_builder, $_state, memref, indices, nontemporal,
alignment != 0 ? $_builder.getI64IntegerAttr(alignment) :
nullptr); In PR #151690 the default parameter for these constructors was changed from a
Just to be complete, I believe in both cases the operation could have the field could be I think having the documentation indicate that a value of zero indicates no specific alignment requirements is still correct as the Operation's alignment field is still an integer (when present) and it being zero would still signifies no specific alignment requirements. I think we could also make the alignment attribute required by removing the OptionalAttr and then setting the alignment field point to I could also change the line to say that a value of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for the detailed explanation! Looks like things have become a bit complex recently 😅
I've just realised that we no longer support func.func @load_with_alignment(%memref : memref<200x100xf32>, %i : index, %j : index) -> vector<8xf32> {
%0 = vector.load %memref[%i, %j] { alignment = 0 } : memref<200x100xf32>, vector<8xf32>
return %0 : vector<8xf32>
} Now: $ bin/mlir-opt temp.mlir
temp.mlir:2:36: error: custom op 'vector.load' 'vector.load' op attribute 'alignment' failed to satisfy constraint: 64-bit signless integer attribute whose value is positive and whose value is a power of two > 0
%0 = vector.load %memref[%i, %j] { alignment = 0 } : memref<200x100xf32>, vector<8xf32>
^ So, zero-alignment is no longer valid :) This makes sense to me - otherwise To me, all of this calls for a few updates:
WDYT? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, you are right. Regarding next steps:
This is fine, I'll remove this line.
I think just updating the ones that are
Sure. Sounds good! Thanks @banach-space There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @banach-space I am not sure about predicates yet. I think as long as alignment is modeled as an integer attribute, If the only requirements are positive and power of two, then they are already self-descriptive. One could also have something like:
def IntPositivePowerOf2 : AllAttrOf<[IntPositive, IntPowerOf2]>;
class IntValidAlignment<Attr attr>: ConfinedAttr<attr, [IntPositivePowerOf2]>; What exactly do you propose as a predicate here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Basically, something like: def IntPositivePowerOf2 : AllAttrOf<[IntPositive, IntPowerOf2]>;
class IntValidAlignment<Attr attr>: ConfinedAttr<attr, [IntPositivePowerOf2]>; You are right that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was having some doubts because |
||
}]; | ||
let extraClassDeclaration = [{ | ||
MemRefType getMemRefType() { | ||
|
@@ -2012,7 +2014,6 @@ def Vector_MaskedStoreOp : | |
store operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
A value of 0 indicates no specific alignment requirement. | ||
}]; | ||
let extraClassDeclaration = [{ | ||
MemRefType getMemRefType() { | ||
|
@@ -2054,7 +2055,9 @@ def Vector_GatherOp : | |
Variadic<Index>:$indices, | ||
VectorOfNonZeroRankOf<[AnyInteger, Index]>:$index_vec, | ||
VectorOfNonZeroRankOf<[I1]>:$mask, | ||
AnyVectorOfNonZeroRank:$pass_thru)>, | ||
AnyVectorOfNonZeroRank:$pass_thru, | ||
ConfinedAttr<OptionalAttr<I64Attr>, | ||
[AllAttrOf<[IntPositive, IntPowerOf2]>]>:$alignment)>, | ||
Results<(outs AnyVectorOfNonZeroRank:$result)> { | ||
|
||
let summary = [{ | ||
|
@@ -2111,6 +2114,31 @@ def Vector_GatherOp : | |
"`into` type($result)"; | ||
let hasCanonicalizer = 1; | ||
let hasVerifier = 1; | ||
|
||
let builders = [ | ||
OpBuilder<(ins "VectorType":$resultType, | ||
"Value":$base, | ||
"ValueRange":$indices, | ||
"Value":$index_vec, | ||
"Value":$mask, | ||
"Value":$passthrough, | ||
CArg<"llvm::MaybeAlign", "llvm::MaybeAlign()">:$alignment), [{ | ||
return build($_builder, $_state, resultType, base, indices, index_vec, mask, passthrough, | ||
alignment.has_value() ? $_builder.getI64IntegerAttr(alignment->value()) : | ||
nullptr); | ||
}]>, | ||
OpBuilder<(ins "TypeRange":$resultTypes, | ||
"Value":$base, | ||
"ValueRange":$indices, | ||
"Value":$index_vec, | ||
"Value":$mask, | ||
"Value":$passthrough, | ||
CArg<"llvm::MaybeAlign", "llvm::MaybeAlign()">:$alignment), [{ | ||
return build($_builder, $_state, resultTypes, base, indices, index_vec, mask, passthrough, | ||
alignment.has_value() ? $_builder.getI64IntegerAttr(alignment->value()) : | ||
nullptr); | ||
}]> | ||
]; | ||
} | ||
|
||
def Vector_ScatterOp : | ||
|
@@ -2119,7 +2147,9 @@ def Vector_ScatterOp : | |
Variadic<Index>:$indices, | ||
VectorOfNonZeroRankOf<[AnyInteger, Index]>:$index_vec, | ||
VectorOfNonZeroRankOf<[I1]>:$mask, | ||
AnyVectorOfNonZeroRank:$valueToStore)> { | ||
AnyVectorOfNonZeroRank:$valueToStore, | ||
ConfinedAttr<OptionalAttr<I64Attr>, | ||
[AllAttrOf<[IntPositive, IntPowerOf2]>]>:$alignment)> { | ||
|
||
let summary = [{ | ||
scatters elements from a vector into memory as defined by an index vector | ||
|
@@ -2153,6 +2183,11 @@ def Vector_ScatterOp : | |
correspond to those of the `llvm.masked.scatter` | ||
[intrinsic](https://llvm.org/docs/LangRef.html#llvm-masked-scatter-intrinsics). | ||
|
||
An optional `alignment` attribute allows to specify the byte alignment of the | ||
scatter operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The violation is strangely defined here, why isn't this specified as UB? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this particular case, there we some operations defined in this other PR #144344 which used this wording as documentation. I think changing the wording to be undefined behaviour is reasonable. Would something like the following be preferrable?
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think
For the loads maybe we should instead use:
or something like that. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think immediate UB is more suitable since using aligned loads on unaligned pointers is known to crash on some architectures There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point, but that means we can't speculate an aligned load anymore right? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can open a PR to change the wording. Thanks @joker-eph ! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I checked LLVM and that seems like what we expect:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I personally prefer to avoid phrases like this unless we can provide a specific example. More generic terms like "UB" or "poison" tend to be more universal. |
||
|
||
Examples: | ||
|
||
```mlir | ||
|
@@ -2177,14 +2212,29 @@ def Vector_ScatterOp : | |
"type($index_vec) `,` type($mask) `,` type($valueToStore)"; | ||
let hasCanonicalizer = 1; | ||
let hasVerifier = 1; | ||
|
||
let builders = [ | ||
OpBuilder<(ins "Value":$base, | ||
"ValueRange":$indices, | ||
"Value":$index_vec, | ||
"Value":$mask, | ||
"Value":$valueToStore, | ||
CArg<"llvm::MaybeAlign", "llvm::MaybeAlign()">: $alignment), [{ | ||
return build($_builder, $_state, base, indices, index_vec, mask, valueToStore, | ||
alignment.has_value() ? $_builder.getI64IntegerAttr(alignment->value()) : | ||
nullptr); | ||
}]> | ||
]; | ||
} | ||
|
||
def Vector_ExpandLoadOp : | ||
Vector_Op<"expandload">, | ||
Arguments<(ins Arg<AnyMemRef, "", [MemRead]>:$base, | ||
Variadic<Index>:$indices, | ||
FixedVectorOfNonZeroRankOf<[I1]>:$mask, | ||
AnyVectorOfNonZeroRank:$pass_thru)>, | ||
AnyVectorOfNonZeroRank:$pass_thru, | ||
ConfinedAttr<OptionalAttr<I64Attr>, | ||
[AllAttrOf<[IntPositive, IntPowerOf2]>]>:$alignment)>, | ||
Results<(outs AnyVectorOfNonZeroRank:$result)> { | ||
|
||
let summary = "reads elements from memory and spreads them into a vector as defined by a mask"; | ||
|
@@ -2216,6 +2266,11 @@ def Vector_ExpandLoadOp : | |
correspond to those of the `llvm.masked.expandload` | ||
[intrinsic](https://llvm.org/docs/LangRef.html#llvm-masked-expandload-intrinsics). | ||
|
||
An optional `alignment` attribute allows to specify the byte alignment of the | ||
load operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
|
||
Note, at the moment this Op is only available for fixed-width vectors. | ||
|
||
Examples: | ||
|
@@ -2246,14 +2301,39 @@ def Vector_ExpandLoadOp : | |
"type($base) `,` type($mask) `,` type($pass_thru) `into` type($result)"; | ||
let hasCanonicalizer = 1; | ||
let hasVerifier = 1; | ||
|
||
let builders = [ | ||
OpBuilder<(ins "VectorType":$resultType, | ||
"Value":$base, | ||
"ValueRange":$indices, | ||
"Value":$mask, | ||
"Value":$passthrough, | ||
CArg<"llvm::MaybeAlign", "llvm::MaybeAlign()">:$alignment), [{ | ||
return build($_builder, $_state, resultType, base, indices, mask, passthrough, | ||
alignment.has_value() ? $_builder.getI64IntegerAttr(alignment->value()) : | ||
nullptr); | ||
}]>, | ||
OpBuilder<(ins "TypeRange":$resultTypes, | ||
"Value":$base, | ||
"ValueRange":$indices, | ||
"Value":$mask, | ||
"Value":$passthrough, | ||
CArg<"llvm::MaybeAlign", "llvm::MaybeAlign()">:$alignment), [{ | ||
return build($_builder, $_state, resultTypes, base, indices, mask, passthrough, | ||
alignment.has_value() ? $_builder.getI64IntegerAttr(alignment->value()) : | ||
nullptr); | ||
}]> | ||
]; | ||
banach-space marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
def Vector_CompressStoreOp : | ||
Vector_Op<"compressstore">, | ||
Arguments<(ins Arg<AnyMemRef, "", [MemWrite]>:$base, | ||
Variadic<Index>:$indices, | ||
FixedVectorOfNonZeroRankOf<[I1]>:$mask, | ||
AnyVectorOfNonZeroRank:$valueToStore)> { | ||
AnyVectorOfNonZeroRank:$valueToStore, | ||
ConfinedAttr<OptionalAttr<I64Attr>, | ||
[AllAttrOf<[IntPositive, IntPowerOf2]>]>:$alignment)> { | ||
|
||
let summary = "writes elements selectively from a vector as defined by a mask"; | ||
|
||
|
@@ -2284,6 +2364,11 @@ def Vector_CompressStoreOp : | |
correspond to those of the `llvm.masked.compressstore` | ||
[intrinsic](https://llvm.org/docs/LangRef.html#llvm-masked-compressstore-intrinsics). | ||
|
||
An optional `alignment` attribute allows to specify the byte alignment of the | ||
store operation. It must be a positive power of 2. The operation must access | ||
memory at an address aligned to this boundary. Violations may lead to | ||
architecture-specific faults or performance penalties. | ||
|
||
Note, at the moment this Op is only available for fixed-width vectors. | ||
|
||
Examples: | ||
|
@@ -2312,6 +2397,17 @@ def Vector_CompressStoreOp : | |
"type($base) `,` type($mask) `,` type($valueToStore)"; | ||
let hasCanonicalizer = 1; | ||
let hasVerifier = 1; | ||
let builders = [ | ||
OpBuilder<(ins "Value":$base, | ||
"ValueRange":$indices, | ||
"Value":$mask, | ||
"Value":$valueToStore, | ||
CArg<"llvm::MaybeAlign", "llvm::MaybeAlign()">:$alignment), [{ | ||
return build($_builder, $_state, base, indices, valueToStore, mask, | ||
alignment.has_value() ? $_builder.getI64IntegerAttr(alignment->value()) : | ||
nullptr); | ||
}]> | ||
]; | ||
} | ||
|
||
def Vector_ShapeCastOp : | ||
|
Uh oh!
There was an error while loading. Please reload this page.