-
Notifications
You must be signed in to change notification settings - Fork 15.1k
[Clang] Introduce OverflowBehaviorType for fine-grained overflow control #148914
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
@llvm/pr-subscribers-lldb @llvm/pr-subscribers-clang-driver Author: Justin Stitt (JustinStitt) ChangesIntroduce The new
A key aspect of this feature is its interaction with existing mechanisms. This implementation also includes specific promotion rules to preserve the intended overflow behavior within expressions and new diagnostics to warn against accidentally discarding the attribute. See the docs and tests for examples and more info. RFCsCCs (because of your participation in RFCs or previous implementations)@kees @AaronBallman @vitalybuka @melver @efriedma-quic Patch is 138.97 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/148914.diff 66 Files Affected:
diff --git a/clang/docs/OverflowBehaviorTypes.rst b/clang/docs/OverflowBehaviorTypes.rst
new file mode 100644
index 0000000000000..18e337e181e8a
--- /dev/null
+++ b/clang/docs/OverflowBehaviorTypes.rst
@@ -0,0 +1,269 @@
+=====================
+OverflowBehaviorTypes
+=====================
+
+.. contents::
+ :local:
+
+Introduction
+============
+
+Clang provides a type attribute that allows developers to have fine-grained control
+over the overflow behavior of integer types. The ``overflow_behavior``
+attribute can be used to specify how arithmetic operations on a given integer
+type should behave upon overflow. This is particularly useful for projects that
+need to balance performance and safety, allowing developers to enable or
+disable overflow checks for specific types.
+
+The attribute can be enabled using the compiler option
+``-foverflow-behavior-types``.
+
+The attribute syntax is as follows:
+
+.. code-block:: c++
+
+ __attribute__((overflow_behavior(behavior)))
+
+Where ``behavior`` can be one of the following:
+
+* ``wrap``: Specifies that arithmetic operations on the integer type should
+ wrap on overflow. This is equivalent to the behavior of ``-fwrapv``, but it
+ applies only to the attributed type and may be used with both signed and
+ unsigned types. When this is enabled, UBSan's integer overflow and integer
+ truncation checks (``signed-integer-overflow``,
+ ``unsigned-integer-overflow``, ``implicit-signed-integer-truncation``, and
+ ``implicit-unsigned-integer-truncation``) are suppressed for the attributed
+ type.
+
+* ``no_wrap``: Specifies that arithmetic operations on the integer type should
+ be checked for overflow. When using the ``signed-integer-overflow`` sanitizer
+ or when using ``-ftrapv`` alongside a signed type, this is the default
+ behavior. Using this, one may enforce overflow checks for a type even when
+ ``-fwrapv`` is enabled globally.
+
+This attribute can be applied to ``typedef`` declarations and to integer types directly.
+
+Examples
+========
+
+Here is an example of how to use the ``overflow_behavior`` attribute with a ``typedef``:
+
+.. code-block:: c++
+
+ typedef unsigned int __attribute__((overflow_behavior(no_wrap))) non_wrapping_uint;
+
+ non_wrapping_uint add_one(non_wrapping_uint a) {
+ return a + 1; // Overflow is checked for this operation.
+ }
+
+Here is an example of how to use the ``overflow_behavior`` attribute with a type directly:
+
+.. code-block:: c++
+
+ int mul_alot(int n) {
+ int __attribute__((overflow_behavior(wrap))) a = n;
+ return a * 1337; // Potential overflow is not checked and is well-defined
+ }
+
+"Well-defined" overflow is consistent with two's complement wrap-around
+semantics and won't be removed via eager compiler optimizations (like some
+undefined behavior might).
+
+Overflow behavior types are implicitly convertible to and from built-in
+integral types.
+
+Note that C++ overload set formation rules treat promotions to and from
+overflow behavior types the same as normal integral promotions and conversions.
+
+Interaction with Command-Line Flags and Sanitizer Special Case Lists
+====================================================================
+
+The ``overflow_behavior`` attribute interacts with sanitizers, ``-ftrapv``,
+``-fwrapv``, and Sanitizer Special Case Lists (SSCL) by wholly overriding these
+global flags. The following table summarizes the interactions:
+
+.. list-table:: Overflow Behavior Precedence
+ :widths: 15 15 15 15 20 15
+ :header-rows: 1
+
+ * - Behavior
+ - Default(No Flags)
+ - -ftrapv
+ - -fwrapv
+ - Sanitizers
+ - SSCL
+ * - ``overflow_behavior(wrap)``
+ - Wraps
+ - No trap
+ - Wraps
+ - No report
+ - Overrides SSCL
+ * - ``overflow_behavior(no_wrap)``
+ - Traps
+ - Traps
+ - Traps
+ - Reports
+ - Overrides SSCL
+
+It is important to note the distinction between signed and unsigned types. For
+unsigned integers, which wrap on overflow by default, ``overflow_behavior(no_wrap)``
+is particularly useful for enabling overflow checks. For signed integers, whose
+overflow behavior is undefined by default, ``overflow_behavior(wrap)`` provides
+a guaranteed wrapping behavior.
+
+The ``overflow_behavior`` attribute can be used to override the behavior of
+entries from a :doc:`SanitizerSpecialCaseList`. This is useful for allowlisting
+specific types into overflow instrumentation.
+
+Promotion Rules
+===============
+
+The promotion rules for overflow behavior types are designed to preserve the
+specified overflow behavior throughout an arithmetic expression. They differ
+from standard C/C++ integer promotions but in a predictable way, similar to
+how ``_Complex`` and ``_BitInt`` have their own promotion rules.
+
+* **OBT and Standard Integer Type**: In an operation involving an overflow
+ behavior type (OBT) and a standard integer type, the result will have the
+ type of the OBT, including its overflow behavior, sign, and bit-width. The
+ standard integer type is implicitly converted to match the OBT.
+
+ .. code-block:: c++
+
+ typedef char __attribute__((overflow_behavior(no_wrap))) no_wrap_char;
+ // The result of this expression is no_wrap_char.
+ no_wrap_char c;
+ unsigned long ul;
+ auto result = c + ul;
+
+* **Two OBTs of the Same Kind**: When an operation involves two OBTs of the
+ same kind (e.g., both ``wrap``), the result will have the larger of the two
+ bit-widths. If the bit-widths are the same, an unsigned type is favored over
+ a signed one.
+
+ .. code-block:: c++
+
+ typedef unsigned char __attribute__((overflow_behavior(wrap))) u8_wrap;
+ typedef unsigned short __attribute__((overflow_behavior(wrap))) u16_wrap;
+ // The result of this expression is u16_wrap.
+ u8_wrap a;
+ u16_wrap b;
+ auto result = a + b;
+
+* **Two OBTs of Different Kinds**: In an operation between a ``wrap`` and a
+ ``no_wrap`` type, a ``no_wrap`` is produced. It is recommended to avoid such
+ operations, as Clang may emit a warning for such cases in the future.
+ Regardless, the resulting type matches the bit-width, sign and behavior of
+ the ``no_wrap`` type.
+
+Diagnostics
+===========
+
+Clang provides diagnostics to help developers manage overflow behavior types.
+
+-Wimplicitly-discarded-overflow-behavior
+----------------------------------------
+
+This warning is issued when an overflow behavior type is implicitly converted
+to a standard integer type, which may lead to the loss of the specified
+overflow behavior.
+
+.. code-block:: c++
+
+ typedef int __attribute__((overflow_behavior(wrap))) wrapping_int;
+
+ void some_function(int);
+
+ void another_function(wrapping_int w) {
+ some_function(w); // warning: implicit conversion from 'wrapping_int' to
+ // 'int' discards overflow behavior
+ }
+
+To fix this, you can explicitly cast the overflow behavior type to a standard
+integer type.
+
+.. code-block:: c++
+
+ typedef int __attribute__((overflow_behavior(wrap))) wrapping_int;
+
+ void some_function(int);
+
+ void another_function(wrapping_int w) {
+ some_function(static_cast<int>(w)); // OK
+ }
+
+This warning acts as a group that includes
+``-Wimplicitly-discarded-overflow-behavior-pedantic`` and
+``-Wimplicitly-discarded-overflow-behavior-assignment``.
+
+-Wimplicitly-discarded-overflow-behavior-pedantic
+-------------------------------------------------
+
+A less severe version of the warning, ``-Wimplicitly-discarded-overflow-behavior-pedantic``,
+is issued for implicit conversions from an unsigned wrapping type to a standard
+unsigned integer type. This is considered less problematic because both types
+have well-defined wrapping behavior, but the conversion still discards the
+explicit ``overflow_behavior`` attribute.
+
+.. code-block:: c++
+
+ typedef unsigned int __attribute__((overflow_behavior(wrap))) wrapping_uint;
+
+ void some_function(unsigned int);
+
+ void another_function(wrapping_uint w) {
+ some_function(w); // warning: implicit conversion from 'wrapping_uint' to
+ // 'unsigned int' discards overflow behavior
+ // [-Wimplicitly-discarded-overflow-behavior-pedantic]
+ }
+
+-Wimplicitly-discarded-overflow-behavior-assignment
+---------------------------------------------------
+
+This warning is issued when an overflow behavior type is implicitly converted
+to a standard integer type as part of an assignment, which may lead to the
+loss of the specified overflow behavior. This is a more specific version of
+the ``-Wimplicitly-discarded-overflow-behavior`` warning, and it is off by
+default.
+
+.. code-block:: c++
+
+ typedef int __attribute__((overflow_behavior(wrap))) wrapping_int;
+
+ void some_function() {
+ wrapping_int w = 1;
+ int i = w; // warning: implicit conversion from 'wrapping_int' to 'int'
+ // discards overflow behavior
+ // [-Wimplicitly-discarded-overflow-behavior-assignment]
+ }
+
+To fix this, you can explicitly cast the overflow behavior type to a standard
+integer type.
+
+.. code-block:: c++
+
+ typedef int __attribute__((overflow_behavior(wrap))) wrapping_int;
+
+ void some_function() {
+ wrapping_int w = 1;
+ int i = static_cast<int>(w); // OK
+ int j = (int)w; // C-style OK
+ }
+
+
+-Woverflow-behavior-attribute-ignored
+-------------------------------------
+
+This warning is issued when the ``overflow_behavior`` attribute is applied to
+a type that is not an integer type.
+
+.. code-block:: c++
+
+ typedef float __attribute__((overflow_behavior(wrap))) wrapping_float;
+ // warning: 'overflow_behavior' attribute only applies to integer types;
+ // attribute is ignored [-Woverflow-behavior-attribute-ignored]
+
+ typedef struct S { int i; } __attribute__((overflow_behavior(wrap))) S_t;
+ // warning: 'overflow_behavior' attribute only applies to integer types;
+ // attribute is ignored [-Woverflow-behavior-attribute-ignored]
+
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 970825c98fec1..0c9139cb252d2 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -378,6 +378,8 @@ New Compiler Flags
- New options ``-g[no-]key-instructions`` added, disabled by default. Reduces jumpiness of debug stepping for optimized code in some debuggers (not LLDB at this time). Not recommended for use without optimizations. DWARF only. Note both the positive and negative flags imply ``-g``.
+- New option ``-foverflow-behavior-types`` added to enable parsing of the ``overflow_behavior`` type attribute.
+
Deprecated Compiler Flags
-------------------------
@@ -478,6 +480,10 @@ related warnings within the method body.
- Clang will print the "reason" string argument passed on to
``[[clang::warn_unused_result("reason")]]`` as part of the warning diagnostic.
+- Introduced a new type attribute ``__attribute__((overflow_behavior))`` which
+ currently accepts either ``wrap`` or ``no_wrap`` as an argument, enabling
+ type-level control over overflow behavior.
+
Improvements to Clang's diagnostics
-----------------------------------
diff --git a/clang/docs/SanitizerSpecialCaseList.rst b/clang/docs/SanitizerSpecialCaseList.rst
index 2c50778d0f491..f4c9274b89b68 100644
--- a/clang/docs/SanitizerSpecialCaseList.rst
+++ b/clang/docs/SanitizerSpecialCaseList.rst
@@ -133,6 +133,40 @@ precedence. Here are a few examples.
fun:*bar
fun:bad_bar=sanitize
+Interaction with Overflow Behavior Types
+----------------------------------------
+
+The ``overflow_behavior`` attribute provides a more granular, source-level
+control that takes precedence over the Sanitizer Special Case List. If a type
+is given an ``overflow_behavior`` attribute, it will override any matching
+``type:`` entry in a special case list.
+
+This allows developers to enforce a specific overflow behavior for a critical
+type, even if a broader rule in the special case list would otherwise disable
+instrumentation for it.
+
+.. code-block:: bash
+
+ $ cat ignorelist.txt
+ # Disable signed overflow checks for all types by default.
+ [signed-integer-overflow]
+ type:*
+
+ $ cat foo.c
+ // Force 'critical_type' to always have overflow checks,
+ // overriding the ignorelist.
+ typedef int __attribute__((overflow_behavior(no_wrap))) critical_type;
+
+ void foo(int x) {
+ critical_type a = x;
+ a++; // Overflow is checked here due to the 'no_wrap' attribute.
+
+ int b = x;
+ b++; // Overflow is NOT checked here due to the ignorelist.
+ }
+
+For more details on overflow behavior types, see :doc:`OverflowBehaviorTypes`.
+
Format
======
diff --git a/clang/docs/UndefinedBehaviorSanitizer.rst b/clang/docs/UndefinedBehaviorSanitizer.rst
index 0a2d833783e57..f98eda1e9399c 100644
--- a/clang/docs/UndefinedBehaviorSanitizer.rst
+++ b/clang/docs/UndefinedBehaviorSanitizer.rst
@@ -380,6 +380,28 @@ This attribute may not be
supported by other compilers, so consider using it together with
``#if defined(__clang__)``.
+Disabling Overflow Instrumentation with ``__attribute__((overflow_behavior(wrap)))``
+------------------------------------------------------------------------------------
+
+For more fine-grained control over how integer overflow is handled, you can use
+the ``__attribute__((overflow_behavior(wrap)))`` attribute. This attribute can
+be applied to ``typedef`` declarations and integer types to specify that
+arithmetic operations on that type should wrap on overflow. This can be used to
+disable overflow sanitization for specific types, while leaving it enabled for
+all other types.
+
+For more information, see :doc:`OverflowBehaviorTypes`.
+
+Enforcing Overflow Instrumentation with ``__attribute__((overflow_behavior(no_wrap)))``
+---------------------------------------------------------------------------------------
+
+Conversely, you can use ``__attribute__((overflow_behavior(no_wrap)))`` to
+enforce overflow checks for a specific type, even when ``-fwrapv`` is enabled
+globally. This is useful for ensuring that critical calculations are always
+checked for overflow, regardless of the global compiler settings.
+
+For more information, see :doc:`OverflowBehaviorTypes`.
+
Suppressing Errors in Recompiled Code (Ignorelist)
--------------------------------------------------
diff --git a/clang/docs/index.rst b/clang/docs/index.rst
index 6c792af66a62c..3205709aa395a 100644
--- a/clang/docs/index.rst
+++ b/clang/docs/index.rst
@@ -40,6 +40,7 @@ Using Clang as a Compiler
SanitizerCoverage
SanitizerStats
SanitizerSpecialCaseList
+ OverflowBehaviorTypes
BoundsSafety
BoundsSafetyAdoptionGuide
BoundsSafetyImplPlans
diff --git a/clang/include/clang/AST/ASTContext.h b/clang/include/clang/AST/ASTContext.h
index 2b9cd035623cc..dbfc8df6fba5e 100644
--- a/clang/include/clang/AST/ASTContext.h
+++ b/clang/include/clang/AST/ASTContext.h
@@ -14,6 +14,7 @@
#ifndef LLVM_CLANG_AST_ASTCONTEXT_H
#define LLVM_CLANG_AST_ASTCONTEXT_H
+#include "Type.h"
#include "clang/AST/ASTFwd.h"
#include "clang/AST/CanonicalType.h"
#include "clang/AST/CommentCommandTraits.h"
@@ -259,6 +260,7 @@ class ASTContext : public RefCountedBase<ASTContext> {
mutable llvm::ContextualFoldingSet<DependentBitIntType, ASTContext &>
DependentBitIntTypes;
mutable llvm::FoldingSet<BTFTagAttributedType> BTFTagAttributedTypes;
+ mutable llvm::FoldingSet<OverflowBehaviorType> OverflowBehaviorTypes;
llvm::FoldingSet<HLSLAttributedResourceType> HLSLAttributedResourceTypes;
llvm::FoldingSet<HLSLInlineSpirvType> HLSLInlineSpirvTypes;
@@ -899,6 +901,8 @@ class ASTContext : public RefCountedBase<ASTContext> {
bool isTypeIgnoredBySanitizer(const SanitizerMask &Mask,
const QualType &Ty) const;
+ bool isUnaryOverflowPatternExcluded(const UnaryOperator *UO);
+
const XRayFunctionFilter &getXRayFilter() const {
return *XRayFilter;
}
@@ -999,6 +1003,15 @@ class ASTContext : public RefCountedBase<ASTContext> {
comments::FullComment *getCommentForDecl(const Decl *D,
const Preprocessor *PP) const;
+ /// Attempts to merge two types that may be OverflowBehaviorTypes.
+ ///
+ /// \returns A QualType if the types were handled, std::nullopt otherwise.
+ /// A null QualType indicates an incompatible merge.
+ std::optional<QualType>
+ tryMergeOverflowBehaviorTypes(QualType LHS, QualType RHS, bool OfBlockPointer,
+ bool Unqualified, bool BlockReturnType,
+ bool IsConditionalOperator);
+
/// Return parsed documentation comment attached to a given declaration.
/// Returns nullptr if no comment is attached. Does not look at any
/// redeclarations of the declaration.
@@ -1843,6 +1856,13 @@ class ASTContext : public RefCountedBase<ASTContext> {
QualType getBTFTagAttributedType(const BTFTypeTagAttr *BTFAttr,
QualType Wrapped) const;
+ QualType getOverflowBehaviorType(const OverflowBehaviorAttr *Attr,
+ QualType Wrapped) const;
+
+ QualType
+ getOverflowBehaviorType(OverflowBehaviorType::OverflowBehaviorKind Kind,
+ QualType Wrapped) const;
+
QualType getHLSLAttributedResourceType(
QualType Wrapped, QualType Contained,
const HLSLAttributedResourceType::Attributes &Attrs);
diff --git a/clang/include/clang/AST/ASTNodeTraverser.h b/clang/include/clang/AST/ASTNodeTraverser.h
index 8ebabb2bde10d..e684dcd5c27e8 100644
--- a/clang/include/clang/AST/ASTNodeTraverser.h
+++ b/clang/include/clang/AST/ASTNodeTraverser.h
@@ -445,6 +445,9 @@ class ASTNodeTraverser
void VisitBTFTagAttributedType(const BTFTagAttributedType *T) {
Visit(T->getWrappedType());
}
+ void VisitOverflowBehaviorType(const OverflowBehaviorType *T) {
+ Visit(T->getUnderlyingType());
+ }
void VisitHLSLAttributedResourceType(const HLSLAttributedResourceType *T) {
QualType Contained = T->getContainedType();
if (!Contained.isNull())
diff --git a/clang/include/clang/AST/Expr.h b/clang/include/clang/AST/Expr.h
index 523c0326d47ef..4fbea7272d004 100644
--- a/clang/include/clang/AST/Expr.h
+++ b/clang/include/clang/AST/Expr.h
@@ -1477,6 +1477,14 @@ class DeclRefExpr final
return DeclRefExprBits.IsImmediateEscalating;
}
+ bool isOverflowBehaviorDiscarded() const {
+ return DeclRefExprBits.IsOverflwBehaviorDiscarded;
+ }
+
+ void setOverflowBehaviorDiscarded(bool Set) {
+ DeclRefExprBits.IsOverflwBehaviorDiscarded = Set;
+ }
+
void setIsImmediateEscalating(bool Set) {
DeclRefExprBits.IsImmediateEscalating = Set;
}
diff --git a/clang/include/clang/AST/PropertiesBase.td b/clang/include/clang/AST/PropertiesBase.td
index 1215056ffde1b..26dd53760c3a2 100644
--- a/clang/include/clang/AST/PropertiesBase.td
+++ b/clang/include/clang/AST/PropertiesBase.td
@@ -81,6 +81,8 @@ def AutoTypeKeyword : EnumPropertyType;
def Bool : PropertyType<"bool">;
def BuiltinTypeKind : EnumPropertyType<"BuiltinType::Kind">;
def BTFTypeTagAttr : PropertyType<"const BTFTypeTagAttr *">;
+def OverflowBehaviorKind
+ : EnumPropertyType<"OverflowBehaviorType::OverflowBehaviorKind">;
def CallingConv : EnumPropertyType;
def DeclarationName : PropertyType;
def DeclarationNameKind : EnumPropertyType<"DeclarationName::NameKind">;
diff --git a/clang/include/clang/AST/RecursiveASTVisitor.h b/clang/include/clang/AST/RecursiveASTVisitor.h
index 5cb2f57edffe4..4a681a58b1cc9 100644
--- a/clang/include/clang/AST/RecursiveASTVisitor.h
+++ b/clang/include/clang/AST/RecursiveASTVisitor.h
@@ -1151,6 +1151,9 @@ DEF_TRAVERSE_TYPE(CountAttributedType, {
DEF_TRAVERSE_TYPE(BTFTagAttributedType,
{ TRY_TO(TraverseType(T->getWrappedType())); })
+DEF_TRAVERSE_TYPE(OverflowBehaviorType,
+ { TRY_TO(TraverseType(T->getUnderlyingType())); })
+
DEF_TRAVERSE_TYPE(HLSLAttributedResourceType,
{ TRY_TO(TraverseType(T->getWrappedType())); })
@@ -1462,6 +1465,9 @@ DEF_TRAVERSE_TYPELOC(CountAttributedType,
DEF_TRAVERSE_TYPELOC(BTFTagAttributedType,
{ TRY_TO(TraverseTypeLoc(TL.getWrappedLoc())); })
+DEF_TRAVERSE_TYPELOC(OverflowBehaviorType,
+ { TRY_TO(TraverseTypeLoc(TL.getWrappedLoc())); })
+
DEF_TRAVERSE_TYPELOC(HLSLAttributedResourceType,
...
[truncated]
|
fb45ed3
to
c301e29
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple comments from a quick glance.
645069d
to
a5d8ad5
Compare
gentle ping. |
fad8d73
to
579a796
Compare
ping. I rebased due to some small conflicts. question to reviewers: Should I squash these ~10 commits into one for ease of review? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments, I will try a more through review later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about inner qualifiers? The patch doesn't address the situation.
For a test like:
using ConstInt = const int;
using WrapConstInt1 = ConstInt __attribute__((overflow_behavior(wrap)));
using WrapConstInt2 = const int __attribute__((overflow_behavior(wrap)));
WrapConstInt1 and WrapConstInt2 should be the same type, as this is what makes most sense, I think.
This would be the same rules as qualifiers on array types, the inner and outer qualifiers mean the same thing and are added together for canonicalization.
thanks @mizvekov for the initial review. I've tried to address all your concerns with commit As for your comment:
Do you think maybe that should be another PR that I can send later on? Should it come before this PR? Where do you think a better home is for that overflow pattern elision stuff? And for your inner qualifiers question:
How should we address these? FWIW, currently, the two types in your example do not compare equal: std::cout << std::boolalpha
<< std::is_same<WrapConstInt1, WrapConstInt2>::value << "\n"; // false I am not sure what you mean by "inner qualifiers" from your example. My understanding of inner qualifiers involves pointer types like |
I think this is a simple enough change, it's just about moving where the function is implemented. I think Type.h would be a better fit, but I'd wait a little bit for other opinions before changing anything.
So there are two positions qualifiers can appear in this type.
It doesn't seem plausible to me that these qualifiers have different meanings, that is to say, that the two types in my example should be different types, instead of two spellings of the same type. One solution here would be to adopt the same rules as ArrayTypes for this situation. You can implement this by changing your canonicalization for |
I'm getting conflicting ideas from this sentence. Do you mean to say that the two types from your example should be the same type? |
1a8ad6d
to
0042195
Compare
clang/lib/AST/ASTDiagnostic.cpp
Outdated
return DecoratedString; | ||
} | ||
|
||
TryConvertOverflowBehaviorTypeToDiagnosticString(Context, Ty, S); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you feel the full attribute syntax is too ugly to print in diagnostics, it's probably too ugly to type in source code anyway :)
We should perhaps reconsider the spelling then.
Note this is not easy to get right, for example if the function above has to do something special about PointerType, why wouldn't it have to do anything special about ReferenceTypes, and the other strange things we have to support, like objective-c block pointers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so should I drop this special handling and store it in a different branch for another time? FYI, if we go with pure attribute printing there will be a lot of churn in my tests as they all have the short-hand types.
BTW, I think we decided not to go with making __wrap
and __no_wrap
keywords as maybe some projects already use these from the preprocessor? I am not sure on any precedence between when to make something attribute-only v.s. when to introduce a keyword.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you feel the full attribute syntax is too ugly to print in diagnostics, it's probably too ugly to type in source code anyway :)
I suppose I expected most immediate users of this feature to define a macro for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These names starting with double underscores are reserved for the implementation anyway, we wouldn't care about user code using them, unless there was a very exceptional circumstance, like if they were being misused broadly, but then we could just pick a slightly different spelling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you would like to proceed with the attribute spelling anyway, that's okay.
I guess you have the ext_vector precedent, even though I don't particularly like that.
I just don't think the current approach of producing these shorthand notations is workable, if it has to handle all the complexities of how these types can be nested in other types.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ojhunt Can we move forward with a keyword spelling or do you strongly object to it? I am actively working on your other feedback regarding converting some warnings to errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming the existence of the narrowing behavior I think that this burns the keywords for a non-narrowing overflow behaviour selection that I believe is more useful, adoptable, and lines up with what I have heard many many developers ask for.
I recognize that the RFC approved these semantics, but I don't think these semantics are useful enough to burn these basic keywords, and I don't believe these keywords accurately represent their semantics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, that last comment may have been nonsense -- I thought this was the keyword thread I restarted, and I saw there's been a commit to change the names which I need to read through.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I just looked at the change to ob_wrap
and ob_trap
and that resolves my issues with the keywords.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah sorry I pushed that commit then went to lunch. I should've linked to that here. But thanks for catching it.
Signed-off-by: Justin Stitt <[email protected]>
0042195
to
0bee82e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks much better, thanks!
Single nit, else LGTM.
In looking at what's here, I suddenly realize that I think we accidentally deviated a bit on naming, in that the trap behavior got named "no_wrap" which is ambiguous. For example, a future overflow resolution behavior can be added like "saturate", which isn't wrapping either, and suddenly "no_wrap" isn't clear what it's doing. The "no_wrap" runtime overflow resolution is to trap (which is correct here), so I think it should be explicitly named "trap" instead of "no_wrap". (If there was a reason I've forgotten for not naming it "trap", though, please ignore me there -- I don't want to stall this with bikeshedding.) Additionally, just for some namespace sanity (especially from the perspective of Linux's use of OBTs), can we have the type specifier named "_ob$behavior": __ob_wrap, __ob_trap ? That would make things a bit cleaner for Linux's resulting macros for OBT usage to de-conflict possible namespace collisions there... |
The type specifier is optional, you can still use the attribute syntax here, so if you are hiding this behind macros anyway, might as well just use that spelling instead? Unless I have misunderstood. |
I guess I'm thinking about 2 aspects: 1) "__wrap" is a very short name, and it feels like it is begging to collide or at least be confusing. 2) Linux has a common way to detect and enable/disable compiler features via macros that may expand to a given attribute (e.g. even if a compiler feature is available, Linux may want to disable it at the source level). I'd like to avoid confusion there. |
I see. Well on one hand, we already have a bunch of implementation specific keywords that are very small like that. The latest one which is The thinking here is that the language spec formally reserves these identifiers for the implementation, so in theory this shouldn't be a problem, and compiler implementors will often take this liberty without a second thought. @ojhunt @AaronBallman and @erichkeane have more or less recently added new such keywords, so I ask them, what's our criteria here, what sort of precedent we have? Certainly we don't seem to be consistent whether we pick a _Capital or a __snake style spelling, would that make any difference for the linux kernel, if they were spelled _Wrap and _Trap instead? |
Formally, I don't think there's any concern; all reserved identifiers are equally reserved. In practice, libc implementations and other libraries tend to stomp over the reserved namespace for common words, so it's probably worth doing a GitHub search or something. |
I think naming collisions are a secondary issue - mentioned in Kees' comments. I think the main issue is that "no_wrap" ought to be called "trap" everywhere. I am fine with the renaming... I already have a patch to do just that too. I am curious what others think. |
Yeah renaming no_wrap to trap is good, thanks for catching that. |
Yeah, my primary review note was generally no_wrap -> trap. The other aspect of the |
@mizvekov Any objections to I'm getting test failures because
|
Yeah, that looks fine to me. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't believe this warrants a keyword, and I don't believe the RFC supports the addition of such. As far as I can make out all the RFC discussion is in terms of an attribute.
The attribute spellings are concerning as well - no_wrap
to me implies trapping behavior, but the implementation seems to mean "remain UB".
The separate tracking of wrap and no_wrap is confusing - it means a type can end up being both wrap and no_wrap and creates the potential for equivalent appearing tests (wrap
vs !no_wrap
) to not actually be equivalent. This should really just be an enum of wrap
, no_wrap
, unspecified
and that resolves the issue entirely.
The type conversion logic is kind of confusing to me - I'm worried that it is doing some of its own type comparisons rather than using the existing type conversions on the underlying types, and then building the correctly merged (or reject) overflow behaviors.
LANGOPT(PaddingOnUnsignedFixedPoint, 1, 0, NotCompatible, | ||
"unsigned fixed point types having one extra padding bit") | ||
|
||
LANGOPT(OverflowBehaviorTypes, 1, 0, NotCompatible, "overflow behavior types") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this name could be better, something like ExplicitTypeOverflowBehavior
maybe? @mizvekov @AaronBallman (I'm terrible at naming things, but this current name seems confusing to me)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the existing name, it has stuck long enough.
getAddressSpace() == LangAS::wasm_funcref; | ||
} | ||
|
||
bool QualType::isWrapType() const { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than duplicating the code this should just be a getOverflowBehavior()
function
clang/lib/Parse/ParseDecl.cpp
Outdated
isInvalid = DS.SetTypeQual(DeclSpec::TQ_restrict, Loc, PrevSpec, DiagID, | ||
getLangOpts()); | ||
break; | ||
case tok::kw___wrap: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these keywords really necessary? This isn't being implemented as an actual type qualifier so it doesn't really seem to reach the level of needing an actual keyword. Especially given that there is an attribute form as well - my general understanding is that the cases of attribute + keyword for same features are largely the result of historical accidents, introducing new cases of this from day one seems unnecessary
@AaronBallman and @cor3ntin maybe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please continue discussion in #148914 (comment)
So that we don't have to repeat all the arguments.
case NK_Constant_Narrowing: { | ||
// A constant value was narrowed. | ||
|
||
// Overflow behavior destination types with a 'wrap' kind can elide |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand this? Why would wrapping behavior impact this diagnostic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check out OverflowBehaviorTypes.rst
's header regarding C++ Narrowing Conversions
. The idea is that wrapping types are allowed to abuse wrap-around and intentional truncation/narrowing, we shouldn't warn in these cases.
u32 a;
fun_stuff(&a);
// we are a power user, we've specified behavior for this u8.
u8 __wrap b = a; // potentially abuse truncation/narrowing without littering (u8) casts in places.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is that a reasonable assumption?
This change in behavior means that an error by the developer is now silently ignored. Now an unintentional narrowing of the data is silently ignored.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is that a reasonable assumption?
__wrap
is opt-in and is designed to behave differently at bit boundaries when overflowing them or when taking on a value larger than available storage space.
This change in behavior means that an error by the developer is now silently ignored. Now an unintentional narrowing of the data is silently ignored.
There is no change in behavior to any developer, this is a new featureset entirely. We aren't changing anything under anyone's feet (and if we are, that is a bug/oversight in my code). This feature is for power users who have clearly expressed their intent in utilizing funky wrap-around arithmetic.
But yes, it is possible to write bad code, even when using OBTs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, you are misunderstanding the point of this diagnostic.
The error is not the wrapping behavior, it is the author getting the underlying type wrong.
e.g. the author wrote uint32_t
when they meant to write unint64_t
. The fact that they want the type to have defined wrapping semantics is irrelevant to that diagnosis.
But yes, it is possible to write bad code, even when using OBTs.
Correct, and the literally entire point of this warning is to prevent this exact error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without the OBT check in the diagnostic logic we get the following warning:
constexpr short __ob_wrap cx2 = {100000}; // constant expression evaluates to 100000 which cannot be narrowed to type '__ob_wrap short'
And I didn't want this to be the case for wrapping things. So it is just a design decision really. It is consistent with the OBT-flavored -fsanitize=implicit-unsigned-integer-truncation
handling too. I just really want free reign over bit boundary stuff with __ob_wrap
types.
It should be noted the OBT check only cares about the destination type.
} else if (S.IsOverflowBehaviorTypePromotion(FromType, ToType)) { | ||
// OverflowBehaviorType promotions | ||
SCS.Second = ICK_Integral_Promotion; | ||
FromType = ToType.getUnqualifiedType(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems wrong -- the behavior should be to just perform the standard type conversions on the underlying types and then merge the overflow behaviors as appropriate on the resulting type.
There should not need to be any custom behavior from handling the removal and re-attachment of the overflow semantics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As per the documentation and previously agreed upon semantics, these overflow behavior types have special promotion properties. We do this to preserve bitwidths when it matters.
ToComplex->getElementType()); | ||
} | ||
|
||
bool Sema::IsOverflowBehaviorTypePromotion(QualType FromType, QualType ToType) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that these functions are querying information other Than the overflow behavior is concerning to me, it implies divergence in type conversion behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes the implied divergence in type conversion behavior is not just implied, it is real.
I think OverflowBehaviorTypes.rst
's header Promotion Rules
or Conversion Semantics
best covers this.
and some RFC discussion appears above and below this comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, are you saying that it retained the change to arithmetic operators to break promotion?
I don't think the RFC is for bike shedding discussions about the spelling of the type, this can be decided between the reviewers. This has been discussed here, please read existing comments and let's continue in the appropriate thread, so we don't have to restart and repeat ourselves. See: #148914 (comment) |
For keywords, use __ob prefix (__ob_wrap, __ob_trap). Rename no_wrap to trap everywhere. Some warnings have been changed to errors. Transition to single-specifier guarantees, use pre-existing specifier diags Drop usage of OverflowBehaviorSpelling enum, it is now obsolete as we are using BadSpecifier() to handle specifier collisions. Signed-off-by: Justin Stitt <[email protected]>
This comment became quite long, so I just want to add a preamble. First off, I'm sorry that I missed the RFC for this or I would have raised these objections then - If I did see it I would have blindly assumed that it was solely removing the UB nature of overflow, with no other semantic changes, and so probably not paid it any heed. This is a feature I have wanted for a long time, but the narrowing behavior is just not ok - I would have to essentially make this a banned feature in any code base I have influence over, for the reasons I'll be discussing below. I’m also not sure I understand the semantics of narrowing when uint64_t a = ...
uint32_t __no_wrap b = ...
auto result = a + b The documentation makes it clear that I could dump these issues into the RFC thread but I don’t think that’s necessarily appropriate - @AaronBallman or Anyway, on to the wall of text comment (as people who are on WG21 can attest I have an unerring ability to do this, despite my best efforts): The description of this PR states that this feature is fine grained control of fwrapv, etc that make wrapping have defined semantics rather than undefined. But the change in semantics here is not that, it is a significantly different behaviour, and has the effect of making previously correct code to incorrect code. I.e code that is correct, and has no UB (e.g. no overflow), is changed to be incorrect - introducing both overflow and truncation into code that previously had none. A common refrain in the replies to my comments has been this is a new feature that is opt in, but the problem is there are many cases developers would want to opt in to these in ways that would be relatively global. It's super easy to imagine a project doing: typedef uint8_t __nowrap uint8_safe_t;
typedef uint16_t __nowrap uint16_safe_t;
typedef uint32_t __nowrap uint32_safe_t;
typedef uint63_t __nowrap uint63_safe_t;
typedef uint8_t __wrap uint8_wrap_t;
typedef uint16_t __wrap uint16_wrap_t;
typedef uint32_t __wrap uint32_wrap_t;
typedef uint63_t __wrap uint63_wrap_t; Then adding a linter rule that requires all new code to use one these instead of the existing types. In codebases I have working in, this is exactly what they would want to do. But the narrowing change makes such a change impossible - interaction between new and existing code, or system APIs, means the narrowing can happen anywhere, and similarly literals like These changes in semantics can introduce security vulnerabilities where previously none existed. Take the format string example in the RFC, which was called out as a result of this change in behavior, which is the reason I assumed it had be removed: unsigned long long a;
size_t __wrap b;
printf("%llx", a + b); on systems where You also get silent truncation in previously correct code: struct S32 {
uint32_t __wrap a; // I want defined semantics for arithmetic on this field
// e.g. no UB -> compiler created security vulnerability
};
struct S64 {
uint64_t __wrap a;
};
uint64_t doSomething();
uint64_t foo(S32 s) {
return doSomething() + s.a; // silently truncate to uint32_t, and silently expand out to uint64_t;
} You can imagine code this also impacting code like: auto /*uint32_t*/ base = getBase();
auto /*uint<less than 32>_t __wrap*/ offset = ...; // so many formats and use cases have bytes here
if (offset >= size)
return;
buffer[base + offset] = ...; // you're now writing out of expected bounds I also suspect these semantics mean that (assuming the above uint64_t foo(uint32_t __wrap a, uint64_t wrap b) {
return doSomething() + a + b;
} could now produce a different result (it's already different from what existing correct code does) from uint64_t foo(uint32_t __wrap a, uint64_t wrap b) {
return doSomething() + b + a;
} Due to the narrowing of Similar to this, previously correct C++ becomes wrong, and changes substantially: // just some nonsense to demonstrate the viral nature of this change in narrowing
template <class T> struct S {
S(T a) : a(a) {}
T a;
}
template <class A, class B> auto doSomething(S<A> s1, S<B> s2) -> S<decltype(s1.a + s2.b)> {
return S(s1.a + s2.a);
}
Modern C++ has I think The reality is that given the change to narrowing behavior this attribute does not actually solve the real world problems I’ve seen, or that I have had developers express a desire for support with, and it makes existing correct code incorrect if adopted, to the extent that it can create new vulnerabilities rather than removing them. Given the above issues, not only would this not be a solution to what developers have asked for, it would likely be necessary to forbid the adoption of the That said, these are just the issues that I see from my point of view - the RFC says that this is intended to support linux kernel use cases, so I’d like to know whether these issues and hazards seem like a problem to them. |
Sorry, I didn't include the most obvious security vulnerability: uint16_t __wrap count = ...;
malloc(sizeof(SomeType) * count); The narrowing behavior truncates the allocation size. Just to be clear though, the developer may not have explicitly state uint64_t count = ...;
malloc(sizeof(SomeType) * count); The RHS expression may have already been truncated - it might directly have the type |
@rnk The Clang Area team might want to be aware of the above comments, it's probably worth a discussion at a future meeting. |
All good, your review is vital. I really have no personal attachments to any particular design decisions made -- I want to make sure this feature is useful to as many developers as possible. Quick question before I go meet with Kees: How do you propose we handle overflow semantics with less-than-int stuff. The core reason all of this narrowing and promotion magic exists is to solve the problem of: u8 __ob_wrap a = 255;
if (a+1 == 0){...} // 1 is `int`, therefore whole expression promoted to `int`.
// Whoops! We no longer wrap-around at the 8-bit boundary because we are in `int` type boundaries now. Not truncating things back down to the wrap type renders the So, @ojhunt how do you ideally want all these narrowing semantics to work? |
You would need to give me some indication that this is a behavior developers want or expect. I think it's useful to think about how this attribute will be used in practice, including the linux kernel: #if __has_attribute(...) // or has keyword or whatever
#define __wrap_attr __obt_wrap
#else
#define __wrap_attr
#endif
Given that usage - which is how it would have to be set up - how would your example of uint8_t __wrap_attr a = 255;
if (a + 1 == 0) { ... } To work in any real use case? It will not work on any compiler that does not have this attribute. But this code is also fundamentally not realistic - a real version of this code would look like: uint8_t __wrap_attr a = 255;
if (a + 1 > 0xff) { ... } Except with the __wrap_attr this will no longer ever be true. In fact any type bounds check like this does not work: if (some_int + some_intN > __INTN_MAX__) The only case that the truncation of large values does is your above
Why do you think you need to do this? The problem with overflow is not "the value is larger than I expected" it is "overflow leads to UB allowing compiler introduced vulnerabilities", and "ftrapv and fwrapv are global, but we want different behavior in different contexts". This change to narrowing does not have anything to do with that.
The exact same way promotion works today: there should not be any narrowing. There is no reason to change this behavior to narrow: it makes the feature confusing, introduces security vulnerabilities, makes adoption unusable, and literally discards data in ways that produce warnings in many default configuration. Implicit narrowing is widely regarded as a mistake nowadays, and this not only adds a new version of implicit narrowing, it removes the warnings about that narrowing. Returning to your example: if (a + 1 == 0) { .. } this is not something I believe anyone would expect, and more to the point is not something anyone could actually use. Removing narrowing does not make the wrapping behavior meaningless: int f(bool b) {
int8_t __obt_trap a = 1;
int32_t i = __INT_MAX__ ;
if (b)
i = i + a; // int32_t + uint8_t __obt_trap => int32_t + int32_t __obt_trap => trap due to the overflow
return i;
} But more to the point, consider this: int64_t a = ...;
int32_t __obt_wrap /* or __obt_trap */ b = ...;
int64_t result = a + b; Would any developer would expect a truncated 32bit result? or an overflow trap? That doesn't match We can also consider an unsigned version of the above: uint64_t a = ...;
uint32_t __obt_wrap /* or __obt_trap */ b = ...;
uint64_t result = a + b; Overflow of Basically, rather that asking "how would template <class A, class B> void f(A a, B b) {
static_assert(same_type_v<decltype((A)0 + (B __wrap)0), decltype((A)0 + (B)0) __obt_wrap>);
static_assert(same_type_v<decltype((A __wrap)0 + (B)0), decltype((A)0 + (A)0) __obt_wrap>);
static_assert(same_type_v<decltype((A)0 + (B __obt_trap)0), decltype((A)0 + (B)0) __obt_trap>);
static_assert(same_type_v<decltype((A __obt_trap)0 + (B)0), decltype((A)0 + (A)0) __obt_trap>);
} Because these qualifiers introduce implicit exposure that is not directly exposed in local source, e.g that types below could be uint64_t u64 = 1ull<<32;
uint32_t __wrap u32 = 1;
u64 = u64 + u32; Which from what I can tell results in Let's imagine a warning about this then all of the existing correct code now needs to have casts everywhere, and those casts can easily drop the qualifier, ie. the above would need to be "corrected" with uint64_t u64 = 1ull<<32;
uint32_t __obt_wrap u32 = 1;
u64 = u64 + (some_type)u32; But what should |
Introduce
OverflowBehaviorType
(OBT), a new type attribute in Clang that provides developers with fine-grained control over the overflow behavior of integer types. This feature allows for a more nuanced approach to integer safety, achieving better granularity than global compiler flags like-fwrapv
and-ftrapv
.The new
__attribute__((overflow_behavior(behavior)))
can be applied to integer types (both signed and unsigned) as well as typedef declarations, where behavior is one of the following:wrap
: Guarantees that arithmetic operations on the type will wrap on overflow, similar to-fwrapv
. This suppresses UBSan's integer overflow checks for the attributed type.no_wrap
: Enforces overflow checking for the type, even when global flags like-fwrapv
would otherwise suppress it.A key aspect of this feature is its interaction with existing mechanisms.
OverflowBehaviorType
takes precedence over global flags and, notably, over entries in the Sanitizer Special Case List (SSCL). This allows developers to "allowlist" critical types for overflow instrumentation, even if they are disabled by a broad rule in an SSCL.This implementation also includes specific promotion rules to preserve the intended overflow behavior within expressions and new diagnostics to warn against accidentally discarding the attribute.
See the docs and tests for examples and more info.
RFCs
RFC v2
RFC v1
CCs (because of your participation in RFCs or previous implementations)
@kees @AaronBallman @vitalybuka @melver @efriedma-quic