[Flang][Preprocessor] Avoid creating an empty token when a kind suffix is torn by a pasting operator #139795
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… operator This confuses the logic that implements the pasting itself.
llvmbot
added
flang
|
@llvm/pr-subscribers-flang-parser Author: Roger Ferrer Ibáñez (rofirrim) ChangesThis input "tears" the expected tokens of an integer-literal due to a pasting operator Creating an empty token by accident (due to two consecutive If zero-length tokens are acceptable, then instead of this patch we may have to fix the logic in Full diff: https://github.com/llvm/llvm-project/pull/139795.diff 2 Files Affected:
diff --git a/flang/lib/Parser/prescan.cpp b/flang/lib/Parser/prescan.cpp
index 3bc2ea0b37508..004e4f013f90a 100644
--- a/flang/lib/Parser/prescan.cpp
+++ b/flang/lib/Parser/prescan.cpp
@@ -937,6 +937,7 @@ bool Prescanner::HandleKindSuffix(TokenSequence &tokens) {
if (*at_ != '_') {
return false;
}
+ auto underscore = *at_;
TokenSequence withUnderscore, separate;
EmitChar(withUnderscore, '_');
EmitCharAndAdvance(separate, '_');
@@ -951,6 +952,13 @@ bool Prescanner::HandleKindSuffix(TokenSequence &tokens) {
}
withUnderscore.CloseToken();
separate.CloseToken();
+ // If we only saw "_" and nothing else, we have handled enough but we do not
+ // want to close the token here, or we will generate an extra token of length
+ // zero.
+ if (separate.SizeInTokens() == 1) {
+ EmitChar(tokens, underscore);
+ return true;
+ }
tokens.CloseToken();
if (separate.SizeInTokens() == 2 &&
preprocessor_.IsNameDefined(separate.TokenAt(1)) &&
diff --git a/flang/test/Preprocessing/torn-token-pasting-1.F90 b/flang/test/Preprocessing/torn-token-pasting-1.F90
new file mode 100644
index 0000000000000..5e080129a94d1
--- /dev/null
+++ b/flang/test/Preprocessing/torn-token-pasting-1.F90
@@ -0,0 +1,9 @@
+! RUN: %flang -E %s 2>&1 | FileCheck %s
+! CHECK: IF(10>HUGE(1_4).OR.10<-HUGE(1_4)) CALL foo()
+#define CHECKSAFEINT(x,k) IF(x>HUGE(1_ ## k).OR.x<-HUGE(1_##k)) CALL foo()
+
+program main
+ implicit none
+
+ CHECKSAFEINT(10, 4)
+end program main
|
|
@jeanPerier maybe you know who could review this? Thanks a lot! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This input "tears" the expected tokens of an integer-literal due to a pasting operator
##. When lexing1_##we generate the sequence of tokens['1_', ''], the second being an empty token of length zero. The second token is created at the end ofPrescanner::NextToken.Creating an empty token by accident (due to two consecutive
CloseTokenwithout consuming anything) can causeTokenSequence::pop_backto assert.If zero-length tokens are acceptable, then instead of this patch we may have to fix the logic in
TokenPastingfound inpreprocessor.cpp.