-
Notifications
You must be signed in to change notification settings - Fork 830
Don't find witnesses for typars with conditional constraints #19123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
95be013
4598ac3
8642e8c
d845820
0daa0d6
4e2e71e
37e5ef1
ea7adb3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -508,14 +508,47 @@ module FSharpExprConvert = | |
|
|
||
| and GetWitnessArgs cenv (env: ExprTranslationEnv) (vref: ValRef) m tps tyargs : FSharpExpr list = | ||
| let g = cenv.g | ||
| if g.langVersion.SupportsFeature(Features.LanguageFeature.WitnessPassing) && not env.suppressWitnesses then | ||
| let witnessExprs = | ||
| if g.langVersion.SupportsFeature(Features.LanguageFeature.WitnessPassing) && not env.suppressWitnesses then | ||
| /// There are two *conditional* properties a typar can have: equality and comparison. | ||
| /// A generic type having that constraint may be conditional on whether a specific type parameter to that generic has that | ||
| /// constraint. | ||
| /// This function returns `true` iff after unification, the type definition contains any conditional typars. | ||
| /// | ||
| /// Note that these conditions are only marked on typars that actually appear in the code, *not* on phantom types. | ||
| /// So `hasConditionalTypar` should tell us exactly when the type parameter is actually being used in the type's equality or | ||
| /// comparison. | ||
| let rec hasConditionalTypar ty = | ||
| match stripTyEqns g ty with | ||
| | TType_var (tp, _) -> tp.ComparisonConditionalOn || tp.EqualityConditionalOn | ||
| | TType_app (_, tinst, _) -> tinst |> List.exists hasConditionalTypar | ||
| | _ -> false | ||
|
|
||
| let witnessExprs = | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess we could check up front for this case rather than catching the error - that might be nicer and more precise, but would incur the check much more frequently. |
||
| match ConstraintSolver.CodegenWitnessesForTyparInst cenv.tcValF g cenv.amap m tps tyargs with | ||
| // There is a case where optimized code makes expressions that do a shift-left on the 'char' | ||
| // type. There is no witness for this case. This is due to the code | ||
| // let inline HashChar (x:char) = (# "or" (# "shl" x 16 : int #) x : int #) | ||
| // in FSharp.Core. | ||
| // in FSharp.Core. | ||
| | ErrorResult _ when vref.LogicalName = "op_LeftShift" && List.isSingleton tyargs -> [] | ||
| // We don't need a witness either at compile time or runtime when there are conditional typars. | ||
| // Attempting to call a comparison operation with the type causes a compile-time check that all the generic type args | ||
| // support comparison (thanks to the ComparisonConditionalOn mechanism); the compile-time check doesn't need witnesses, | ||
| // it's just pure constraint solving. | ||
| // Nor do we need a witness for runtime logic: the compiler generates a `CompareTo` method (see | ||
| // `MakeValsForCompareAugmentation`) which handles the comparison by dynamically type-testing, not going through a witness. | ||
| // | ||
| // So we don't need to generate a witness. | ||
| // | ||
| // In fact, we *can't* generate a witness, because the constraint on the type parameter is only conditional: a rigid type | ||
| // parameter, defined without the `comparison` constraint, cannot have the constraint added to it later (that's what "rigid" | ||
| // means). It would change the type signature of the type to add this constraint to the type parameter! | ||
| // | ||
| // This code path is only reached through the auto-generated comparison/equality code, which only calls single-constraint | ||
| // intrinsics: there's exactly one constraint per type parameter in each of those two cases. | ||
| // In theory, if a function had an autogenerated `'a : comparison and 'b : SomethingElse`, where the `SomethingElse` was | ||
| // not comparison and failed for a different reason, we'd spuriously hide that failure here; but in fact the only code | ||
| // paths which get here have no other constraints. | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This assertion is very hard for me to verify; proving a negative is difficult. |
||
| | ErrorResult _ when List.exists hasConditionalTypar tyargs -> [] | ||
| | res -> CommitOperationResult res | ||
| let env = { env with suppressWitnesses = true } | ||
| witnessExprs |> List.map (fun arg -> | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3504,3 +3504,193 @@ let ``Test ProjectForWitnesses4 GetWitnessPassingInfo`` () = | |
| printfn "actual:\n\n%A" actual | ||
| actual | ||
| |> shouldEqual expected | ||
|
|
||
| //--------------------------------------------------------------------------------------------------------- | ||
Smaug123 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| // Regression tests for ImmediateSubExpressions on generic types with conditional comparison/equality | ||
| // https://github.com/dotnet/fsharp/issues/19118 | ||
| // | ||
| // The bug: FCS crashes when accessing ImmediateSubExpressions on auto-generated comparison code | ||
| // for generic DUs/records whose type parameters have ComparisonConditionalOn but not actual | ||
| // comparison constraints. This is because GetWitnessArgs tries to generate witnesses for the | ||
| // comparison constraint, but fails because the type parameter is rigid and can't have constraints added. | ||
|
|
||
| module internal ProjectForWitnessConditionalComparison = | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does it affect codegen (in the style EmittedIlTests ) or behavior of the witness at runtime? I haven't search for the most appropriate test suite for exercising witness behavior, but I do not trust my "by reading code" judgement for witnesses and would prefer a demonstration via tests that acutally make use of the witness (via quotations), to make sure things still work at runtime. Apart from that, nicely done! 👍
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just to check - so you'd like some snapshots of the emitted IL, including calls to the comparison methods, like in https://github.com/dotnet/fsharp/blob/6ef4403f32ecc35634dc6d73c6e106d1c4866682/tests/FSharp.Compiler.ComponentTests/EmittedIL/TupleElimination.fs (for example), and I'll check that they haven't changed before and after this fix? By "quotations" here - are you referring to just the fact that the F# code is in strings in e.g. the EmittedIL snapshot tests? Or did you have something else in mind?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Quotations are the reason witness methods exists. Normally, all SRTP functions are inlined at place of usage, so their original definition would not have to be in the IL at all. With Quotations, you can construct an expression programmatically and then attempt to evaluate it. https://github.com/fsharp/fslang-design/blob/main/FSharp-5.0/FS-1071-witness-passing-quotations.md Vanilla sample:
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So I guess two main classes of test case: evaluate a quotation of an SRTP method that takes a parameter that gets statically resolved to the DU type, vs reflectively invoke the $W-named witness-taking version of that method? I guess also in the two cases of "generic parameter to the DU is specialised to something comparable" vs "not"? (Recording a prediction: my current mental model predicts that this PR doesn't change the behaviour in either case, because we weren't generating the witness before due to the same error we are catching and handling now.) And these test cases should be simply behavioural evaluation of F# code, not assertions about the generated IL or the state of the compiler?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, just |
||
|
|
||
| /// Helper to walk ALL expressions in a file, including ImmediateSubExpressions | ||
| /// This triggers the bug because it forces conversion of auto-generated comparison code | ||
| let walkAllExpressions (source : string) = | ||
| let fileName1 = System.IO.Path.ChangeExtension(getTemporaryFileName (), ".fs") | ||
| try | ||
| FileSystem.OpenFileForWriteShim(fileName1).Write(source) | ||
| let options = createProjectOptions [source] [] | ||
| let exprChecker = FSharpChecker.Create(keepAssemblyContents=true, useTransparentCompiler=CompilerAssertHelpers.UseTransparentCompiler) | ||
| let wholeProjectResults = exprChecker.ParseAndCheckProject(options) |> Async.RunImmediate | ||
|
|
||
| if wholeProjectResults.Diagnostics.Length > 0 then | ||
| for diag in wholeProjectResults.Diagnostics do | ||
| printfn "Diagnostic: %s" diag.Message | ||
|
|
||
| for implFile in wholeProjectResults.AssemblyContents.ImplementationFiles do | ||
| // Walk all declarations and their expressions, including ImmediateSubExpressions | ||
| let rec walkExpr (e: FSharpExpr) = | ||
| // Access ImmediateSubExpressions - this is what triggered #19118 | ||
| for subExpr in e.ImmediateSubExpressions do | ||
| walkExpr subExpr | ||
|
|
||
| let rec walkDecl d = | ||
| match d with | ||
| | FSharpImplementationFileDeclaration.Entity (_, subDecls) -> | ||
| for subDecl in subDecls do | ||
| walkDecl subDecl | ||
| | FSharpImplementationFileDeclaration.MemberOrFunctionOrValue (_, _, e) -> | ||
| walkExpr e | ||
| | FSharpImplementationFileDeclaration.InitAction e -> | ||
| walkExpr e | ||
|
|
||
| for decl in implFile.Declarations do | ||
| walkDecl decl | ||
| finally | ||
| try | ||
| FileSystem.FileDeleteShim fileName1 | ||
| with | ||
| | _ -> () | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic DU with no constraints should not crash`` () = | ||
| // This is the core bug repro - a generic DU where the type parameter has | ||
| // ComparisonConditionalOn but no actual comparison constraint | ||
| let source = """ | ||
| module M | ||
|
|
||
| type Bar<'appEvent> = | ||
| | Wibble of 'appEvent | ||
| """ | ||
| // This should not throw. Before the fix, it crashed with ConstraintSolverMissingConstraint. | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic DU with multiple type parameters should not crash`` () = | ||
| let source = """ | ||
| module M | ||
|
|
||
| type MultiParam<'a, 'b, 'c> = | ||
| | Case1 of 'a | ||
| | Case2 of 'b * 'c | ||
| | Case3 of 'a * 'b * 'c | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic record with no constraints should not crash`` () = | ||
| let source = """ | ||
| module M | ||
|
|
||
| type MyRecord<'t> = { Value: 't; Name: string } | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic struct DU should not crash`` () = | ||
| let source = """ | ||
| module M | ||
|
|
||
| [<Struct>] | ||
| type StructDU<'a> = | ||
| | StructCase of value: 'a | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - nested generic types should not crash`` () = | ||
| let source = """ | ||
| module M | ||
|
|
||
| type Outer<'a> = | ||
| | OuterCase of Inner<'a> | ||
|
|
||
| and Inner<'b> = | ||
| | InnerCase of 'b | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic DU with explicit comparison constraint works`` () = | ||
| // When the type parameter has the comparison constraint, witness generation should work; | ||
| // no crash occurred even before the bug was fixed. This test is here for completeness. | ||
| let source = """ | ||
| module M | ||
|
|
||
| type WithConstraint<'a when 'a : comparison> = | ||
| | Constrained of 'a | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - non-generic DU works`` () = | ||
| // Non-generic types always worked fine (no generics = no witness issues). This test is here for completeness. | ||
| let source = """ | ||
| module M | ||
|
|
||
| type SimpleUnion = | ||
| | Case1 of int | ||
| | Case2 of string | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic DU with NoComparison attribute should not crash`` () = | ||
| // With NoComparison, no comparison code is generated, so no crash ever occurred even before the bug was fixed. | ||
| // This test is here for completeness. | ||
| let source = """ | ||
| module M | ||
|
|
||
| [<NoComparison>] | ||
| type NoCompare<'a> = | ||
| | NoCompareCase of 'a | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic DU with NoEquality attribute should not crash`` () = | ||
| let source = """ | ||
| module M | ||
|
|
||
| [<NoEquality; NoComparison>] | ||
| type NoEq<'a> = | ||
| | NoEqCase of 'a | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - generic DU used in function should not crash`` () = | ||
| // Test that using the generic DU in actual code still works | ||
| let source = """ | ||
| module M | ||
|
|
||
| type Option2<'t> = | ||
| | Some2 of 't | ||
| | None2 | ||
|
|
||
| let mapOption2 f opt = | ||
| match opt with | ||
| | Some2 x -> Some2 (f x) | ||
| | None2 -> None2 | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
|
|
||
| [<Fact>] | ||
| let ``ImmediateSubExpressions - complex generic type hierarchy should not crash`` () = | ||
| let source = """ | ||
| module M | ||
|
|
||
| type Result<'ok, 'err> = | ||
| | Ok of 'ok | ||
| | Error of 'err | ||
|
|
||
| type Validated<'a> = Result<'a, string list> | ||
|
|
||
| let validate pred msg value : Validated<'a> = | ||
| if pred value then Ok value | ||
| else Error [msg] | ||
| """ | ||
| ProjectForWitnessConditionalComparison.walkAllExpressions source | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As Opus says, I believe correctly:
Indeed, the code contains this:
and this: