Skip to content

Fix recursive function type error with for-in loops (#2216)#2220

Closed
Definisi wants to merge 5 commits intoluau-lang:masterfrom
Definisi:fix-recursive-function-type-error-2216
Closed

Fix recursive function type error with for-in loops (#2216)#2220
Definisi wants to merge 5 commits intoluau-lang:masterfrom
Definisi:fix-recursive-function-type-error-2216

Conversation

@Definisi
Copy link

Summary

Fixes #2216 - "Recursive Function Type Error"

This PR fixes the error Type function instance intersect<blocked-XXXXX, ~nil> is uninhabited that occurred when a recursive function used a for-in loop over its own recursive call result.

Problem

The following code would produce a confusing type error:

--!strict
type tb_any = {[any]:any}

function flatten(t: tb_any): tb_any
    local out: tb_any = {}
    for k, v in flatten(t) do
        out[k] = v
    end
    return out
end

Before Fix (Error Output)

Type inference failed to complete, you may see some confusing types and type errors.
Type function instance intersect<blocked-110838, ~nil> is uninhabited
This is likely to be a bug, please report it at https://github.com/luau-lang/luau/issues

After Fix (No Error)

The code type-checks successfully with no errors.

Root Cause

The issue was in ConstraintGenerator::lookup(). When looking up a type for a phi DefId (which represents captured variables across function boundaries):

  1. GlobalPrepopulator sets lvalueTypes[phiDef] = BlockedType for all global references during prepopulation
  2. When checking a recursive function call inside the function body, the reference gets a capture phi DefId
  3. The lookup function first checked if the phi was directly bound in the scope
  4. It found the BlockedType from prepopulation and returned it immediately
  5. This shadowed the correct function signature that should have been found through the phi's operands

The BlockedType never got resolved because of a dependency cycle, causing the intersect<blocked, ~nil> type function to fail.

Fix

Modified ConstraintGenerator::lookup() to prefer resolving phi types through their operands first (for single-operand phis), before falling back to direct binding lookup:

// For phis with exactly 1 operand, prefer resolving through the operand first.
if (!prototype && phi->operands.size() == 1)
{
    if (auto operandTy = lookup(scope, location, phi->operands.at(0), prototype))
        return operandTy;
}
// Fall back to direct phi lookup
if (auto found = scope->lookup(def))
    return *found;

This ensures that for recursive function references, the lookup correctly resolves through the phi operand (which points to the Cell DefId bound to the function signature) instead of returning the prepopulated BlockedType.

Additional Fixes

  • Fixed occurs() in BuiltinTypeFunctions.cpp which was incorrectly checking for UnionType instead of IntersectionType
  • Added recursive type detection for the intersect type function to handle degenerate recursive types

Test Plan

  • Added test case oss_2216_recursive_function_with_for_in_loop that verifies the exact code pattern from the issue
  • All existing tests pass (4470 tests)

Fix the "Type function instance intersect<blocked, ~nil> is uninhabited"
error that occurred when a recursive function used a for-in loop over
its own recursive call result.

The issue was in ConstraintGenerator::lookup - when looking up a type
for a phi DefId, the function would return a prepopulated BlockedType
instead of resolving through the phi operands to find the actual
function signature.

Changes:
- Modified lookup() to prefer resolving phi types through operands first
  for single-operand phis before falling back to direct binding lookup
- Fixed occurs() in BuiltinTypeFunctions.cpp to check IntersectionType
  instead of UnionType
- Added recursive detection for intersect type function to handle
  degenerate recursive types

Added test case for the exact pattern from issue luau-lang#2216.
Only prefer resolving through the phi operand when the phi itself
is bound to a BlockedType. This is more conservative and avoids
changing behavior for cases where the phi has a concrete type.

Previously, we always tried the operand first for single-operand
phis, which could change error reporting behavior for functions
without annotations.
Changed GlobalPrepopulator::visit(AstExprGlobal*) to only set
lvalueTypes for Cell defs, not Phi defs. Phi defs represent
captured references (like recursive function calls) and should
resolve through their operands to get the actual function type,
not a prepopulated BlockedType.

This is a more targeted fix that addresses the root cause:
GlobalPrepopulator was treating recursive references the same as
forward references, but recursive references should use the
actual function signature being defined, not a placeholder
BlockedType.

Reverted the lookup() changes to keep the original behavior.
When looking up a single-operand phi that has a BlockedType binding
(from GlobalPrepopulator), check if the operand has a concrete
FunctionType. If so, prefer the operand to get the actual function
signature instead of the placeholder BlockedType.

This specifically handles recursive function calls where the function
has explicit type annotations - the operand will have the FunctionType
with the proper signature, while the phi's direct binding is a
BlockedType from prepopulation.

For functions without explicit annotations (like those with @deprecated),
the operand won't be a FunctionType, so we fall back to the phi's
binding which will be properly resolved with deprecation info.
When the intersect type function encounters pending (blocked) types,
instead of returning null and blocking entirely, return an
IntersectionType that can be simplified later. This allows recursive
function calls to proceed even when the return type is temporarily
blocked.

This prevents the "uninhabited type function" error for recursive
functions with explicit return type annotations used in for-in loops.
@Definisi Definisi closed this Jan 29, 2026
@Definisi Definisi deleted the fix-recursive-function-type-error-2216 branch January 29, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recursive Function Type Error

1 participant