Implement mechanism for strict mode

This implements support for the `strict mode` mechanism proposed in #54903.
There was a desire to see an implementation of this along side #60018
since that are conceptually adjacent. However, other than some shared
infrastructure, this version is farther from #60018 than the original
proposal for reasons mentioned below. The motivation and basic idea
are largely unchanged.

# Implemented semantics

The user facing interface to this mechanism is `@strict` (currently
in `Base.Experimental`, but expected to be moved out in the course
of the 1.14 release process. Rather than paraphrase, let me just
provide the first part of the docstring that describes the semantics:

```
Set the `strict` mode for the current module to all flags specified in `flags`.
Each `strict` mode flag is an independent option that disallows certain syntax or
semantics that may be undesirable in *some* contexts. Package authors should
consider which flags are appropriate for their package and set them at the top
of the applicable module.

# General semantics and philosophy

Note that additional strict mode flags are not necessarily *safer* or *better*
in any way; they are a reflection of of the reality that different users have
different tradeoffs for their codebases and what may be a sensible restriction
in a mature package could be very annoying in the REPL.

As designed, there are several general guidelines that apply to strict mode flags.
To the extent possible, they should be kept for future flag additions.

1. Code that evaluates without error in strict mode should also evaluate without
   error under the ordinary julia execution semantics.

2. Strict mode should not affect parsing. If it is desirable to disallow a
   particular syntax pattern, it should be recognized at the lowering stage.
   If this is currently not possible, the parser should be modified to emit an
   appropriate marker that can be checked at lowering time.

3. Strict mode is not intended for for issues that are clearly bugs.
   Those should instead use the syntax versioning mechanism (see
   [`Base.Experimental.@set_syntax_version`](@ref)). However, `strict` mode flags
   that gain widespread adoption may eventually be considered as candidates for
   syntax evolution.

Strict mode flags are automatically inherited by submodules, but can be
overriden by an explicit `@strict` invocation in the submodule. Strict mode
flags are partitioned by world age.

# Specifying flags

The `flags` expression is runtime-evaluated and should evaluate to a collection
of `Symbols` as specified below. In addition, nested collections of symbols are
allowed and will be flattened. This is intended to support specifying strict
mode flags in a central location and enforcing them across multiple dependents.
```

# Module vs Project.toml opt-ins
Of particular note is that I ended up deciding on a per-module opt-in
rather than a Project.toml opt in (like was originally proposed
in #54903, and is implemented for syntax evolution in #60018).
This is for the following reasons:

1. The #60018 experience has shown that project.toml opt ins are
   semantically somewhat awkward and need to be implemented both in the
   language and the package manager. This was fine for the syntax
   version, but strict mode is richer (and potentially much richer in
   the future) and adding this complexity into code loading seems
   undesirable.

2. One of the design objectives is to allow user-defined collections of
   strict mode flags enforced centrally across multiple packages. In
   this design this is easy by having a MyOrganizationBase package that
   defines a variable with the set of flags to enable. Doing something
   like this in Project.toml opens a whole can of worms on how to
   represent that.

3. I believe the concern about wanting to enable parse-time strict mode
   can be adequately addressed by having the parser emit a special
   marker that can then get picked up and checked against the strict
   mode by lowering (such marker addition possibly making use of the
   syntax evolution mechanism). If this is not how it works, the parser
   would need additional input state specifying the strict mode flags.
   #60018 has shown that changing parser state flags dynamically is
   undesirable, because people don't have a good sense of what the
   parse unit is. As such, I don't want the parser to look at strict
   mode flags at all.

4. As implemented here `@strict` inherits binding-world-age semantics.
   Since these are now well defined as of 1.12, this addresses a lot of
   the ordering concerns that were brought up in the discussion of
   #54903.

5. I think it may be useful to opt into certain strict mode flags for
   some modules in a package only (unlike the syntax version, where I
   don't expect this to be common). E.g. packages may define modules
   that define their core API or segregate their core algorithms from
   support code and may want more strict coding styles for such core
   modules.

There remains a bit of a concern that this is less friendly to IDEs.
I'm sympathetic to that, but the analysis required to compute the
strict mode is a lot simpler than other analyses (so a language server
should easily be able to do that) and I think it's outweighed
particularly by the desire for user-definable collections (which
requires the IDE to do some sort of analysis of however that is
specified anyway). Given that, I think this mechanism is as IDE friendly
as it gets, since the required capability is simply to compute the
value of a constant obtained from a macro expansion (so no special
strict-mode specific analysis required).

# Implementation details

The core of the implementation is simple, to determine the active
strict mode flags, we simply look up the
`_internal_module_strict_flags` binding in the appropriate module
and see which flags are set. The exact types and values of this
binding are explicitly and intentionally implementation details
and `@strict` decides how to set it. This is inteded to allow
flexibility of implementation in the future here.

To faciliate the above described semantics of `@strict`, this binding
has a couple of special features:

1. It gets automatically imported from a module's parent module upon
   module creation.
2. Unlike bindings created through syntax, invalidations from imports
   to `const` is permitted.

Otherwise the mechanism behaves as an ordinary binding, including
obeying world age semantics, and being Revise-able, etc. In particular,
if you Revise the `@strict` setting in a top-level module it will
automatically (through binding invalidation) be updated in all
submodules. It was important to me that doing this would not leave
the settings inconsistent.

# Implemented strict mode flags

Two strict mode flags are implemented in this PR, but they should
largely be considered straw-man implementations to show how to
access the flags set from within either the runtime or lowering.
Everything works end-to-end, but we may want to do some extra work
refining the precise semantics of these flags once we've merged
the core mechanism. The reason for choosing these flags is simply
that they were easy to implement. Several of the other proposed
flags would require additional analysis in lowering, which should
go in their own PRs. Implemented flags are as follows:

```

 * `typeimports`

    This flag turns the 1.12 warning for implicit import of types into an error.
    Note that the implicit import default may be removed in a future Julia syntax
    iteration, in which case this flag will become a no-op for such versions.

    ```jldoctest
    julia> @Base.Experimental.strict :typeimports

    julia> String(x) = 1
    ERROR: `@strict :typeimports` disallows extending types without explicit import in TypeImports: function Base.String must be explicitly imported to be extended
    ```

 * `:nointliteraliterators`

    Disallows (at the lowering stages) literal integers as iterators in `for` loops.
    This protects against expressions like `for i in 10` which are commonly intended
    to be `for i in 1:10`.

    ```jldoctest
    julia> for i in 10
               println(i)
           end
    10

    julia> @Base.Experimental.strict :nointliteraliterators

    julia> for i in 10
               println(i)
           end
    ERROR: syntax: `@strict :nointliteraliterators` disallows integer literal iterators here around none:1
    Stacktrace:
     [1] top-level scope
       @ none:1
```

Author: Keno

base/Base_compiler.jl

@@ -2,8 +2,8 @@
 
 module Base
 
-Core._import(Base, Core, :_eval_import, :_eval_import, true)
-Core._import(Base, Core, :_eval_using, :_eval_using, true)
+Core._import(Base, Core, :_eval_import, :_eval_import, 0x1)
+Core._import(Base, Core, :_eval_using, :_eval_using, 0x1)
 
 using .Core.Intrinsics, .Core.IR
 
@@ -148,6 +148,12 @@ function _setup_module!(mod::Module, Core.@nospecialize syntax_ver)
     Core._using(mod, _topmod(mod), UInt8(0))
     Core.declare_const(mod, :include, IncludeInto(mod))
     Core.declare_const(mod, :eval, Core.EvalInto(mod))
+    parent = ccall(:jl_module_parent, Ref{Module}, (Any,), mod)
+    if parent === mod
+    else
+        Core._import(mod, parent, :_internal_module_strict_flags,
+            :_internal_module_strict_flags, 0x3)
+    end
     if syntax_ver === nothing
         return nothing
     end

base/boot.jl

@@ -726,7 +726,7 @@ let
             a = getfield(paths, i).args
             length(a) == 1 || fail()
             s = getindex(a, 1)
-            Core._import(to, from, s, s, explicit)
+            Core._import(to, from, s, s, explicit ? 0x1 : 0x0)
             i += 1
         end
     end

base/docs/basedocs.jl

@@ -2782,7 +2782,7 @@ See also [`global`](@ref), [`setglobal!`](@ref), [`get_binding_type`](@ref Core.
 Core.declare_global
 
 """
-    declare_const(module::Module, name::Symbol, [x])
+    declare_const(module::Module, name::Symbol, [x, [flags::UInt8]])
 
 Create or replace the constant `name` in `module` with the new value `x`.  When
 replacing, `x` does not need to have the same type as the original constant.
@@ -2817,11 +2817,14 @@ See also [`const`](@ref).
 Core.declare_const
 
 """
-   _import(to::Module, from::Module, asname::Symbol, [sym::Symbol, imported::Bool])
+   _import(to::Module, from::Module, asname::Symbol, [sym::Symbol, flags::Uint8])
 
 With all five arguments, imports `sym` from module `from` into `to` with name
-`asname`.  `imported` is true for bindings created with `import` (set it to
-false for `using A: ...`).
+`asname`. `flags` is a bitmask of JL_IMPORT_FLAG_* flags.
+    - JL_IMPORT_FLAG_EXPLICIT is set for bindings created with `import` (unset for `using A: ...`).
+    - JL_IMPORT_FLAG_ALLOW_UNDEF allows the imported binding to be undefined at
+      import time (this is ordinarily a warning). Note that the binding may be
+      resolved later if it becomes available in the imported module.
 
 With only the first three arguments, creates a binding for the module `from`
 with name `asname` in `to`.

base/experimental.jl

@@ -841,4 +841,111 @@ function var"@VERSION"(__source__::Union{LineNumberNode, Core.MacroSource}, __mo
     end
 end
 
+# N.B.: Must match the values in julia_internal.h
+const JL_STRICT_IMPORT_TYPE = 0x01
+const JL_STRICT_LITERAL_ITERATORS = 0x02
+
+function strict_mode_flags(flag::Symbol)
+    if flag === :typeimports
+        return JL_STRICT_IMPORT_TYPE
+    elseif flag === :nointliteraliterators
+        return JL_STRICT_LITERAL_ITERATORS
+    else
+        error("unknown strict mode flag: $flag")
+    end
+end
+strict_mode_flags(flags) =
+    mapreduce(strict_mode_flags, |, flags; init=0x0)
+
+function set_strict_mode(mod::Module, @nospecialize(flags))
+    Core.declare_const(mod, :_internal_module_strict_flags,
+        strict_mode_flags(flags), Base.JL_CONST_MAY_REPLACE_IMPORTS)
+    return nothing
+end
+
+"""
+    Base.Experimental.@strict flags
+
+Set the `strict` mode for the current module to all flags specified in `flags`.
+Each `strict` mode flag is an independent option that disallows certain syntax or
+semantics that may be undesirable in *some* contexts. Package authors should
+consider which flags are appropriate for their package and set them at the top
+of the applicable module.
+
+# General semantics and philosophy
+
+Note that additional strict mode flags are not necessarily *safer* or *better*
+in any way; they are a reflection of of the reality that different users have
+different tradeoffs for their codebases and what may be a sensible restriction
+in a mature package could be very annoying in the REPL.
+
+As designed, there are several general guidelines that apply to strict mode flags.
+To the extent possible, they should be kept for future flag additions.
+
+1. Code that evaluates without error in strict mode should also evaluate without
+   error under the ordinary julia execution semantics.
+
+2. Strict mode should not affect parsing. If it is desirable to disallow a
+   particular syntax pattern, it should be recognized at the lowering stage.
+   If this is currently not possible, the parser should be modified to emit an
+   appropriate marker that can be checked at lowering time.
+
+3. Strict mode is not intended for for issues that are clearly bugs.
+   Those should instead use the syntax versioning mechanism (see
+   [`Base.Experimental.@set_syntax_version`](@ref)). However, `strict` mode flags
+   that gain widespread adoption may eventually be considered as candidates for
+   syntax evolution.
+
+Strict mode flags are automatically inherited by submodules, but can be
+overriden by an explicit `@strict` invocation in the submodule. Strict mode
+flags are partitioned by world age.
+
+# Specifying flags
+
+The `flags` expression is runtime-evaluated and should evaluate to a collection
+of `Symbols` as specified below. In addition, nested collections of symbols are
+allowed and will be flattened. This is intended to support specifying strict
+mode flags in a central location and enforcing them across multiple dependents.
+
+# Currently supported flags
+
+ * `typeimports`
+
+    This flag turns the 1.12 warning for implicit import of types into an error.
+    Note that the implicit import default may be removed in a future Julia syntax
+    iteration, in which case this flag will become a no-op for such versions.
+
+    ```jldoctest
+    julia> @Base.Experimental.strict :typeimports
+
+    julia> String(x) = 1
+    ERROR: `@strict :typeimports` disallows extending types without explicit import in TypeImports: function Base.String must be explicitly imported to be extended
+    ```
+
+ * `:nointliteraliterators`
+
+    Disallows (at the lowering stages) literal integers as iterators in `for` loops.
+    This protects against expressions like `for i in 10` which are commonly intended
+    to be `for i in 1:10`.
+
+    ```jldoctest
+    julia> for i in 10
+               println(i)
+           end
+    10
+
+    julia> @Base.Experimental.strict :nointliteraliterators
+
+    julia> for i in 10
+               println(i)
+           end
+    ERROR: syntax: `@strict :nointliteraliterators` disallows integer literal iterators here around none:1
+    Stacktrace:
+     [1] top-level scope
+       @ none:1
+"""
+macro strict(flags)
+    Expr(:call, set_strict_mode, __module__, esc(flags))
+end
+
 end # module

base/module.jl

@@ -113,7 +113,7 @@ function _eval_import(imported::Bool, to::Module, from::Union{Expr, Nothing}, pa
         if name !== nothing
             asname = asname === nothing ? name : asname
             check_macro_rename(name, asname, keyword)
-            Core._import(to, m, asname, name, imported)
+            Core._import(to, m, asname, name, imported ? JL_IMPORT_FLAG_EXPLICIT : 0x0)
         else
             Core._import(to, m, asname === nothing ? nameof(m) : asname)
         end

base/runtime_internals.jl

@@ -262,6 +262,11 @@ const BINDING_FLAG_ANY_IMPLICIT_EDGES = 0x8
 
 const JL_MODULE_USING_REEXPORT = 0x1
 
+const JL_IMPORT_FLAG_EXPLICIT    = 0x1
+const JL_IMPORT_FLAG_ALLOW_UNDEF = 0x2
+
+const JL_CONST_MAY_REPLACE_IMPORTS = 0x1
+
 is_defined_const_binding(kind::UInt8) = (kind == PARTITION_KIND_CONST || kind == PARTITION_KIND_CONST_IMPORT || kind == PARTITION_KIND_IMPLICIT_CONST || kind == PARTITION_KIND_BACKDATED_CONST)
 is_some_const_binding(kind::UInt8) = (is_defined_const_binding(kind) || kind == PARTITION_KIND_UNDEF_CONST)
 is_some_imported(kind::UInt8) = (kind == PARTITION_KIND_IMPLICIT_GLOBAL || kind == PARTITION_KIND_IMPLICIT_CONST || kind == PARTITION_KIND_EXPLICIT || kind == PARTITION_KIND_IMPORTED)

src/ast.c

@@ -39,6 +39,7 @@ typedef struct _jl_ast_context_t {
     value_t ssavalue_sym;
     value_t slot_sym;
     jl_module_t *module; // context module for `current-julia-module-counter`
+    uint8_t module_strict_flags; // cached strict mode flags for module
     struct _jl_ast_context_t *next; // invasive list pointer for getting free contexts
 } jl_ast_context_t;
 
@@ -151,11 +152,19 @@ static value_t fl_julia_scalar(fl_context_t *fl_ctx, value_t *args, uint32_t nar
     return fl_ctx->F;
 }
 
+static value_t fl_module_strict_flags(fl_context_t *fl_ctx, value_t *args, uint32_t nargs)
+{
+    argcount(fl_ctx, "julia-current-module-flags", nargs, 0);
+    jl_ast_context_t *ctx = jl_ast_ctx(fl_ctx);
+    return fixnum(ctx->module_strict_flags);
+}
+
 static jl_value_t *scm_to_julia_(fl_context_t *fl_ctx, value_t e, jl_module_t *mod);
 
 static const builtinspec_t julia_flisp_ast_ext[] = {
     { "defined-julia-global", fl_defined_julia_global }, // TODO: can we kill this safepoint
     { "current-julia-module-counter", fl_module_unique_name },
+    { "julia-current-module-flags", fl_module_strict_flags },
     { "julia-scalar?", fl_julia_scalar },
     { NULL, NULL }
 };
@@ -181,6 +190,7 @@ static void jl_init_ast_ctx(jl_ast_context_t *ctx) JL_NOTSAFEPOINT
     ctx->ssavalue_sym = symbol(fl_ctx, "ssavalue");
     ctx->slot_sym = symbol(fl_ctx, "slot");
     ctx->module = NULL;
+    ctx->module_strict_flags = 0;
     set(symbol(fl_ctx, "*scopewarn-opt*"), fixnum(jl_options.warn_scope));
 }
 
@@ -204,13 +214,16 @@ static jl_ast_context_t *jl_ast_ctx_enter(jl_module_t *m) JL_GLOBALLY_ROOTED JL_
         jl_init_ast_ctx(ctx);
     }
     ctx->module = m;
+    if (m)
+        ctx->module_strict_flags = jl_module_strict_flags(m, jl_current_task->world_age);
     return ctx;
 }
 
 static void jl_ast_ctx_leave(jl_ast_context_t *ctx)
 {
     uv_mutex_lock(&flisp_lock);
     ctx->module = NULL;
+    ctx->module_strict_flags = 0;
     ctx->next = jl_ast_ctx_freed;
     jl_ast_ctx_freed = ctx;
     uv_mutex_unlock(&flisp_lock);

src/ast.scm

@@ -489,12 +489,13 @@
 (define vinfo:type cadr)
 (define (vinfo:set-type! v t) (set-car! (cdr v) t))
 
-(define (vinfo:capt v) (< 0 (logand (caddr v) 1)))
-(define (vinfo:asgn v) (< 0 (logand (caddr v) 2)))
-(define (vinfo:never-undef v) (< 0 (logand (caddr v) 4)))
-(define (vinfo:read v) (< 0 (logand (caddr v) 8)))
-(define (vinfo:sa v) (< 0 (logand (caddr v) 16)))
-(define (vinfo:nospecialize v) (< 0 (logand (caddr v) 128)))
+(define (test-bit x b) (< 0 (logand x b)))
+(define (vinfo:capt v) (test-bit (caddr v) 1))
+(define (vinfo:asgn v) (test-bit (caddr v) 2))
+(define (vinfo:never-undef v) (test-bit (caddr v) 4))
+(define (vinfo:read v) (test-bit (caddr v) 8))
+(define (vinfo:sa v) (test-bit (caddr v) 16))
+(define (vinfo:nospecialize v) (test-bit (caddr v) 128))
 (define (set-bit x b val) (if val (logior x b) (logand x (lognot b))))
 ;; record whether var is captured
 (define (vinfo:set-capt! v c)  (set-car! (cddr v) (set-bit (caddr v) 1 c)))

src/builtins.c

@@ -1537,13 +1537,18 @@ JL_CALLABLE(jl_f_declare_global)
 
 JL_CALLABLE(jl_f_declare_const)
 {
-    JL_NARGS(declare_const, 2, 3);
+    JL_NARGS(declare_const, 2, 4);
     JL_TYPECHK(declare_const, module, args[0]);
     if (nargs == 3)
         JL_TYPECHK(declare_const, symbol, args[1]);
     jl_binding_t *b = jl_get_module_binding((jl_module_t *)args[0], (jl_sym_t *)args[1], 1);
-    jl_value_t *val = nargs == 3 ? args[2] : NULL;
-    jl_declare_constant_val(b, (jl_module_t *)args[0], (jl_sym_t *)args[1], val);
+    jl_value_t *val = nargs >= 3 ? args[2] : NULL;
+    uint8_t flags = 0;
+    if (nargs == 4) {
+        JL_TYPECHK(declare_const, uint8, args[3]);
+        flags = jl_unbox_uint8(args[3]);
+    }
+    jl_declare_constant_val(b, (jl_module_t *)args[0], (jl_sym_t *)args[1], val, flags);
     return nargs > 2 ? args[2] : jl_nothing;
 }
 
@@ -1569,14 +1574,14 @@ JL_CALLABLE(jl_f__import)
     }
     else if (nargs == 5) {
         JL_TYPECHK(_import, symbol, args[3]);
-        JL_TYPECHK(_import, bool, args[4]);
+        JL_TYPECHK(_import, uint8, args[4]);
         jl_module_import(jl_current_task, (jl_module_t *)args[0], (jl_module_t *)args[1],
-                         (jl_sym_t *)args[2], (jl_sym_t *)args[3], args[4] == jl_true);
+                         (jl_sym_t *)args[2], (jl_sym_t *)args[3], jl_unbox_uint8(args[4]));
     }
     return jl_nothing;
 }
 
-// _using(to::Module, from::Module)
+// _using(to::Module, from::Module, [flags::UInt8=0])
 JL_CALLABLE(jl_f__using)
 {
     JL_NARGS(_using, 2, 3);

src/gf.c

@@ -4454,7 +4454,7 @@ jl_value_t *jl_new_generic_function_with_supertype(jl_sym_t *name, jl_module_t *
     JL_GC_PUSH1(&ftype);
     ftype->name->singletonname = name;
     jl_gc_wb(ftype->name, name);
-    jl_declare_constant_val3(NULL, module, tname, (jl_value_t*)ftype, PARTITION_KIND_CONST, new_world);
+    jl_declare_constant_val3(NULL, module, tname, (jl_value_t*)ftype, PARTITION_KIND_CONST, new_world, 0);
     jl_value_t *f = jl_new_struct(ftype);
     ftype->instance = f;
     jl_gc_wb(ftype, f);