CI: Testing Buildkite stdlib filtering: make sure all stdlibs are run on release-*

[DilumAluthge]

No description provided.

[DilumAluthge]

In the logs, I'm seeing:

EXTERNAL_STDLIBS_SKIPPED:  ArgTools DelimitedFiles LazyArtifacts LibCURL NetworkOptions SHA Statistics SuiteSparse Tar

@KristofferC This seems incorrect - shouldn't we run all external stdlibs here? Because the target (base) branch of my PR is release-1.11?

[DilumAluthge]

I haven't verified this hypothesis yet, but my suspicion is that we are handling the case of e.g. a push to release-* or a push to backports-release-*, but we aren't handling the cases of:

1. A PR that targets release-*
2. A PR that targets backports-release-*

I think that it would be nice if those two cases also ran all external stdlibs? My thinking this that those kinds of PRs are relatively low frequency, and thus running all external stdlibs wouldn't be a huge burden on our CI queues. And if a PR is targeting one of those branches, then presumably that PR is involved in some way in the release process, so IMO it seems useful to have all external stdlibs being run on those PRs?

[KristofferC]

We can first test the hypothesis by creating some dummy release-1.141 branch and check

[DilumAluthge]

I'll try that.

I've cancelled CI on this PR, to save CI time.

[KristofferC]

EXTERNAL_STDLIBS_SKIPPED: ArgTools DelimitedFiles LazyArtifacts LibCURL NetworkOptions SHA Statistics SuiteSparse Tar

Why did it not skip e.g. Pkg?

[DilumAluthge]

EXTERNAL_STDLIBS_SKIPPED: ArgTools DelimitedFiles LazyArtifacts LibCURL NetworkOptions SHA Statistics SuiteSparse Tar

Why did it not skip e.g. Pkg?

Yeah I also found that confusing.

[DilumAluthge]

EXTERNAL_STDLIBS_SKIPPED: ArgTools DelimitedFiles LazyArtifacts LibCURL NetworkOptions SHA Statistics SuiteSparse Tar

Why did it not skip e.g. Pkg?

Yeah I also found that confusing.

Although note that we do separate some tests into "net" vs "non-net". E.g. I think Pkg and LibGit2 are "net", and thus go into a separate job (where we don't do rr).

[DilumAluthge]

We can first test the hypothesis by creating some dummy release-1.141 branch and check

Here is the Git branch: https://github.com/JuliaLang/julia/tree/release-9.999

I have triggered a Buildkite¹ build here: https://buildkite.com/julialang/julia-release-9-dot-999/builds/1

Footnotes

1. Note: I created a separate Buildkite pipeline (https://buildkite.com/julialang/julia-release-9-dot-999) for this. This is because the julia-master Buildkite pipeline (https://buildkite.com/julialang/julia-master) will not run release branches. This is intentional. ↩

[KristofferC]

The link doesn't work for me.

[DilumAluthge]

Ah. I fixed a setting - try it now.

[KristofferC]

 clang -mmacosx-version-min=11.0  -std=gnu11 -pipe -fPIC -fno-strict-aliasing -D_FILE_OFFSET_BITS=64 -Wformat -Wformat-security -mcrc -I[buildroot]/src -I[buildroot]/src -I[buildroot]/src/support -I[buildroot]/usr/include -ffreestanding -shared -O3 -g  -DDEP_LIBS='"libgcc_s.1.1.dylib:libopenlibm.dylib:@:@libjulia-internal.9.999.dylib:@libjulia-codegen.9.999.dylib:"' ./loader_lib.o ./loader_trampolines.o -Wl,-rpath,'@loader_path/' -o [buildroot]/usr/lib/libjulia.9.999.0.dylib  -Wl,-compatibility_version,9.999 -Wl,-current_version,9.999.0  -ffreestanding -L[buildroot]/usr/lib -L[buildroot]/usr/lib
ld: malformed 32-bit x.y.z version number: 9.999
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [[buildroot]/usr/lib/libjulia.9.999.0.dylib] Error 1

ERROR: LoadError: Unsatisfiable requirements detected for package TestPkg [152c8e49]:
 TestPkg [152c8e49] log:
 ├─possible versions are: 0.1.0 or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions: 0.1.0
 └─restricted by julia compatibility requirements to versions: uninstalled — no versions left
Stacktrace:

etc. Think it is better to try with something that is still semver compatible.

[DilumAluthge]

https://buildkite.com/julialang/julia-release-9-dot-999/builds/2

I changed it to 1.99.0, hopefully that'll fix those issues.

[KristofferC]

Did that run with the custom buildkite branch? Looking at the diff I don't see it being changed.

[DilumAluthge]

Ugh, I must have Friday brain or something.

Fixed. https://buildkite.com/julialang/julia-release-9-dot-999/builds/3

[KristofferC]

Branch is: release-9.999
Pipeline is: julia-release-9-dot-999
On important branch 'release-9.999' or pipeline 'julia-release-9-dot-999': running all external stdlib tests

[KristofferC]

Regarding PRs against the backport branch that do not touch the .version but still have the external stdlib tests run, I think this could be handled as an additional incremental change if it is desired.