allow excluding repositories from Project -buildpath via tags

[chrisrueger]

It is related to a forum post some time ago which was the base for Repo-Tagging:

• https://bnd.discourse.group/t/baseline-repo-exclude-from-the-build-path/391/2

e.g. only consider "resolve" repos for Project. getRepositories()

This fixes a cornerish-case problem with compilation errors when you temporarily add a new Repository to your workspace which is purely for lookup/research/browsing purposes and should neither be considered for resolution nor to the buildpath or Eclipse Classpath. It should just "be there" but do nothing.

It fixes a problem I had when I temporarily added an Eclipse Repo for the research purpose above and suddenly got compile errors because the new repo contained a 2.x version of slf4j.simple while the code expected to get an 1.7.x via

-buildpath: slf4j.simple;version=latest

But maybe this is a practical feature, not only for this corner case.

Example

Let's say I temporarily want to add this repo for research & lookup purposes.

-plugin.1.EclipseMilestone:\
	aQute.bnd.repository.p2.provider.P2Repository; \
	 	url = https://download.eclipse.org/eclipse/updates/4.36-I-builds/I20250515-1800/; \
	 	name = "Eclipse-4_36";\
	 	tags = "noResolve"

adding

tags = "noResolve"

allows me to exclude this repository from any -buildpath.

This has two positive effects:

• also the Eclipse classpath ignores this repo (because currently bnd adds -buildpath of all projects to build the Eclipse classpath)
• the build works in case it got broken by the new repository (see slf4j.simple case above)

TODO

This is experimental and only for discussion.
What are drawbacks of this approach? Is it useful? Or should I just not do this or use a narrower version-range instead of 'latest' (-buildpath: slf4j.simple;version=latest)

[laeubi]

Should it not be NO_RESOLVE ?

[chrisrueger]

I think it is correct as it is. My thinking was to use the same logic which is used for Resolving.

The logic in bnd 7.1.0 there is:
Resolving considers only repos with either no tags (for backwards compatibility) or the tag "resolve".

2688cb5#diff-096900d1f971d5ff7606d028e316560c8e89c25031bb473a493b776642f7f347R34 called by

https://github.com/chrisrueger/bnd/blob/180a01a327d1ca3b001206ace8d024dd3994cd7a/biz.aQute.resolve/src/biz/aQute/resolve/BndrunResolveContext.java#L410

In other words: The repositories a Project should "care" about are the same repositories which are considered for resolving by BndrunResolveContext.java

[laeubi]

It should just "be there" but do nothing.

I think it would be more useful to allow adding repositories to individual (bndrun) files, the whole problem seem to stem from the fact that repositories need to be added on the workspace level.

[chrisrueger]

It should just "be there" but do nothing.

I think it would be more useful to allow adding repositories to individual (bndrun) files, the whole problem seem to stem from the fact that repositories need to be added on the workspace level.

Maybe that would solve a different problem.

The change I added is on the Project and the project gets a list of Repositories to do its work for -buildpath and what not. The project itself and bndrun are at this point not related (bndrun comes into play later).

So the current discrepancy (in my opinion) is that:

• -buildpath considers ALL Repos in the Workspace
• .bndrun resolving can look at only a subset (because of the tagging and considering repos with either no tags (Backwards compatibility or the tag "resolve")

My idea is to have both use the same logic using Repo Tags.

[laeubi]

The classpath is controlled completely by the user and nothing is resolved (as far as I know) it actually look up the bundles, therefore only using resolve repositories might be confusing. So I would argue that if you don't want a higher version then simply adjust the version range in your classpath instead of try to "ignore" a source of bundles.

Bndruns are working differently there is a resolve phase (either explicitly or implicitly) where only the metadata of respositories is used to collect all transitive required things. These are then used in the execution phase where the actual bundles are fetched (from the repositories).

[chrisrueger]

The classpath is controlled completely by the user and nothing is resolved (as far as I know) it actually look up the bundles,

What does "controlled by the user" mean? How can I control it?

Let me try again:

The effect I saw after adding the additional Repository was that:

• a command line build failed
• and Eclipse showed also compile errors

Because the new repo made a higher version available which was chosen by the -buildpath because of the latest (classifier). So far, so correct.

I debugged a bit and saw that the ProjectBuilder.java is calling Project.getBuildPath().

Also BndContainerInitializer (the Eclipse part) does it.

And project.getBuildPath() currently considers ALL repos.

The current behavior is absolutly correct ( am not saying that it is wrong). And you are right, I could solve it by not using "latest" and use a stricter version range.

BUT: I would like to "control" it in a way, that I can exclude a Repository from -buildpath (as also requested in the Forum) and also from resolver stuff.
I just want to have the control-knob at a different place.

Basically the same way I can control it in .bndrun files.
In .bndrun I can specify either:

• a list of repos via -runrepo
• or provide no -runrepo which causes to use all Repositories without tags or the tag "resolve".

If the confusion is about the tag name "resolve" although I am talking about -buildpath then yes, we could also think about introducing a "buildpath" repo-tag which is only used in the context to control it for buildpath.

[pkriens]

I think when we worked on the tags it was implied that we would also apply them to the buildpath.

I think a special tag for this would be much better than reusing the resolve tag. I.e. the resolve tag is about runtime dependencies and the buildpath should be about API compile time dependencies.

[chrisrueger]

Thanks @pkriens , and welcome back :)

I think when we worked on the tags it was implied that we would also apply them to the buildpath.

I just realized that recently when I re-read the forum post in the context of this PR here. I completely forgot about it when we worked on tags. I was so focused on .bndrun at that time.

I think a special tag for this would be much better than reusing the resolve tag. I.e. the resolve tag is about runtime dependencies and the buildpath should be about API compile time dependencies.

Ok makes sense and basically supports my last sentence above ("we could also think about introducing a "buildpath" repo-tag which is only used in the context to control it for buildpath.")

Ok, I will add a new tag.
Is buildpath for the tagname ok? What about -testpath?
Or would compile be better?

[chrisrueger]

@pkriens
If we introduce a new tag for compilation e.g. compile and we already have resolve I think we might have created a 7.1.0 backwards compatibility problem situation.

Example:

• If somebody has already tagged a repo with something e.g. resolve

Now if we filter for the existence of tag with

if (tags == null || tags.includesAny(Constants.REPOTAGS_COMPILE))

then the repo which already has a tag, but definitely does NOT have the new 'compile' tag.
So suddenly builds would fail in 7.2.0 (see the breaking commit 61cd38a )

I wonder how the filter condition should look like?
Do we maybe need to the opposite? a nocompile tag? (I added that in commit 00d339a )

Maybe this is what @laeubi was thinking with his comment:

Should it not be NO_RESOLVE ?

So maybe we should re-think the tagging a bit. I have the feeling our current filtering made sense back then, when we were just had one tag (resolve) - let call it 'includes' filtering.
But now with more tags, we maybe need to think about 'excludes' filtering.

[chrisrueger]

I added back the positive tag check "compile" (in addition to negative "nocompile" precedence ) in e467b0b
I solved my backwards compatibility dilemma by now also providing TRANSITION phase:

• For one release cycle in 7.2.0 we silently promote every repository that has resolve but not compile into the new compile when the caller asks for compile.

• Until version 7.1.x having the tag list resolve implied visibility in
  both the resolve and the compile phase.

• Starting with 7.2.0 that implication is deprecated.

  • 7.2.0 still honours it for backward compatibility and emits a warning.

• 7.3 will no longer honour it.

[peterkir]

I have many discussions about compile time and runtime dependencies with my colleaques and would appreciate having an more explicit way of making it visible on a repository base.
For the current tag usage I fould the following documentation https://bnd.bndtools.org/chapters/870-plugins.html#tagging-of-repository-plugins
Could you extend this with and example for using a repo only for compile and another only for runtime dependencies.
I was not able to quickly grab the description above, especially as it is also mixing the intent/api and the implementation with consideration of the existing historical implementation.

[chrisrueger]

Could you extend this with and example for using a repo only for compile and another only for runtime dependencies.
@peterkir

So with this PR here it would be possible to specify:

• RepoA:
  • no tags (e.g. existing repo)

Meaning: the backwards-compatibility default. repo is considered for all phases (resolve and compile)

• RepoB:
  • tags='compile'

Meaning: repo is only considered for -buildpath, but not resolution (resolution means the 'Resolve' button in .bndrun editor. And with -buildpath currently it also means -testpath)

• RepoC:
  • tags='resolve'

Meaning (actual) repo is only considered for resolution, but not -buildpath
But: in 7.2.0 I added a transition hack so that this a repo with 'resolve' is considered also for 'compile'. This is to avoid breaking current builds. It will be removed in 7.3.0 where the actual meaning would apply. This needs to be communicated in release notes.

• RepoD:
  • tags='resolve,compile'

Meaning: repo is only considered for both resolution and -buildpath

Yesterday I added also no+[tag] negative tags.

• RepoE:
  • tags='resolve,nocompile'

Meaning (actual) repo is only considered for resolution, but not -buildpath.
This is e.g. needed in the 7.2.0 Transition phase mentioned above to exclude a repo from -buildpath but keep it in resolution

I tried adding that also in the tests 977955e

Let's discuss

[chrisrueger]

Discussed this in the call.
I will do some more rework.

about the 7.1.0 Backwards Compatibility problem

introduce a tags.active list

# build.bnd
tags.active=resolve

and basically do something like this (pseudocode)

public List<RepositoryPlugin> getRepositories() {

        return workspace.getRepositories()
            .stream()
            .filter(repo -> {
                Tags tags = repo.getTags();

                if (!tags.active.contains("compile") || tags.includesAny("compile")) {
                    return true;
                }

                if (!tags.active.contains("resolve") || tags.includesAny("resolve")) {
                    return true;
                }

                return false;
            })
            .toList();
    }

also change the signatore to

public List<RepositoryPlugin> getRepositories(String... tags) {}

and

Repo Tags

Basically discussed to have the following repo-tags:

• compile
• test
• resolve
• debug

.bnd
-buildpath -> project.getRepositories(compile)
-testpath -> project.getRepositories(compile,test)

.bndrun
-runrepos -> bndrun.getRepositories(resolve)

TODO debug tag

-> bndrun.getRepositories(resolve, debug) but conditional depending on some property (TODO discuss with Peter Kirschner)

• also would be good to be able to specify a list of repo-tags e.g. in -runrepos so that you can specify the tags you want to resolve or debug (To be discussed with @peterkir )