license-classifications.yml contains duplicate id Unlicense and LicenseRef-scancode-unlicense

[mawl]

During upgrade of our own license-classifications.yml I have noticed that there are two entries for maybe the same license in your license-classifications.yml:

• Unlicense and LicenseRef-scancode-unlicense

LicenseRef-scancode-unlicense can't be found via web ui in scancode-licensedb and isn't used in one of your curations.

I haven't proved that for other ids - the only thing remarkable is, that the lines of file have increased from ~7400 to ~11500 since we last have updated around Feb, 2023.

[sschuberth]

License classifications are auto-generated by now, by importing ScanCode's LicenseDB as last happened ~3 weeks ago. While that maybe explains the increase in classifications, it does not explain why we have the Unlicense twice. Maybe @fviernau, who wrote the ScanCodeLicenseDbClassifications code, has an idea?

[fviernau]

I'll need to look into this in detail later.

As a general remark, I believe we should have one classification per distinct license identifier.
So, if there are tow IDs for the same license, two classifications are needed.

[mawl]

@fviernau: LicenseRef-scancode-unlicense has no match in the web UI and the json either, so it seems that this classification is obsolete.

[fviernau]

ScanCode license DB defines a unique key for each license.
The corresponding license identifier of each license is: LicenseRef-scancode-$key.
License which are on the SPDX license list additionally have an: spdx_license_key.

The code classifies both license identifiers corresponding to licenses on the SPDX license list, see:

ort-config/tools/curations/buildSrc/src/main/kotlin/ScanCodeLicenseDbClassifications.kt

Lines 194 to 201 in cb8fa52

	val categorizations = licenseDetails.flatMap { details ->
	details.getLicenseIds().map { licenseId ->
	LicenseCategorization(
	id = licenseId,
	categories = details.getCategories()
	)
	}
	} + LICENSE_EXCEPTION_CATEGORIZATIONS

.

  {
    "license_key": "unlicense",
    "category": "Public Domain",
    "spdx_license_key": "Unlicense",
    "other_spdx_license_keys": [],
    "is_exception": false,
    "is_deprecated": false,
    "json": "unlicense.json",
    "yaml": "unlicense.yml",
    "html": "unlicense.html",
    "license": "unlicense.LICENSE"
  }

This behavior was intended. To me it still makes sense. I propose to simply close this issue. What do you think @sschuberth ?

[sschuberth]

This behavior was intended. To me it still makes sense. I propose to simply close this issue. What do you think @sschuberth ?

I think we should align with how ORT derives SPDX license IDs for ScanCode licenses, which is to use spdx_license_key if present, and only fall back to LicenseRef-scancode-$key if it is not present, see this code.

With that logic, we'd only have one entry for the Unlicense, which I agree with @mawl to make sense. There should also be no risk to miss a classification by omitting the LicenseRef-scancode-unlicense entry, as the lookup by ORT would always happen via the Unlicense key.

[fviernau]

There should also be no risk[..] as the lookup by ORT would always happen via the Unlicense key.

That lookup is not the only source of license identifiers. The license identifiers may also come from curations.
(Also considering the case, when a license which has not been part of the SPDX license list gets added to that list.)

So, in order to minimize the amount of manual work, we could simply keep these classified?

I wonder if a mechanism, which allows defining equality relationship between licenses, or rather alias relationship
would be beneficial also in other scenarios. It could be used here to tell ORT that some LicenseRef-* actually is some SPDX license and to instruct ORT to map all occurences.

[sschuberth]

Also considering the case, when a license which has not been part of the SPDX license list gets added to that list.

Ok, that is a convincing case, although it happens quite rarely.

or rather alias relationship would be beneficial also in other scenarios.

BTW, LicenseLynx can provide such aliases.

It could be used here to tell ORT that some LicenseRef-* actually is some SPDX license and to instruct ORT to map all occurences.

I fear that aliases in classifications could open up a can of worms, like how to deal with aliases explicitly being classified differently.

[sschuberth]

So, in order to minimize the amount of manual work, we could simply keep these classified?

All in all, this discussion makes me wish that we would drop classifications completely from this repository. It's close to the border of giving legal advice anyway, which we always wanted to avoid. I believe we should it make easy in ORT to import / use existing classifications (also see oss-review-toolkit/ort#9620), but not provide any.