2.41(.1) breaks Deutschlandticket Aztec code

[rugk]

STR

1. Scan your source pkpass/website with https://zügli.app/ to verify the D-Ticket works.
2. Import pkpass file.
3. Click save without changing the binary data shown there.
4. Scan it from Catima with https://zügli.app/

What happens

The Catima imported file cannot be read! This causes quite an inconvenience when being checked by a ticket inspector. (because they apparently also got errors)

It also seems to affect already saved old tickets/Aztec codes).

What should happen

Do not mangle with contents?

important version notes

I could reproduce this one one over with

• Catima v2.41.1
• both with the vendor provided pkass and the one you can get when scanning a valid ticket with Zügli.
• also when scanning with the app de.erichambuch.ticketreader (Version 2.39.0) as an alternative scanner implementation

I could not reproduce it with:

• Catima 2.40.0 (other device)

Both from F-Droid.

Also from ticket inspector checking timings, the breakage highly correlates with the app update.

additional notes

For obvious reasons I cannot provide the ticket here, but the Zügli team got a mail with details, I can also forward that to you if needed.

[TheLastProject]

If these are non-UTF8 codes this is very likely the same issue as #2921 and thus should be already fixed in 2.41.2. Please double check. Otherwise I will need a pkpass/barcode to debug the difference I fear.

[TheLastProject]

It would really be great to test newly imported pkpass files too.

2.41.X does change the behaviour of pkpass file importing to actually store the given encoding which, as written in the other issue, may cause unexpected changes.

So while 2.41.2 will change back old passes (with unknown encoding as it wasn't saved), it may still go wrong with pkpass files which explicitly state an encoding as those will get stored now. That was the part of the fix I wasn't 100% sure about.

[TheLastProject]

Even better if I could see the pkpass to see what they look like (I don't live in Germany so I have never seen a Deutschlandticket before). Email would be great :)

[TheLastProject]

@rugk Instead of sending me the full pkpass file (which you're free to do), could you please confirm for me if the pass.json in it has messageEncoding set to iso-8859-1 and, if not, to what it is set?

Sleeping over the fix for #2921, I'm starting to get the feeling it's not enough to fix pkpass and #2934 is necessary. Having it confirmed the pkpass file looks as I think it looks instead of guessing will help me be extra sure about this fix being the way to go, although I'm pretty sure it is the correct fix (well, as far as I can test, I can't test SJIS).

[rugk]

It would really be great to test newly imported pkpass files too.

Same issue with the affected file, BTW.

As for the pkpass file I'll drop you a mail at the mail in your mail set on GitHub.

As for v2.41.2 I'll need to wait for the F-Droid update…

[TheLastProject]

Okay, I can tell 2.41.2 will not fix this. However if you say it doesn't scan since 2.41.X we've got a bit of an issue here.

I can fix the pkpass version using #2934, however, I can't fix a direct camera scan without breaking #2555 again, as the barcode data is filled with binary characters and zxing itself thinks it should use UTF-8 for it, so a physical scan (which will set the barcode encoding to "automatic" as it can't know the format) will use the incorrect version.

I will at least go and make another new release quickly that fixes the pkpass version, but I'm not sure what to do about the non-pkpass version (luckily there is a workaround, users can manually set the encoding to ISO-8859-1, but that's not really great).