Windows problem with SetConsoleOutputCP CP_UTF8 and chcp 65001

[madah81pnz1]

Some background:
https://hydrogenaudio.org/index.php/topic,128969.0.html
https://hydrogenaudio.org/index.php/topic,129063.0.html

Which seems to match the problems reported here:
https://stackoverflow.com/questions/39736901/chcp-65001-codepage-results-in-program-termination-without-any-error
https://superuser.com/questions/1118106/can-the-utf-8-code-page-identifier-65001-be-different-on-other-computers
https://stackoverflow.com/questions/878972/windows-cmd-encoding-change-causes-python-crash

So to summarize: SetConsoleOutputCP with CP_UTF8 or chcp 65001 only works on Windows 8 and later, and is likely to cause crashes in Windows 7 and earlier.
But still it seems bugs appear in Windows 10; output might be duplicated due to WriteFile() returns the number of UTF-16 characters written, not UTF-8 bytes.

The conclusion then is to only use the W APIs directly on Windows; WriteConsoleW() with UTF-16 if writing to console, and UTF-8 if redirected to file. This should work the same on all Windows versions (or at least since Windows XP).

For writing binary, WriteFile() should be used instead of WriteConsoleW(). But writing binary to the console doesn't make sense on Windows anyway, best to prevent this in some way?

Care must be taken to make sure piping works as expected, for example:
metaflac --export-tags-to=- file1.flac | metaflac --import-tags-from=- flac2.flac

Might be worth testing tags with broken UTF-8 also, as I suspect this will likely have problems in Windows due to being converted to UTF-16; if it works at all, it will be a lossy operation, which the user might find surprising, especially if the goal is to copy tags.

But also with binary output/input:
metaflac --list --data-format=binary file1.flac | metaflac --append --data-format=binary file2.flac

Due to limitations on PowerShell, it seems binary can't work due to it being UTF-16 only, and so it would be best to implement something like --data-format=hex.

I can do some testing on Windows XP in due time, and will post my findings here on what works and what doesn't.

[madah81pnz1]

One problem is fputws(), which only translates from wide to the current codepage. Using it to print emojis doesn't work. According to documentation, setting _O_BINARY might work, but then it also means \r\n won't be translated.
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fputs-fputws?view=msvc-170

I think it have a working fix, by removing SetConsoleOutputCP and removing all _setmode calls, and using WriteFile with the original UTF-8 instead of fputws(). This appears to be similar to how Python does it.

The downside is that this doesn't translate \n to \r\n on Windows anymore. Python does the newline translation itself, which flac could also do, but this adds complexity. Cmd, powershell and msys2 all seem to be fine with only \n, atleast on Windows 10.

[madah81pnz1]

Next problem is stdin. Reading directly from console with ReadConsoleW() works well, you get the expected UTF-16 and can convert into UTF-8.
The problem is when stdin is redirected:

echo ARTIST=åäöÅÄÖ | metaflac.exe --remove-all-tags --import-tags-from=- file.flac

The actual buffer as read by ReadFile:

0x41 0x52 0x54 0x49 0x53 0x54 0x3d 0x86 0x84 0x94 0x8f 0x8e 0x99 0x20 0x0d 0x0a
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The highlighted bytes are in codepage 850, not UTF-8.
Here --no-utf8-convert becomes useful, but if you DON'T pass in --no-utf8-convert then everything from a redirected stdin will be converted from the current codepage.
This also means it is impossible to use emojis unless the input already happens to use UTF-8 AND you pass --no-utf8-convert to metaflac.

flac 1.5.0 without --no-utf8-convert results in mojibake:

X:\>chcp 65001
Active code page: 65001

X:\>echo ARTIST=åäöÅÄÖ | metaflac.exe --remove-all-tags --import-tags-from=- file.flac

X:\>metaflac.exe --list file.flac
(...)
METADATA block #2
  type: 4 (VORBIS_COMMENT)
  is last: true
  length: 86
  vendor string: reference libFLAC git-3f1ecff8 20260304
  comments: 1
    comment[0]: ARTIST=åäöÅÄÖ

flac 1.5.0 with --no-utf8-convert has correct result:

X:\>chcp 65001
Active code page: 65001

X:\>echo ARTIST=åäöÅÄÖ | metaflac.exe --no-utf8-convert --remove-all-tags --import-tags-from=- file.flac

X:\>metaflac.exe --list file.flac
(...)
METADATA block #2
  type: 4 (VORBIS_COMMENT)
  is last: false
  length: 71
  vendor string: reference libFLAC git-3f1ecff8 20260304
  comments: 1
    comment[0]: ARTIST=åäöÅÄÖ

But if you try to set the original codepage back to 850, it won't work, you get mojibake.
This is flac 1.4.3 (since it doesn't override the codepage):

X:\>chcp 850
Active code page: 850

X:\>echo ARTIST=åäöÅÄÖ | metaflac.exe --no-utf8-convert --remove-all-tags --import-tags-from=- file.flac
file.flac: ERROR: tag value for 'ARTIST' is not valid UTF-8

X:\>echo ARTIST=åäöÅÄÖ | metaflac.exe --remove-all-tags --import-tags-from=- file.flac

X:\>metaflac.exe --list file.flac
(...)
METADATA block #2
  type: 4 (VORBIS_COMMENT)
  is last: false
  length: 75
  vendor string: reference libFLAC git-3f1ecff8 20260304
  comments: 1
    comment[0]: ARTIST=†„”�Ž™

[madah81pnz1]

Another problem is that encode_utf8() and decode_utf8() in utf8.c uses CP_ACP for MultiByteToWideChar()/WideCharToMultiByte().
This is one cause of mojibake, since it converts via codepage 1252 (on my system), even if the console has been set to codepage 65001 (CP_UTF8).

#if defined _WIN32
	SetConsoleOutputCP(65001);
	SetConsoleCP(65001);
	printf("GetACP()             = %u\n", (unsigned)GetACP());
	printf("GetOEMCP()           = %u\n", (unsigned)GetOEMCP());
	printf("GetConsoleOutputCP() = %u\n", (unsigned)GetConsoleOutputCP());
	printf("GetConsoleCP()       = %u\n", (unsigned)GetConsoleCP());
#endif

Output:

GetACP()             = 1252
GetOEMCP()           = 850
GetConsoleOutputCP() = 65001
GetConsoleCP()       = 65001

Doing the conversion in set_vc_field() is too late in my opinion, since this data can come from many sources; command line args, stdin, pipe or a file on disk.

On Windows, the (wide) command line args and the console stdin/stdout is always UTF-16, so it doesn't make sense to do any conversion to or from CP_ACP, or any other codepage.

When reading/writing to a file on disk, either directly or redirected, it is best to assume UTF-8 and ignore any codepage, since that is least surprising to a user that e.g. edits that file in a text editor. The reason is also that the CP_OEM codepage used for the console is usually different from the CP_ACP codepage used for non-Unicode aware apps; the user would likely see mojibake.

Piping between processes is then the tricky case, when the codepage has to be honoured. Here there is no other way to agree upon which encoding to use, other than via the console codepage. If the user wants UTF-8 they would use something like chcp 65001 or pass in --no-utf8-convert.

Edit: The problem in decode_utf8() explains the issue in my previous comment, the "flac 1.5.0 without --no-utf8-convert results in mojibake":

  comment[0]: ARTIST=åäöÅÄÖ

This matches this python code, which shows how the encode/decode happens:

>>> "åäöÅÄÖ".encode("UTF-8").decode("Windows-1252")
'åäöÅÄÖ'

And also this:

  comment[0]: ARTIST=†„”�Ž™

Matches:

>>> "åäöÅÄÖ".encode("cp850").decode("Windows-1252", errors='replace')
'†„”�Ž™'