Test and utilities improvements

[EchterAgo]

I'm starting this discussion for ideas about the tests. There are some things that can be improved:

• remove old tests from windows-build-test.yml
• fix the drive letter issue in utils (maybe not an issue with the test itself)
• add some better scripting for analyzing dump files including remote kdb support
• add !stacks and !memusage to parsedump
• make some tests for previously fixed issues
• update the rest of the python tests to use utils.py
• update debug instructions, driver installation with the WDK deployment option fails a lot, maybe make it use zfsinstaller. we should get unmount / driver unload always working properly too

[sskras]

I would like to ask a beginner's question:

has the tests for Windows something to do with the test for *nix systems (like the common test-bootstrapping script or at least the common folder for different scripts or different, OS-specific subfolders?

[EchterAgo]

AFAICT they don't but I'm wondering if they should. I haven't really looked at the other tests yet.

[lundman]

OK so ultimately I want to run the zfstest suite of tests, it is in ksh script, thousand of files, and has a whole bunch of Unix assumptions, like sudo, uid, groups and all that.

The best way around that is to run zfstest in WSL shell, and have it execute zfs as zfs.exe in "Windows mode", to do things to the drives mounted/mapped to WSL.

We have a proof-of-concept of that somewhere, and it can run a few tests with some prep. One is to make the Windows drive "map" to Unix - it would be good if this could be automated.

Beyond that, we do need a few more tests, that are more Windows specific. Running ifstest.exe is a good idea, since we will need to make sure that passes to appease the Microsoft elders. And certainly no reason not to run btrfs' test.exe as well.
We also have a few tests done for things we have already come across. We should keep adding things here that are Windows specific.

But I do not think we should add ZFS specific tests, they should be part of zfstests to ensure compliance with the tests upstream has, and adds to.

[EchterAgo]

I got a self hosted github runner installed on a beefy machine now and will use that to try to do the WSL thing. Should be much easier to debug when I have direct access to the runner.

[EchterAgo]

The new runner is quite fast, much faster than the GitHub provided ones. It took some time to set it up, you basically have to build the image yourself from https://github.com/actions/runner-images, which only works on Azure. Azure didn't let me download the generated image to use on my own machine so I modified the packer configuration to build the image on one of my local libvirtds.

After that I made a new image backed by the packer image, downloaded the runner and set it up. Note that the runner can not be set to run as Network Service or Local Service, it has to run as Local System or some other user account, otherwise the root certificate import fails because permission is denied.

I already managed to BSOD the virtual machine running the tests :\

[EchterAgo]

So it turns out the same runner runs all the tests in parallel. We could make this sequential to reduce failures, but I think our tests should be robust in this situation. With this new self-hosted runner I can also still get information out after the machine crashes.

Edit: only the first 2 jobs are run on the same machine, the rest seems to run on github hosted runners. I might need more runners or some way to override runs-on

[EchterAgo]

I think we should also have a test that tests exercises lazy writes, flush on unmount and a check that everything has been written by importing the pool.

[sskras]

@andrebraga commented 30 minutes ago in #469:

for the time being having basic sanity checks ("as unprivileged user: can create file, can remove file just created; same with folders") would be great.

Sure, and maybe some of these ops are already being tested, eg:

openzfs/.github/workflows/windows-build-test.yml

Lines 2507 to 2514 in bce453c

	- name: test
	run: |
	${{ steps.drivenoslash.outputs.drive }}
	cd ${{ steps.drive.outputs.drive }}\winfstest
	$env:PYTHONPATH='${{ steps.drive.outputs.drive }}\winfstest'
	python.exe -u "${{github.workspace}}\contrib\windows\tests\winfstests.py" -path ${{github.workspace}}\

I am just not yet sure about how does winfstests.py exactly work, eg:

openzfs/contrib/windows/tests/winfstests.py

Lines 216 to 235 in bce453c

	for test in ['.\\t\\base\\00.t', '.\\t\\base\\01.t',
	'.\\t\\base\\02.t', '.\\t\\base\\03.t',
	'.\\t\\base\\04.t', '.\\t\\base\\05.t',
	'.\\t\\base\\06.t', '.\\t\\base\\07.t',
	'.\\t\\base\\08.t', '.\\t\\base\\09.t',
	'.\\t\\base\\10.t',
	'.\\t\\reparse\\00.t', '.\\t\\reparse\\01.t',
	'.\\t\\reparse\\02.t', '.\\t\\reparse\\03.t',
	'.\\t\\streams\\00.t', '.\\t\\streams\\01.t',
	'.\\t\\streams\\02.t', '.\\t\\streams\\03.t',
	'.\\t\\streams\\04.t']:
	preTest(str(test) + " tests:")
	# f = PureWindowsPath(get_driveletters()[0][1])
	ret = runWithPrint(["python3.exe", str(test)])
	time.sleep(10)
	if ret.returncode != 0:
	print("FAIL")
	postTest()

What are all these file names like .\t\base\08.t, please ?