Stop skipping tests in test_against_taxsim.py

[rickecon]

In the current version of the repository (v. 0.0.1), the five tests are skipped in test_against_taxsim.py.

fiscalsim_us/tests/microsimulation/test_against_taxsim.py sssss

These tests are skipped if the operating system is Windows or Mac OSx. But they should pass if run on a Linux operating system. We should update the code to satisfy the following conditions.

• Verify that tests pass on a local machine with a Linux operating system. @mitchellpound can run this on his Linux machine.
• Get the test to detect the operating system (Linux, Windows, Mac OSx), select the appropriate taxsim executable, and run the tests. I think there is some code already in place to do this. We should probably not store any executables in the repository, so we should probably have the test download the appropriate taxsim executable.

I asked PolicyEngine-US about this in their repo (see this issue), and the answer was that they set it up to work on a Linux machine because that is the OS of the GitHub CI, but didn't finish setting it up for all operating systems.

cc: @ss7886

[ss7886]

I tweaked test_against_taxsim.py just a little bit and got all five tests passing on my machine. I can submit a PR so you can try running the tests on Mac as well @rickecon.

[rickecon]

@ss7886 . Yes. Please submit those changes as a PR.

[rickecon]

@ss7886 found that on his Windows machine, there seems to be an incompatibility between the pandas.HDFStore() method and pytables or tables. He found that pd.HDFStore tried to download tables version 3.6.1.

• This thread on the pandas repository (BUG: PeriodIndex inconsistent deserialization with HDF5 - PyTables pandas-dev/pandas#41978) documents a similar issue and provides a workaround.
• If that doesn't work, I recommend trying to use an HDF5 specific package, maybe h5py to load and save HDF5 files, then read those intermediate objects into pandas. This would take the load off of the pd.HDFStore method in policyengine-core's dataset.py module.

I would like @ss7886 and @mitchellpound to work on this.

[ss7886]

@mitchellpound could you send me screenshots of the results you're getting when you run make test on your linux machine (just showing what tests are passing, failing, or skipping)? Rick told me that you were failing the tests in test_against_taxsim.py and test_cps.py. I have a couple ideas off the bat of how we could get those to run across all platforms, but maybe we could look at that together sometime for just half an hour or so and see if anything works.

[mitchellpound]

@ss7886 Here's the summary of the make test.

platform linux -- Python 3.9.15, pytest-5.4.3, py-1.11.0, pluggy-0.13.1
rootdir: /home/mitchlb/Research/fiscalsim-us
plugins: dependency-0.5.1
collected 1072 items                                                                                   

fiscalsim_us/tests/test_variables.py ........................................................... [  5%]
................................................................................................ [ 14%]
................................................................................................ [ 23%]
................................................................................................ [ 32%]
................................................................................................ [ 41%]
................................................................................................ [ 50%]
................................................................................................ [ 59%]
................................................................................................ [ 68%]
................................................................................................ [ 77%]
................................................................................................ [ 86%]
................................................................................................ [ 95%]
.........................................                                                        [ 98%]
fiscalsim_us/tests/microsimulation/test_against_taxsim.py FFFFF                                  [ 99%]
fiscalsim_us/tests/microsimulation/test_microsim.py .                                            [ 99%]
fiscalsim_us/tests/microsimulation/data/test_imports.py ..                                       [ 99%]
fiscalsim_us/tests/microsimulation/data/acs/test_acs.py ss                                       [ 99%]
fiscalsim_us/tests/microsimulation/data/cps/test_cps.py Fs                                       [100%]

======================================= short test summary info ========================================
FAILED fiscalsim_us/tests/microsimulation/test_against_taxsim.py::test_federal_tax_against_taxsim - V...
FAILED fiscalsim_us/tests/microsimulation/test_against_taxsim.py::test_state_income_tax_against_taxsim[MD]
FAILED fiscalsim_us/tests/microsimulation/test_against_taxsim.py::test_state_income_tax_against_taxsim[MA]
FAILED fiscalsim_us/tests/microsimulation/test_against_taxsim.py::test_state_income_tax_against_taxsim[NY]
FAILED fiscalsim_us/tests/microsimulation/test_against_taxsim.py::test_state_income_tax_against_taxsim[WA]
FAILED fiscalsim_us/tests/microsimulation/data/cps/test_cps.py::test_cps_dataset_generates[2020] - ta...
================== 6 failed, 1063 passed, 3 skipped, 38 warnings in 61.44s (0:01:01) ===================