More flexible 2dclean areas.

[aewallwi]

Closes #99 . This pull request changes how cleaning areas are calculated for 2d clean in a way that only affects multiple clean regions (it does not change the functionality for when single clean regions are provided which could break the API). This change allows for calculating more flexible regions.

_clean_filter is provided a list of filter centers for each dimension and a list of corresponding filter region widths.

Originally, _clean_filter would compute a vector along each dimension, set to one in each filter region. The overall filtering area A would be computed by taking the outer product of the two vectors, A = (a_0^0 + a_0^1, + a_0^2 ... + a_0^N-1) outer (a_1^0 + a_1^1 + ... a_1^(N-1) )

I have changed this calculation to effectively computes the overall cleaning area, A as

A = (a_0^0 outer a_1^0) | (a_0^1 outer a_1^1) | ... U (a_0^N-1 outer a_1^N-1)

where | is bitwise or. With this system, one can reproduce areas from the original scheme but not vice versa so it is more flexible.

This scheme is also applied to the 'plus' 2d filtering mode which originally computed plus regions for every possible pair of time / frequency regions provided but now only computes plus regions for each pair provided (requires the number of time / frequency filter regions to be the same).

[aewallwi]

I use the following comparison to ensure that the old and new versions of clean are the same when a single cleaning region is provided for both plus and rect modes. I generate test-data using copy-pasted code from test_dspec.test_fourier_filter

dt = 10.7374267578125
sdf = 100e3
nf = 100
ntimes = 120
freqs = np.arange(-nf/2, nf/2) * sdf + 150e6
times = np.arange(-ntimes/2, ntimes/2) * dt
frs = np.fft.fftshift(np.fft.fftfreq(ntimes, d=dt))
dfr = np.mean(np.diff(frs))
dlys = np.fft.fftshift(np.fft.fftfreq(nf, d=sdf))
ddly = np.mean(np.diff(dlys))
# simulate some data in fringe-rate and delay space
np.random.seed(0)
dfr, ddly = frs[1] - frs[0], dlys[1] - dlys[0]
d = 200 * np.exp(-2j*np.pi*times[:, None]*(frs[ntimes//2+2]+dfr/4) - 2j*np.pi*freqs[None, :]*(dlys[nf//2+2]+ddly/4))
d += 50 * np.exp(-2j*np.pi*times[:, None]*(frs[ntimes//2-3]) - 2j*np.pi*freqs[None, :]*(dlys[nf//2+3]))
# simulate some flags
f = np.zeros_like(d, dtype=np.bool)
d[:, 20:22] += 1e3
f[:, 20:22] = True
d[20, :] += 1e3
f[20, :] = True
w = (~f).astype(np.float)
bl_len = dlys[nf//2+4]
fr_len = frs[ntimes//2+4]

I run this before the PR update.

mdl_clean_rect, res_clean_rect, info_clean_rect = dspec.fourier_filter(x=[times, freqs], data=d, wgts=w, filter_centers=[[0.],[0.]],
                                                         filter_half_widths=[[fr_len],[bl_len]],
                                                         mode='clean', filter_dims=[1, 0], **{'filt2d_mode':'plus','tol':1e-5})
np.savez('old_clean.npz', mdl_plus=mdl_clean_plus, mdl_rect=mdl_clean_rect, res_plus=res_clean_plus, res_rect=res_clean_rect)

I run this after the PR update

mdl_clean_plus, res_clean_plus, info_clean_plus = dspec.fourier_filter(x=[times, freqs], data=d, wgts=w, filter_centers=[[0.],[0.]],
                                                         filter_half_widths=[[fr_len],[bl_len]],
                                                         mode='clean', filter_dims=[1, 0], **{'filt2d_mode':'plus','tol':1e-5})


mdl_clean_rect, res_clean_rect, info_clean_rect = dspec.fourier_filter(x=[times, freqs], data=d, wgts=w, filter_centers=[[0.],[0.]],
                                                         filter_half_widths=[[fr_len],[bl_len]],
                                                         mode='clean', filter_dims=[1, 0], **{'filt2d_mode':'plus','tol':1e-5})
np.savez('new_clean.npz', mdl_plus=mdl_clean_plus, mdl_rect=mdl_clean_rect, res_plus=res_clean_plus, res_rect=res_clean_rect)

Then

new_products = np.load('new_clean.npz')
old_products = np.load('old_clean.npz')
for array in new_products.files:
    assert np.all(old_products[array] == new_products[array])

[coveralls]

Coverage increased (+0.02%) to 93.259% when pulling 3d3ef63 on 2d_filter_disjoint_rectangles into 154a3a8 on master.