-
Notifications
You must be signed in to change notification settings - Fork 222
Fix silence period and job_kwargs #4071
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
for more information, see https://pre-commit.ci
mode="zeros", | ||
noise_levels=None, | ||
seed=None, | ||
job_kwargs=dict(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
job_kwargs=dict(), | |
**job_kwargs, |
Should it be like this?
And then load the _shared_job_kwargs
doc?
I'll test this PR now!
So it appropriately used the global_job kwargs now during Exposing job_kwargs worked fine for Copy of script I ran and terminal output. > import spikeinterface.full as si
> from spikeinterface.sortingcomponents.peak_detection import detect_peaks
> rec, _ = si.generate_ground_truth_recording([600], 30000, 64, 30)
> rec = si.common_reference(si.bandpass_filter(rec))
> rec_silenced = si.silence_periods(rec, [[30000,30040],[50000, 50400]], mode='noise', job_kwargs=dict(n_jobs=7))
noise_level (workers: 7 processes): 100%|██████████| 20/20 [00:00<00:00, 50.22it/s]
> si.set_global_job_kwargs(**dict(n_jobs=7))
> peaks = detect_peaks(rec_silenced)
noise_level (workers: 7 processes): 100%|██████████| 20/20 [00:00<00:00, 46.42it/s]
detect peaks using locally_exclusive (workers: 7 processes): 100%|██████████| 600/600 [00:13<00:00, 43.03it/s] I will say this went much faster than when I was testing and each process would go back and recalculate noise_levels with n_jobs=1. |
…nto fix_silence_period
for more information, see https://pre-commit.ci
@samuelgarcia @zm711 This should also now fix the problem with detect_peaks. |
for more information, see https://pre-commit.ci
…nto fix_silence_period
At least on my computer it still said it was computing import spikeinterface.full as si
rec, _ = si.generate_ground_truth_recording([600], 30000, 64, 30)
rec = si.common_reference(si.bandpass_filter(rec))
rec1 = si.silence_periods(rec, [[30000,30040],[50000, 50400]], mode='noise', job_kwargs=dict(n_jobs=7))
from spikeinterface.sortingcomponents.peak_detection import detect_peaks
si.set_global_job_kwargs(**dict(n_jobs=7))
peaks= detect_peaks(rec1)
noise_level (workers: 7 processes): 100%|██████████| 20/20 [00:00<00:00, 46.48it/s]
noise_level (workers: 7 processes): 100%|██████████| 20/20 [00:00<00:00, 46.86it/s]
detect peaks using locally_exclusive (workers: 7 processes): 100%|██████████| 600/600 [00:36<00:00, 16.55it/s] |
Is this because it is assuming |
Putative attempt to fix #4038, by explicitly exposing job_kwargs to the silence_period function, in order to allow the user to set them explicitly. In addition, noise_levels are now saved explicitly in extra_kwargs. In theory, once a first call to noise_levels has been made, they should be cached and thus no recomputation should be made.