Skip to content

Workers / Slaves numbers for IPO on clusters #63

@ThomasPouplin

Description

@ThomasPouplin

Sorry to ask this question but I'm a bit lost in parallel computing ...

Could somebody explain the ratio between workers and slaves for running IPO optimally ?

I have access to clusters (>200 CPUs) and would like to optimise the run time on my lipidomics datasets (POS and NEG).

Let's say I use 6 QCs samples to perform the optimization.

I understood the following formula:
Cores = IPO-slaves * XCMS slaves

Where I'm really lost, is where are these workers/slaves described between the Snowparam (n workers) and the optimizeXcmsSet (n slaves).

How can I, for example, set one slave per sample (so 6 slaves), and split workers for each slaves (say 4 workers per slave) ?

At the moment, I just set 8 workers and 8 salves - IPO runs without any errors but maybe not optimally.

This is a part of the code I'm using (on 8 CPUs):

snowparam <- SnowParam(workers = 8, type = "SOCK")
peakpickingParameters = getDefaultXcmsSetStartingParams('centWave') # get defaults

resultPeakpicking = optimizeXcmsSet(
params = peakpickingParameters,
isotopeIdentification = "IPO",
nSlaves = 8,
subdir = 'CentWave'
)

Thanks for your feedback !

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions