DDA identification - phosphopeptides

[MatthewThe]

aim of the new module

1. what is its aim?

Benchmark phosphopeptide identification workflows and assess their identification and localization accuracy.

2. what is its added value compared to already available modules?

Identification rates for phosphopeptide datasets is substantially lower than for unmodified peptides (<25% compared to ~50%). One reason is the loss of the phospho group during fragmentation, which is partially addressed by multistage activation (MSA). Additionally, localization accuracy is important for annotation and downstream analysis.

full description of the new module

1. What file would be available for download on the module website?

Here are a few options:
- Ferries et al. Orbitrap Fusion: PXD007058 (only HCDOT acquisition: “SF_200217_pPeptideLibrary_pool<1 to 5>_HCDOT_rep <1 to 2>”) as used in the [benchmarking study from Marie](https://pubs.acs.org/doi/full/10.1021/acs.jproteome.9b00679).
  - Synthetic peptides allows identification/localization accuracy evaluation using ground truth
  - Does not seem to include multi-stage activation (MSA)
- ProteomeTools PTM: PXD009449 (only phospho runs)
  - Synthetic peptides allows identification/localization accuracy evaluation using ground truth
  - Does not include multi-stage activation (MSA)
- In-house ProteomeTools PTM with MSA: we measured a couple of ProteomeTools phospho pools with MSA at the Kusterlab. Would have to check how big this dataset is.
- Hogrebe et al.: PXD007145 (only LFQ runs)
  - No ground truth: 1. U2OS cells treated with doxorubicin (DOX) and DMSO as control, or 2. mixed species approach, diluted phosphopeptides enriched from yeast at fixed 1:4:10 ratios into a 1:1:1 background of HeLa phosphopeptides.
  - Includes multi-stage activation (MSA)
- I remember talking to Robbin at HUPO in Dresden about this benchmark and him mentioning that he may know a good dataset for this.

- A fasta file with expected (+entrapment?) sequences.

2. What would users need to do?

Search the raw files with a provided fasta file and provide the results on phosphopeptide level. Depending on which dataset we go for, the user would also specify phosphorylation on a non-phosphorylatable amino acid (e.g. alanine) to assess localization accuracy.

3. What would be the type/format of files used as input for the module?

We could either provide the RAW files, or probably better is to convert them to mzML before to prevent mistakes/variation in the conversion.

4. What metrics would be calculated and how?

- Number of identified phosphopeptides
- Number of false identifications (in the case of ground truth datasets)
- Number of false localizations

5. Propose a plot for metrics visualisation

- Number of identified phosphopeptides vs number of false identifications
- Number of identified phosphopeptides vs number of false localizations

potential reviewers

@mlocardpaulet
@RobbinBouwmeester

Will you be able to work on the implementation (coding) yourself, with additional help from the ProteoBench maintainers?

• yes
• no

any other information

Maybe it makes sense to split this up into multiple modules, separating the localization accuracy from the identification. Personally, I would very much like to do a TMT benchmark as well (as we do a lot of that here in the Kusterlab) but let's take it one step at a time :)

[mlocardpaulet]

Hey! Great to see a proposal for a phospho module!

Comment on your last point (in "any other information"): I worked a bit on this in the past, and we observed that differences at the identification level (peptide sequence) has indeed a huge impact on the final number of true positives and false positives. See here for more details. But how would you separate ID from localisation? Do you have any idea how to do that? You can always restrict the comparison on the same set of sequences, which is relatively easy to do when working with synthetic peptides.

In any cases, keep me posted, I'd be happy to help with this (we can then suggest somebody else for "reviewing" later.

[MatthewThe]

My thought was just that we might need different datasets for evaluating ID (e.g. proteome mixture or HeLa runs) and for evaluating localization (e.g. synthetic peptides). I'm not sure how it works best in Proteobench, e.g. if one module can have multiple datasets.

I indeed saw your publication already and thought that it would be a good starting point. Do you still happen to have the input and output files from the paper lying around somewhere?

[mlocardpaulet]

everything should be there: https://zenodo.org/records/3601193
I am not sure how clear/easy to understand the repo. is. Let me know if you want to understand it, or if you want me to send you the files that you need ;)

In terms of organisation of the modules: right now we tend to go for 1 data set per module and 1 main metric. So it can totally be split in 2 modules. But it could also be in one if it makes sense. Then, I guess that then the results of the different data sets should be combined somehow?

[david-bouyssie]

A PhD student from our lab is preparing an article to present a benchmarking dataset that will include both synthetic phosphopeptides (the same than Ferries et al, but newly injected) and mouse T cells enriched phosphoproteome.

Briefly, we have three datasets: one for ident&localization benchmarking (based on the synthetic phosphopeptides) and two datasets for quantification benchmarking (synthetic phosphopeptides + mouse phosphoproteome).
They were acquired on several instruments (Exploris 480, timsTOF SCP) and using different methods (DDA/DDA+FAIMS/DIA).
We also already benchmarked several software tools developed for DDA (Proline, PD & MaxQuant) and DIA (DIA-NN, Spectronaut).
Spoiler: DIA results are quite informative!

It will we take a bit of time to get the paper accepted but we plan to first submit to bioRxiv, so at this stage these datasets could be shared.

[MatthewThe]

That sounds like a great dataset for this purpose!
Might I even suggest that adding this dataset to Proteobench as part of the paper would be a great selling point for it?

How many synthetic phosphopeptides do you have? And do you have a rough estimation for when the paper will be on bioRxiv?

[mlocardpaulet]

@david-bouyssie is there any news on these data? When do you think that these will be accessible to the community?

[david-bouyssie]

We used the same library than in Ferries et al. So it contains around 180 phospho-peptides, some of them having different isomers, and some other ones present as single positional isomer.

No clear date for now, probably early summer time.