Simple mapping of original read IDs (Headers) to final DADA2 ASVs for validation purposes

[CassandraHjo]

I am working on validating the DADA2 pipeline's accuracy using simulated microbial community data. With simulated data, the "ground truth" identity of every single read is encoded directly in its original FASTQ file header (the sequence ID). My primary requirement is to obtain a simple, clean mapping that links the original input read to the final Amplicon Sequence Variant (ASV) identity determined by DADA2. Specifically, I need a reliable way to get this relationship for every single read that entered the pipeline (after trimming and filtering). This mapping is essential for calculating accurate performance metrics, such as the Adjusted Rand Index (ARI), where the output clustering (the ASV) must be compared against the known true identity (the Header) on a read-by-read basis. Crucially, this validation focuses solely on the clustering accuracy (sequence grouping) and does not rely on downstream taxonomic assignment.

Is there an existing, documented method or an internal DADA2 utility function that can provide this direct mapping table after the denoising, merging, and chimera-removal steps are complete?

Thanks.

[benjjneb]

It is straightforward to generate a mapping between denoised ASVs and the input dereplicated reads (post-flltering). This comment explains a bit: #354 (comment)

In short, the output of derepFastq contains the $map element, which is a input-reads-long vector of integers with the value of those integers being the index of the dereplicated "unique sequence" that the read was dereplicated into. The output of the dada function has a similar $map element that is a input-unique-sequences long vector of integers with values corresponding to the index of the denoised ASV that unique (i.e. dereplicated) sequence was denoised to. These can be chained together to get a input-reds-long vector of indices of the ASV to which each read was denoised as such: dada_output$map[derep_output$map].

There isn't such an easy solution for merging, but this can be done with a bit of R scripting using the output of mergers, which is a data.frame that includes the $forward and $reverse columns that contain the indices of the forward ASV (in the corresponding dada output object) and reverse ASV corresponding to each merged sequence.

As for chimera removal, the ASV at that point is the index, so one can just look up which ASVs were removed by seeing which are present in the pre-chimera-removal table but absent in the post-chimera-removal table.

Edit: The help documentation of ?derepFastq, dada, and mergePairs all include some additional description of these elements of their return values objects.