> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cosmosid.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Impact of Pre-Processing on Analyses 

The same Analysis Module with the same set of parameters can produce widely different results depending on the pre-processing settings. Pre-processing parameters can affect diversity values, distance matrices, clustering patterns, feature importance, and statistical comparisons.

## What to look for when setting pre-processing parameters

Several properties of the Data Table determine how strongly pre-processing affects your results. How much these properties matter depends mainly on the type of sequencing data (long-read or short-read 16S, or WGS), the profile type (taxonomic or functional), and the complexity and variability of the sampled environment (e.g., soil samples tend to be more complex, whereas vaginal samples tend to be less complex):

* **Sparsity:** a feature is sparse across the Data Table when it is not detected in most samples of the cohort. Sparse features are typically taxa or genes with extremely low prevalence in the Data Table.
* **Sequencing depth:** how many reads each sample has, and how much this differs between samples in the cohort. Uneven sequencing depth introduces technical bias into comparisons and can be mistaken for a real biological difference in the variable of interest.

### Sparsity and Sequencing depth variability depending on the Cosmos-Hub Profiling Workflow

|                                                                                                                                                                                                                                                                                                                                                                                                                | Suggested Taxonomic Rank (if applicable) | Notes on sparsity depending on profile type                                                                                                                                                                                                                               | Seq. depth variability per sequencing technology                                      |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| <ul><li>[**16S SR Amplicon-sequencing**](https://docs.cosmosid.com/docs/16s-metagenomics-taxonomy)</li><li>[**ITS Amplicon Classification**](https://docs.cosmosid.com/docs/its-amplicon)</li></ul>                                                                                                                                                                                                            | `Genus`                                  | <ul><li>High sparsity</li><li>Rare and sparse features (OTU, ASVs) might be spurious (PCR, copy-number bias)</li></ul>                                                                                                                                                    | <ul><li>Often large, driven by PCR amplification bias and primer efficiency</li></ul> |
| <ul><li>[Long-read Amplicon Profiling](https://docs.cosmosid.com/docs/emu)</li></ul>                                                                                                                                                                                                                                                                                                                           | `Species`                                | <ul><li>High sparsity but rare feature calls have higher confidence</li><li>Features in the Data Table depend on the EMU database used for classification</li></ul>                                                                                                       | <ul><li>Moderate</li></ul>                                                            |
| <ul><li>[CHAMP Human Microbiome Taxonomic and Functional Profiling](https://docs.cosmosid.com/docs/champ2)</li><li>[KEPLER Host-Agnostic Taxonomic Profiling](https://docs.cosmosid.com/docs/kepler-microbiome-profiler)</li><li>[Host-Agnostic Functional Profiling](https://docs.cosmosid.com/docs/functional-analysis)</li><li>[AMR/VF Profiling](https://docs.cosmosid.com/docs/amr-vf-pipeline)</li></ul> | `Species`                                | <ul><li>High sparsity for CHAMP and KEPLER taxonomic profiles</li><li>Lower sparsity for functional profiles generated from module-based databases (GMM, GBM, MetaCyc)</li><li>Higher sparsity for gene-based profiles (CAZymes, GO, Enzyme Commission, AMR/VF)</li></ul> | <ul><li>Moderate, driven by host-DNA content and input biomass</li></ul>              |

### Pre-processing impact on Analysis Module outcomes

| Analysis                           | Sensitivity    | Pre-processing suggestions                                                                                                                                                                                  | Data-type note                                                                                                                                                                                     |
| ---------------------------------- | -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Alpha diversity**                | High           | <ul><li>Control for uneven depth when comparing Observed Richness.</li><li>Rarefaction is not needed when comparing Alpha-diversity evenness indexes (Shannon, Simpson) between different groups.</li></ul> | <ul><li>Functional profiles are often more stable than taxonomic ones because of functional redudancy across taxa.</li><li>ASV-level 16S richness can be inflated by spurious rare taxa.</li></ul> |
| **Beta diversity / PCA Explorer**  | Medium to High | <ul><li>Rarefaction is suggested when using Beta diversity metrics that are computed on presence-absence data (e.g: Jaccard)</li></ul>                                                                      | <ul><li>Functional profiles often separate more subtly than taxonomic ones in the ordination plots, reflecting functional redundancy across taxa.</li></ul>                                        |
| **Heatmap / Stacked Bar Charts**   | Low            | <ul><li>Pre-processing not needed.</li></ul>                                                                                                                                                                |                                                                                                                                                                                                    |
| **ML Classification & regression** | Medium to High | <ul><li>Filtering for Prevalence can reduce the risk of ML model learning from noise and sparse features.</li></ul>                                                                                         | <ul><li>High-dimensional WGS functional data usually benefits most from filtering and aggregation.</li></ul>                                                                                       |
| **Univariate comparison**          | High           | <ul><li>Filtering for Prevalence reduces the multiple-testing burden arising from sparse features and can improve power.</li></ul>                                                                          | <ul><li>Sparse taxonomic features are usually more affected by pre-processing than broader, module-based functional profiles</li></ul>                                                             |

## References

* Zhou R et al. Data pre-processing for analyzing microbiome data. Brief Bioinform / NIH review. 2023;24(6)

* Busato S et al. Compositionality, sparsity, spurious heterogeneity, and other pitfalls in microbiome machine learning. Patterns. 2022;3(12)