Skip to content

README.md

Latest commit

 

History

History
92 lines (66 loc) · 4.41 KB

README.md

File metadata and controls

92 lines (66 loc) · 4.41 KB
Raw

GTEx Gene Expression Analysis Pipeline

A comprehensive suite of tools for extracting, clustering, and analyzing gene expression patterns using GTEx data. This pipeline takes a list of genes and produces biological insights through tissue-specificity clustering, statistical validation, and functional enrichment analysis.

🚀 Quick Start

The easiest way to run the pipeline is via the interactive wizard, which handles file paths, creates directories, and manages the workflow for you.

./run_pipeline_wizard.sh

Alternatively, you can run individual steps manually as described in the Pipeline Steps section.

📋 Prerequisites

Python Environment

  • Python 3.7+
  • Install required packages using the provided requirements file: 
    pip install -r requirements.txt

Data Sources

  1. Gene List: You must provide your own list of genes of interest (one gene symbol per line in a .txt file).
  2. GTEx Data: Download the following from the GTEx Portal (Adult Data). Select "Bulk Tissue Expression" and "Metadata" data types to obtain:
    • Gene TPMs: GTEx_Analysis_2017-06-05_v8_RNASeQCv1.1.9_gene_tpm.gct.gz (or v10 equivalent).
    • Sample Attributes: GTEx_Analysis_v8_Annotations_SampleAttributesDS.txt (or v10 equivalent).

🛠️ Pipeline Steps

1. Data Extraction (extract_all_chunked.py)

Extracts RNA-seq expression data (TPM) for your genes from the massive GTEx dataset.

  • Optimization: Extracted matrices are saved to analysis_runs/ for reuse across different analysis projects.

2. Clustering (gtex_cluster_v2.py)

Groups genes based on their tissue expression profiles using Ward's hierarchical clustering.

  • Calculates Silhouette Scores for every gene to quantify cluster membership quality.

3. Visualization (visualise_silhouette_scores.py)

Generates plots to assess cluster cohesion and identify "core" vs. ambiguous cluster members.

4. Enrichment Analysis

  • STRING (string_cluster_enrichment_report.py): Queries protein interaction networks and functional pathways.
  • Enrichr (enrichr_cluster_enrichment_report.py): Checks for over-representation in GO (2025), KEGG (2025), Human Phenotype Ontology, and more. Logs all four statistical scores (p-value, adjusted p-value, Z-score, and Combined Score).

5. Reporting (generate_biological_descriptions.py)

Synthesizes all results into:

  • Markdown Report: CLUSTER_BIOLOGICAL_DESCRIPTIONS.md
  • HTML Dashboard: CLUSTER_BIOLOGICAL_REPORT.html (includes hyperlinks to all PNG/CSV/TXT artifacts).

🔧 Support & Diagnostic Scripts

Beyond the core pipeline, the following scripts assist in parameter selection and quality control:

Cluster Number Suggestion (suggest_k.py)

Helps determine the optimal number of clusters ($k$) by testing a range of values and calculating Silhouette and SSE (elbow) metrics.

python suggest_k.py --expression-file your_data.csv --kmin 2 --kmax 15 --z-score

Cluster Noise Visualization (visualise_cluster_noise.py)

Generates line plots of gene expression profiles across tissues to qualitatively assess how "noisy" or "tight" a cluster is.

# Plot all clusters in a results directory
./visualise_cluster_noise.py --expression-file your_data.csv --data-dir results_k7 --clusters all

📂 Output Files (in --data-dir)

File Type Filename Pattern Description
Clusters gene_clusters_k{k}_*.csv Table assigning each gene to a cluster.
Scores gene_silhouette_scores_k{k}_*.csv Silhouette score for every gene.
Heatmaps global_heatmap_k{k}_*.png Visualizations of expression patterns.
Quality Plots silhouette_analysis_k{k}_*.png Cluster cohesion visualization.
Reports CLUSTER_BIOLOGICAL_REPORT.html Interactive HTML results dashboard.

💡 Best Practices

  1. Use Z-scores: Always use the --z-score flag to identify relative tissue specificity rather than just expression magnitude.
  2. Run suggest_k.py: Before committing to a final analysis, use the suggestion tool to find the most natural clustering resolution for your gene set.
  3. Inspect the HTML Report: Use the hyperlinks in the HTML report to quickly toggle between heatmaps and gene lists for deeper investigation.