Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
451 lines (330 loc) · 14 KB

CONTRIBUTING.md

File metadata and controls

451 lines (330 loc) · 14 KB
Raw

Contributing to Celldega

Thank you for your interest in contributing to Celldega! 🧬

What You Need

🚀 Quick Start (one-time!)

# Clone Celldega repo
git clone https://github.com/broadinstitute/celldega.git
cd celldega

# Setup everything: Python venv, install dependencies, pre-commit hooks
bash ./scripts/setup.sh

That's it! Your development environment is ready. 🎉

🔄 Daily Development Workflow

1. Start Your Session

source dega/bin/activate    # Activate environment
npm run dev                 # Start development server

2. Make Your Changes

  • Write clear, descriptive code
  • Add tests for new features
  • Test in Jupyter: jupyter lab examples/

3. Before Committing

bash ./scripts/test.sh           # Run all tests
git commit -m "feat: your amazing feature"

That's the entire workflow! Our automation handles formatting, linting, and quality checks.

🛠️ Project Configuration & Tooling

Understanding the project's tooling helps you contribute more effectively. Here's what's configured where:

File/Directory Purpose Tools Configured
🔄 CI/CD & Automation
.github/workflows/ci.yml Continuous Integration pytest, Jest, ESLint, Prettier, Ruff, Safety, Bandit
.github/dependabot.yml Dependency updates Automated Python, npm, and GitHub Actions updates
.pre-commit-config.yaml Git hooks Ruff (Python), Prettier, ESLint, basic file checks
🐍 Python Configuration
pyproject.toml Python package config pytest, Ruff (linting + formatting), build system
🌐 JavaScript Configuration
package.json JavaScript dependencies Jest (testing), ESLint (linting), Prettier (formatting)
eslint.config.js JavaScript linting rules ESLint with import ordering, code quality rules
jest.setup.js Test environment setup Jest with jsdom, testing-library
💻 Development Scripts
scripts/setup.sh Environment setup Python venv, npm install, pre-commit hooks
scripts/test.sh Test runner pytest, Jest, linting checks
scripts/utils.sh Shared utilities Helper functions for scripts
👷🏻 Build & Bundling
build.js JavaScript bundling esbuild with WASM support

What Runs Where

On Every Commit (Pre-commit Hooks):

  • ✅ Ruff formats and lints Python code
  • ✅ Prettier formats JavaScript/JSON/YAML
  • ✅ ESLint checks JavaScript code quality
  • ✅ Basic file checks (trailing whitespace, merge conflicts)

On Every Push/PR (CI Pipeline):

  • 🧪 Testing: pytest (Python), Jest (JavaScript)
  • 🔍 Code Quality: Ruff, ESLint, Prettier checks
  • 🔒 Security: Safety (Python deps), Bandit (code analysis)
  • 📦 Build: Package building and installation tests
  • 🚀 Release: Automated PyPI publishing on tags

🧪 Testing

1. Run Tests

bash ./scripts/test.sh           # Run everything
bash ./scripts/test.sh python    # Python tests only
bash ./scripts/test.sh js        # JavaScript tests only
bash ./scripts/test.sh coverage  # Generate coverage report

2. Writing Tests

🐍 Python (pytest) - Add tests like this:

# tests/unit/test_my_feature.py
from celldega.utils import my_function

def test_my_function():
    result = my_function('input')
    assert result == 'expected'

🌐 JavaScript (Jest) - Add tests like this:

// js/__tests__/myFeature.test.js
import { myFunction } from '../utils/myFeature.js';

test('should handle basic case', () => {
  expect(myFunction('input')).toBe('expected');
});

📓 Rendering Docs Notebooks (Widget Embedding)

The example notebooks under docs/examples/ render their interactive widgets (Landscape, Clustergram, …) statically in the built docs by embedding the widget state into the notebook. mkdocs-jupyter renders that saved state — it does not execute notebooks at build time — so the state must be present in the committed .ipynb.

Render a notebook with embedded widget state using nbconvert:

# In place — re-executes the notebook and embeds widget state
CELLDEGA_ESM_VERSION=<released-version> jupyter nbconvert \
  --to notebook --execute --inplace \
  --ExecutePreprocessor.timeout=900 \
  --ExecutePreprocessor.iopub_timeout=120 \
  docs/examples/brief_notebooks/YourNotebook.ipynb

# Standalone HTML export (writes YourNotebook.html next to it)
CELLDEGA_ESM_VERSION=<released-version> jupyter nbconvert --to html --execute \
  --ExecutePreprocessor.timeout=900 \
  --ExecutePreprocessor.iopub_timeout=120 \
  docs/examples/brief_notebooks/YourNotebook.ipynb

Two flags/vars matter:

  • CELLDEGA_ESM_VERSION=<released-version> (e.g. 0.17.0) — makes each widget load celldega.js from jsdelivr via a tiny _esm shim instead of inlining the ~10 MB bundle once per widget. The pinned version must be published to npm. Without it (or with CELLDEGA_LOCAL_ESM=1) the local bundle is inlined: self-contained and offline-capable, but ~10 MB per widget. For local JS development (npm run dev/HMR), set CELLDEGA_LOCAL_ESM=1 so the widget uses your working celldega.js rather than the CDN.
  • --ExecutePreprocessor.iopub_timeout=120 — the Landscape emits large (10 MB+) messages; the default 4 s timeout drops them, leaving blank widgets. (store_widget_state is already True by default, so widget state is embedded automatically.)

Notes:

  • nbconvert runs the kernel in the notebook's own directory, so a notebook with a repo-root-relative data path will resolve it relative to that directory.

  • Sanity-check that the bundle wasn't inlined (shim ≈ 105 bytes, inlined ≈ 10 MB):

    python -c "import json,sys; s=json.load(open(sys.argv[1]))['metadata'].get('widgets',{}).get('application/vnd.jupyter.widget-state+json',{}).get('state',{}); print('max _esm bytes:', max((len(json.dumps(m['state']['_esm'])) for m in s.values() if '_esm' in m.get('state',{})), default=0))" \
  docs/examples/brief_notebooks/YourNotebook.ipynb

  • These notebooks are matched by the *.ipynb rule in .gitignore, so commit a new one explicitly with git add -f docs/examples/.../YourNotebook.ipynb.

🎨 Code Style

No need to worry about formatting! Our pre-commit hooks automatically:

  • ✅ Format Python code with Ruff
  • ✅ Format JavaScript code with Prettier
  • ✅ Fix linting issues
  • ✅ Check for common mistakes

Naming Conventions

  • Python: snake_case functions, PascalCase classes
  • JavaScript: camelCase functions, PascalCase classes
  • Files: my_module.py, myComponent.js
  • Type Annotations: Always add proper typing for better code clarity and IDE support
Python Typing Examples 
from typing import List, Optional, Dict, Any
import pandas as pd

def process_cell_data(
    cells: pd.DataFrame,
    cluster_labels: Optional[List[int]] = None,
    metadata: Dict[str, Any] = {}
) -> pd.DataFrame:
    """Process cell data with optional clustering information."""
    # Implementation here
    return processed_cells
JavaScript JSDoc Examples 
/**
 * Render cells on deck.gl layer with interactive selection
 * @param {Array<{x: number, y: number, cluster: number}>} cellData - Array of cell coordinates
 * @param {Object} options - Rendering options
 * @param {string} options.colorBy - Property to color cells by
 * @param {boolean} options.interactive - Enable cell selection
 * @returns {Promise<DeckGLLayer>} Configured deck.gl layer
 */
async function renderCellLayer(cellData, options = {}) {
  // Implementation here
  return layer;
}
TypeScript Examples 
interface CellData {
  x: number;
  y: number;
  cluster: number;
  [key: string]: any;
}

interface RenderOptions {
  colorBy?: string;
  interactive?: boolean;
  opacity?: number;
}

async function renderCellLayer(
  cellData: CellData[],
  options: RenderOptions = {}
): Promise<DeckGLLayer> {
  // Implementation here
  return layer;
}

📝 Commit Message Structure

Use these prefixes:

# STRUCTURE: "activity(module): ..."

git commit -m "feat(deck-gl): implement interactive cell selection widget" # New Feature
git commit -m "optim(deck-gl): optimize WebGL shader compilation for speed" # Optimization
git commit -m "fix(deck-gl): resolve memory leak in large dataset rendering" # Bug fix
git commit -m "hotfix(deck-gl): 🔴 resolve data loading issue"       # Hot bug fix (needs urgent rework -> create GitHub/Jira issue)
git commit -m "docs(deck-gl): add WebGL troubleshooting guide for users" # Documentation
git commit -m "test(deck-gl): add integration tests for widget lifecycle" # tests
git commit -m "infra(jest): configure coverage reporting for CI pipeline" # Infrastructure

🐛 Reporting Issues

Found a bug? Use our issue templates:

  • 🐛 Bug Report: Include your environment and steps to reproduce
  • 🎁 Feature Request: Describe what you'd like to see
  • 📚 Documentation: Suggest improvements to docs or examples

💡 Contribution Ideas

For Biology Researchers

  • 🧬 Add example datasets from your research
  • 📊 Create tutorial notebooks
  • 📝 Improve documentation with domain expertise
  • 🧪 Test with real-world data and report issues

For Developers

  • ⚡ Optimize performance for large datasets
  • 🎨 Improve visualization components
  • 🔧 Add new analysis algorithms
  • 🌐 Enhance web integration

For Everyone

  • 📖 Fix typos and improve clarity
  • 🧪 Add tests for edge cases
  • 🎯 Improve user experience
  • 🤝 Help others in discussions

🆘 Getting Help

Stuck? Here's how to get unstuck:

Quick Fixes

./scripts/setup.sh --status    # Check what's wrong
./scripts/setup.sh --reset     # Nuclear option - fresh start
./scripts/setup.sh --verbose   # See detailed output
Common Issues

"Permission denied"

chmod +x scripts/*.sh

"Python/Node.js/npm not found"

  • Install from links above
  • Check versions: python --version, node --version, npm --version

"npm: command not found"

# npm comes with Node.js - install Node.js from nodejs.org
# or update Node.js to get latest npm

"Import errors"

./scripts/setup.sh             # Reinstall dependencies

"Tests failing"

bash ./scripts/test.sh --verbose    # See detailed error messages

"node_modules missing"

npm install                    # Reinstall JavaScript dependencies

Still Stuck?

🏗️ Development Environment Details

Click for advanced setup options

Custom Environment Name

bash ./scripts/setup.sh --env-name my-celldega-env

Development Server Options

npm run dev                # Standard development
npm run dev:watch         # Watch mode with hot reload
npm run build             # Production build
npm run serve             # Serve production build

Testing Options

pytest tests/unit/         # Specific test directory
pytest -k "test_clustering" # Tests matching pattern
pytest --cov-report=html   # HTML coverage report
npm run test:js:watch     # JavaScript tests in watch mode

npm Commands

npm install               # Install all dependencies
npm ci                    # Install from lockfile (CI)
npm run test:js          # Run JavaScript tests
npm run lint:js          # Lint JavaScript code
npm run format:js        # Format JavaScript code
npm outdated             # Check for outdated packages
npm update               # Update dependencies

VS Code Setup

Install these extensions for the best experience:

  • Python - Core Python support
  • Ruff - Python linting and formatting
  • ESLint - JavaScript linting
  • Prettier - Code formatting
  • Jupyter - Notebook support

🏆 Recognition

Contributors are celebrated in:

  • 📋 CONTRIBUTORS.md file
  • 📈 GitHub contributors graph
  • 📰 Release notes for significant contributions
  • 🎉 Special mentions in community updates

🌟 Pro Tips

  1. Start small - Fix documentation or add a simple test first
  2. Ask early - Use Discussions to validate ideas before coding
  3. Test thoroughly - Include edge cases and error conditions
  4. Document changes - Help future contributors understand your work
  5. Be patient - We review all contributions carefully
  6. Use npm efficiently - It comes with Node.js and handles all dependencies well!

🤝 Code of Conduct

We're building something amazing together! Please be:

  • 🤝 Respectful - Everyone was new once
  • 🌍 Inclusive - Welcome diverse perspectives
  • 🏗️ Constructive - Focus on solutions
  • 🎓 Educational - Help others learn

Ready to contribute? Fork the repo and dive in! We can't wait to see what you build. 🚀

Need help? Don't hesitate to reach out - we're here to help you succeed!