Install DeepSomatic r1.9 locally on your server without docker

Requirements

• ubuntu 22.04
• python 3.10.12
• CPUs support SSE4 & AVX
• sudo privilege

Install

1. Download prebuilt DeepSomatic binaries

2. Download DeepSomatic models

3. Prepare environment for /usr/bin/python3 & DeepSomatic using apt

4. Prepare environment for DeepSomatic using run-prereq.sh

   4.1 Install DeepSomatic requirements in /usr/bin/python3

   4.2 or Install DeepSomatic requirements using conda/micromamba virtual environment, e.g., tf virtual environment

5. Install DeepSomatic to /opt

6. Tensorflow CUDA issues

7. Backup the installed deepsomatic binaries & models

1. Download prebuilt DeepSomatic binaries

gsutil -m rsync -r "gs://deepvariant/binaries/DeepVariant/1.9.0/DeepVariant-1.9.0" .


# local directory
/t9k/mnt/WorkSpace/data/ngs/xuzhenyu/dv/DeepVariant-1.9.0

2. Download DeepSomatic models

# deepsomatic models
DEST="./models/deepsomatic/1.9.0/"
mkdir -p "${DEST}"

gsutil -m rsync -r \
  "gs://deepvariant/models/DeepSomatic/1.9.0/" \
  "${DEST}"


# local directory
/t9k/mnt/WorkSpace/data/ngs/xuzhenyu/dv/models

3. Prepare environment for /usr/bin/python3 & DeepSomatic using apt

sudo apt update
sudo apt install apt-utils build-essential python3-dev python3-pip python3-pip-whl \
  libcairo2-dev libgirepository1.0-dev pkg-config libdbus-1-dev parallel

4. Prepare environment for DeepSomatic using run-prereq.sh

bash run-prereq.sh


# local directory
bash dv_tf/run-prereq.sh

4.1 Install DeepSomatic requirements in /usr/bin/python3 or specified python 3.10

/usr/bin/python3 -m pip install -r requirements.txt --no-deps -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple

4.2 Install DeepSomatic requirements using conda/micromamba virtual environment

# or use whatever prepared python 3.10 you like
micromamba create -n tf -c conda-forge -c nvidia tensorflow=2.13.1=cuda118py310h189a05f_1 python=3.10.12 cudatoolkit cudnn
micromamba run -n tf pip install -r requirements.txt --no-deps -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
# if pip installed to ${HOME}/.local, add prefix
PIP_USER=false micromamba run -n tf pip install -r requirements.txt --no-deps -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple


# or use uv to install
micromamba create -n tf -c conda-forge -c nvidia tensorflow=2.13.1=cuda118py310h189a05f_1 python=3.10.12 cudatoolkit cudnn uv pygobject=3.42.1 pkg-config pkgconfig
PIP_USER=false micromamba run -n tf uv pip install -r requirements.txt --no-deps -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple


# if there are issues installing tensorflow, check the ${HOME}/.condarc
# and make sure there are not any customized channels
cat ~/.condarc

5. Install DeepSomatic to /opt

Use sudo if needed

mkdir -p /opt/deepvariant/bin/deepsomatic/

# copy https://github.com/google/deepvariant/raw/refs/heads/r1.9/scripts/run_deepsomatic.py to /opt/deepvariant/bin/deepsomatic
cp /path/to/run_deepsomatic.py /opt/deepvariant/bin/deepsomatic/

# copy prebuilt binaries to /opt/deepvariant/bin
rsync -avP /path/to/deepvariant/binaries /opt/deepvariant/bin/
# if use customized python3 other than /usr/bin/python3, patch the binaries
# modify the 
# `BIN_DIR` and 
# `PYTHON` when needed
cd patch && bash patch_dv_stub.sh

# copy deepsomatic models to /opt/models/deepsomatic
mkdir -p /opt/models/deepsomatic
# modify the `SRC_BASE` when needed
# remember to modify `rsync -avPn` to `rsync -avP`
bash rsync_install_deepsomatic_models.sh

# generate command cli
# remember to modify `PYTHON` variables
bash make_cli.sh

Local shared directory

mkdir -p /opt/deepvariant/bin/deepsomatic/

# copy locally shared run_deepsomatic.py
cp /t9k/mnt/WorkSpace/data/ngs/xuzhenyu/dv/deepvariant/scripts/run_deepsomatic.py /opt/deepvariant/bin/deepsomatic/

# copy prebuilt binaries to /opt/deepvariant/bin
rsync -avP /t9k/mnt/WorkSpace/data/ngs/xuzhenyu/dv/DeepVariant-1.9.0/ /opt/deepvariant/bin/
# if use customized python3 other than /usr/bin/python3, patch the binaries
# modify the 
# `BIN_DIR` and 
# `PYTHON` when needed
bash patch/patch_dv_stub.sh

# copy deepsomatic models to /opt/models/deepsomatic
# remember to modify `SRC_BASE` path to /t9k/mnt/WorkSpace/data/ngs/xuzhenyu/dv/models/deepsomatic/1.9.0/savedmodels
# remember to modify `rsync -avPn` to `rsync -avP`
mkdir -p /opt/models/deepsomatic
bash rsync_install_deepsomatic_models.sh

# generate command cli
# remember to modify `PYTHON` variables
bash make_cli.sh

6. Tensorflow CUDA issues

WIP

7. Backup the installed deepsomatic binaries & models

• /opt/deepvariant
• /opt/models
• /usr/bin/python3 environment
• or conda/micromamba python3 virtual environment

INSTALL by rsync preconfigured binaries

sudo rsync -avP /t9k/mnt/joey/bio_utilities/deepvariant_opt/ /opt/

Test

https://github.com/google/deepsomatic/blob/r1.9/docs/deepsomatic-case-study-wes.md