Open Source Ecosystem

The Mcity Data Engine leverages the vast machine learning and computer vision open source ecosystem that has evolved over the years. Key building blocks are Voxel51, 🤗 Transformers, Tensorboard and Weights and Biases, Anomalib, CVAT, PyTorch, pytest and pdoc.

To contribute to the Mcity Data Engine, knowledge about these tools might be necessary.

It is recommended to update the core dependencies regularly:

pip install --upgrade pip
pip install --upgrade fiftyone transformers datasets huggingface-hub ultralytics wandb
pip freeze > requirements.txt
git add requirements.txt
git commit -m "Updated requirements"
git push

Every update of the requirements.txt triggers an additional test run for Lighthouse builds. In addition to the regular test run, this allows to check if the project is still compatible with the Lighthouse environment.

Voxel 51

Voxel51 is the dataset-backbone of the Mcity Data Engine. All data processing is based on the data representation by Voxel51, which allows us to perform the same set of processes to arbitrary datasets.

In case there are issues with MongoDB, the underlying database Voxel51 uses, run ps aux | grep mongod and kill the fiftyone process.

🤗 Transformers

Hugging Face Transformers is used to perform training and inference runs with readily available models.

Tensorboard & Weights and Biases

All logging is done with Tensorboard and wrapped by Weights and Biases, so that all runs can be easily viewed in the browser. In addition, WandB Launch is integrated to perform queued training runs in isolated Docker containers.

Anomalib

Anomalib is utilized to perform anomaly detection. The concept is to treat a known but desired class as an anomaly. Existing datasets with labels can be separated into a training set that does not include the class (e.g., pedestrians) and an eval set that does. This way, in an unlabeled stream of data, instances of this class might be detect as anomalies.

CVAT

CVAT is used to label datasets.

PyTorch

PyTorch is our ML framework of choice.

pytest

Automated testing is performed within GitHub Actions.

pdoc

Automated documentation is performed within GitHub Actions.