Data platform processing

This platform centralizes and manages data collected from citizen expeditions organized by Astrolabe Expeditions. This repository contains the ETL (Extract, Transform, Load) pipeline components. Other parts of the project include a back-office and API and a map application for visualizing collected data.

Warning

This project is archived and no longer maintained. Feel free to browse the code, but note that no further updates or support will be provided.

What is Astrolabe Expeditions?

Astrolabe Expeditions is an non-profit association that develops participatory science programmes with laboratories to enable citizens to actively contribute to scientific research.

Citizens' expeditions are set up to collect large-scale scientific data and involve citizens in understanding and preserving the ocean.

Local development

Clone the repository

git clone [email protected]:astrolabe-expeditions/data-platform-processing.git

Switch to the repo folder

cd data-platform-processing

Create new virtual env with Virtualenv

virtualenv venv

Activate your venv

source venv/bin/activate

Install dependencies

pip install -r requirements.txt

To test with the web server

python src/server.py

You can now access the server at http://localhost:8080

Local development with docker

You'll need to have Docker installed. It's available on Windows, macOS and most distros of Linux.

Clone the repository

git clone [email protected]:astrolabe-expeditions/data-platform-processing.git

Switch to the repo folder

cd data-platform-processing

Copy the example env file and make the required configuration changes in the .env file

cp .env.example .env

Build everything

docker compose up --build

You can now access the server at http://localhost:8080

How to contribute?

We are eager for contributions and very happy when we receive them! It can be code, of course, but it can also take other forms. The workflow is explained in the contributing guide.