Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add template #1

Merged
merged 38 commits into from
Mar 12, 2025
Merged
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
e148639
Move to pixi
irm-codebase Mar 4, 2025
467e7a9
Separete pyproject into individual configuration files
irm-codebase Mar 4, 2025
ca4796b
Add pixi tests
irm-codebase Mar 4, 2025
34cb537
Move integration tests within the template
irm-codebase Mar 4, 2025
2d46514
Merge branch 'main' into add-template
irm-codebase Mar 4, 2025
b11be15
update README
irm-codebase Mar 4, 2025
18166bd
Add scheduled workflow for copier template updates.
irm-codebase Mar 5, 2025
353376e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 5, 2025
5aec108
Add PR CI and avoid snakefmt template false positive
irm-codebase Mar 5, 2025
2828bee
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 5, 2025
1fb55eb
Fix CI
irm-codebase Mar 5, 2025
cf6592d
use miniconda incubator in CI
irm-codebase Mar 5, 2025
cb225a9
Add local mamba dependency
irm-codebase Mar 5, 2025
d83d963
switch mamba-> conda
irm-codebase Mar 5, 2025
7146a52
Try pixi local and mini-forge in global
irm-codebase Mar 5, 2025
7524b37
Use pixi with conda dependency
irm-codebase Mar 5, 2025
827c665
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 5, 2025
eecaa11
Avoid windows bug: shell commands must use \"
irm-codebase Mar 5, 2025
e1898c1
Specify locations to skip for the template
irm-codebase Mar 5, 2025
a1a5d99
Add pull request CI test to the template.
irm-codebase Mar 5, 2025
af272a1
update readme and changelog
irm-codebase Mar 5, 2025
573246b
PR fixes: add examples to docs, avoid name confusion
irm-codebase Mar 6, 2025
30e1198
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 6, 2025
d1d896d
Improve the explanation in the README
irm-codebase Mar 6, 2025
86fc03d
Use conda-forge to install clio-tools
irm-codebase Mar 6, 2025
65d0e4b
PR fixes: no pixi shell use, remove unused mkdocs plugin, fix github …
irm-codebase Mar 7, 2025
81f1b95
Fix integration test configuration path
irm-codebase Mar 7, 2025
cf5fb7f
Fix relative path bug
irm-codebase Mar 7, 2025
ff5c1b6
Minor docs updates
brynpickering Mar 7, 2025
bb24ded
Update highlight to tip
brynpickering Mar 7, 2025
977c0ba
Remove attempt at highlighting
brynpickering Mar 7, 2025
13cbe9d
Update tests/utils/simple_answers.yaml
sjpfenninger Mar 11, 2025
1098a91
Update tests/utils/simple_answers.yaml
sjpfenninger Mar 11, 2025
d11f453
Update copier.yaml
irm-codebase Mar 11, 2025
71ff30c
Update README.md
irm-codebase Mar 11, 2025
61d9590
PR fixes: improve instructions, improve integration test.
irm-codebase Mar 11, 2025
bd0428d
Remove mermaid
irm-codebase Mar 11, 2025
e8b7571
Simplify: read the docs is optional, improve README
irm-codebase Mar 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Move to pixi
irm-codebase committed Mar 4, 2025
commit e148639944117b6da798b6f16384a46448e91575
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SCM syntax highlighting & preventing 3-way merges
pixi.lock merge=binary linguist-language=YAML linguist-generated=true
11 changes: 8 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -18,9 +18,14 @@ gurobi.log
### Notebooks
.ipynb_checkpoints

# Mutable folders
# Mutable / cache folders
tmp/
temp/

# Mkdocs
.cache/
.pytest_cache/
.mypy_cache/
.ruff_cache/

# pixi environments
.pixi
*.egg-info
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@ repos:

# Python file formatting
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.9.6
rev: v0.9.9
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
8 changes: 1 addition & 7 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,15 +1,9 @@
# Changelog

## 0.0.2
## 0.1.0

### Added

- Module copier template with testing, automated docs.
- Added ruff and mypy checks
- Added testing for the generated template

## 0.0.1 - 2025-02-17

### Added

- Created the repo
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ cff-version: 1.2.0
message: If you use this software or data produced by it, please cite it using the metadata from this file.
title: clio - data module template
repository: https://github.com/calliope-project/data-module-template
license: MIT
license: Apache-2.0
authors:
- given-names: Ivan
family-names: Ruiz Manuel
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -4,15 +4,16 @@ A template for modular data workflows using [snakemake](https://snakemake.readth

## How to use this template

1. Install [copier](https://copier.readthedocs.io/en/stable/) with your favourite package management system (we'll assume [mamba](https://mamba.readthedocs.io/en/latest/) in this example).
1. Install [`copier`](https://copier.readthedocs.io/en/stable/) with your favourite package management system (we'll assume [mamba](https://mamba.readthedocs.io/en/latest/) in this example).
2. Use `copier` to copy this repo.
3. Answer some questions so we pre-fill licensing, citiation files, etc...
4. You're all set up!

```bash
mamba install copier
copier copy https://github.com/calliope-project/data-module-template.git ./path/to/destination
```
```bash
mamba install copier
copier copy https://github.com/calliope-project/data-module-template.git ./path/to/destination
```

3. Answer some questions so can we pre-fill licensing, citiation files, etc...
4. You're all set up!

## Features

12 changes: 6 additions & 6 deletions copier.yaml
Original file line number Diff line number Diff line change
@@ -12,8 +12,8 @@ module_short_name:
type: str
help: Please enter your module's short name as used in snakemake (should be in `snake_case`).
validator: >-
{% if not (module_short_name | regex_search('^[a-z][a-z0-9_]+$')) %}
"Only letters, digits and underscores are valid."
{% if not (module_short_name | regex_search('^[a-z][a-z0-9_]*$')) %}
"Only lowercase letters, digits and underscores are valid."
{% endif %}
module_long_name:
type: str
@@ -23,10 +23,10 @@ module_description:
help: "Please provide a description of your module."
author_given_name:
type: str
help: "Please provide your given name(s)."
help: "Please provide your given name(s) (e.g., Ursula, Juan, Roald)."
author_family_name:
type: str
help: "Please provide your family name(s)."
help: "Please provide your family name(s) (e.g., Le Guin, Rulfo Vizcaino, Dahl)."
author_email:
type: str
help: Please provide your email address.
@@ -40,8 +40,8 @@ github_org:
Please provide the name of the github account or organisation where
this module will be created (e.g., 'calliope-project' in 'github.com/calliope-project').
validator: >-
{% if not (github_org | regex_search('^[a-z][a-z0-9_\-]+$')) %}
"Only letters, digits, hyphens and underscores are valid."
{% if not (github_org | regex_search('^[a-z][a-z0-9_-]*$')) %}
"Only lowercase letters, digits, hyphens and underscores are valid."
{% endif %}
license:
type: str
20 changes: 0 additions & 20 deletions environment.yaml

This file was deleted.

7,667 changes: 7,667 additions & 0 deletions pixi.lock

Large diffs are not rendered by default.

38 changes: 33 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,21 +1,49 @@
### ---
# Project setup
### ---
[project]
name = "data-module-template"
authors = [
{ name = "Contributors listed in AUTHORS" },
]
name = "data-module-template"
requires-python = ">= 3.12"
version = "0.1.0"
readme = "README.md"
requires-python = ">=3.12"
keywords = ["energy systems", "modelling", "workflows"]
license = { text = "MIT" }
dynamic = ["version"]

dependencies = ["mkdocs-mermaid2-plugin>=1.2.1,<2", "mkdocs-git-committers-plugin-2>=2.5.0,<3", "clio-tools @ git+https://github.com/calliope-project/clio.git"]

[project.urls]
website = "https://clio.readthedocs.io/"
repository = "https://github.com/calliope-project/data-module-template"
documentation = "https://clio.readthedocs.io"
changelog = "https://github.com/calliope-project/data-module-template/CHANGELOG"


[tool.pixi.project]
channels = ["conda-forge", "bioconda"]
platforms = ["win-64", "linux-64", "osx-64", "osx-arm64"]

[tool.pixi.pypi-dependencies]
data_module_template = { path = ".", editable = true }

[tool.pixi.tasks]

[tool.pixi.dependencies]
cffconvert = ">=2.0.0,<3"
copier = ">=9.5.0,<10"
graphviz = ">=12.2.1,<13"
mkdocs-material = ">=9.6.7,<10"
mypy = ">=1.15.0,<2"
pytest = ">=8.3.5,<9"
ruff = ">=0.9.9,<0.10"
snakefmt = ">=0.10.2,<0.11"
snakemake-minimal = ">=8.29.0,<9"
networkx = ">=3.4.2,<4" # TODO: remove after clio-tools is released


### ---
# Project tooling
### ---
[tool.mypy]
disable_error_code = ["import-untyped"]

4 changes: 3 additions & 1 deletion tests/template_test.py
Original file line number Diff line number Diff line change
@@ -55,5 +55,7 @@ def test_snakemake_all_failure(simple_template):
def test_snakemake_integration_testing(simple_template):
"""The automatic integration test should run by default."""
assert subprocess.run(
["snakemake", "--use-conda"], check=True, cwd=simple_template / "tests/integration"
["snakemake", "--use-conda"],
check=True,
cwd=simple_template / "tests/integration",
)