Merge pull request #61 from mggg/1.2.0

1.2.0 Release Update

.gitignore

@@ -8,3 +8,16 @@ __pycache__/
 tests/test-reports/*
 */**/*.svg
 tests/*.ipynb
+
+# Python virtual environment files.
+.venv
+.docs_venv
+
+# Ignore the build directory and the dist directory
+# created by the build process.
+/build
+dist
+docs/build
+docs/build/**
+dev
+tutorials/test_*

.pre-commit-config.yaml

@@ -0,0 +1,6 @@
+repos:
+  - repo: https://github.com/psf/black
+    rev: 24.2.0
+    hooks:
+      - id: black
+        files: gerrytools/

.readthedocs.yaml

@@ -0,0 +1,22 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the OS, Python version and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.9"
+
+# Build documentation in the "docs/" directory with Sphinx
+sphinx:
+  builder: dirhtml
+  configuration: docs/source/conf.py
+
+
+python:
+   install:
+   - requirements: docs/docs_requirements.txt

README.md

@@ -1,6 +1,13 @@
 
 # gerrytools
-[![CircleCI](https://dl.circleci.com/status-badge/img/gh/mggg/gerrytools/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/mggg/gerrytools/tree/main) [![codecov](https://codecov.io/gh/mggg/gerrytools/branch/main/graph/badge.svg?token=O09GYF7C9X)](https://codecov.io/gh/mggg/gerrytools) [![PyPI version](https://badge.fury.io/py/gerrytools.svg)](https://badge.fury.io/py/gerrytools) [![docs](https://img.shields.io/badge/%E2%93%98-Documentation-%230099cd)](https://mggg.github.io/gerrytools/) [![website](https://img.shields.io/badge/%F0%9F%8C%90%20-MGGG%20Redistricting%20Lab-%230099cd)](https://mggg.org) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
+
+[![CircleCI](https://dl.circleci.com/status-badge/img/gh/mggg/gerrytools/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/mggg/gerrytools/tree/main) 
+[![codecov](https://codecov.io/gh/mggg/gerrytools/branch/main/graph/badge.svg?token=O09GYF7C9X)](https://codecov.io/gh/mggg/gerrytools) 
+[![PyPI version](https://badge.fury.io/py/gerrytools.svg)](https://badge.fury.io/py/gerrytools) 
+[![docs](https://img.shields.io/badge/%E2%93%98-Documentation-%230099cd)](https://mggg.github.io/gerrytools/) 
+[![website](https://img.shields.io/badge/%F0%9F%8C%90%20-MGGG%20Redistricting%20Lab-%230099cd)](https://mggg.org) 
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) 
+[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
 
 
 A companion to [GerryChain](https://github.com/mggg/GerryChain), GerryTools is
@@ -12,39 +19,73 @@ under a [3-Clause BSD License](https://opensource.org/licenses/BSD-3-Clause).
 
 
 ## Installation
+
 ### Using `pip` (recommended)
+
 To install GerryTools from [PyPi](https://pypi.org/project/gerrytools/), run
 
+```console
+pip install gerrytools
 ```
-$ pip install gerrytools
+
+from the command line. if you would like to use the `mgrp` and `ben` modules as well,
+you can invoke 
+
+```console
+pip install gerrytools[mgrp]
 ```
 
-from the command line.
+you will need to make sure that [Docker Desktop](https://www.docker.com/get-started/)
+is installed on your machine an updated to version >= 4.28.0. For more information on
+getting this set up, please see 
+[our documentation page](https://gerrytools.readthedocs.io/en/latest/topics/docker/)
 
 ## Usage
-### Contents
+
 GerryTools is split up into multiple sub-packages, each designed to simplify and
 standardize redistricting workflows.
 
-* **`gerrytools.data`** deals with the retrieval and processing of data. Here, you
-can find tools for grabbing decennial Census ('10 and '20), ACS 5-year ('12-'20),
-ACS CVAP Special Tab ('12-'20), districtr portal, and 2020 decennial Census geometric
-data. You can also find tools for moving CVAP data to other levels of geometry (e.g. prorating 2019 CVAP on 2019 Census tracts to 2020 blocks).
-* **`gerrytools.geometry`** provides facilities for dealing with geometric and
-related data. There are tools for translating and evaluating GerryChain
-[`Partition`](https://mggg.github.io/GerryChain/api.html#module-gerrychain.partition)s, performing fast geometric dissolutions, creating unit maps (e.g. 2020 blocks to 2020 VTDs), creating
-[dual graphs for GerryChain](https://mggg.github.io/GerryChain/api.html#adjacency-graphs), and optimization algorithms for renaming districts.
-* **`gerrytools.plotting`** contains methods for generating extremely high-quality 
-Lab-standard data visualizations.
-* **`gerrytools.scoring`** provides a vast array of redistricting plan scores. These
-can be used standalone _or_ as GerryChain
-[updaters](https://mggg.github.io/GerryChain/api.html#module-gerrychain.updaters).
-* **`gerrychain.utilities`** has ease-of-use methods for renaming directories
-containing shapefiles (which comes in handy more often than you'd think) and making
-JSON objects out of Python objects (useful when trying to organize information
-for many districting plans in a standard format).
-
-### Example
+* **`gerrytools.ben`** BEN (binary-ensemble) is our general purpose compression
+    algorithm for working with ensembles of plans. In general, the ben algorithm can
+    improve the storage of an ensemble of plans by an order of magnitude. When combined
+    with the special XBEN (eXtreme BEN) portion of the algorithm, many ensembles of
+    plans can be compressed small enough to fit into an email (~25Mb).
+
+* **`gerrytools.data`** deals with the retrieval and processing of data. Here, you can
+    find tools for grabbing decennial Census ('10 and '20), ACS 5-year ('12-'20), ACS CVAP
+    Special Tab ('12-'20), districtr portal, and 2020 decennial Census geometric data. You
+    can also find tools for moving CVAP data to other levels of geometry (e.g. prorating
+    2019 CVAP on 2019 Census tracts to 2020 blocks).
+
+* **`gerrytools.geometry`** provides facilities for dealing with geometric and related
+    data. There are tools for translating and evaluating GerryChain
+    [`Partition`](https://mggg.github.io/GerryChain/api.html#module-gerrychain.partition)s, 
+    performing fast geometric dissolutions, creating unit maps (e.g. 2020 blocks to
+    2020 VTDs), creating 
+    [dual graphs for GerryChain](https://mggg.github.io/GerryChain/api.html#adjacency-graphs),
+    and optimization algorithms for renaming districts.
+
+* **`gerrytools.mgrp`** this module uses a Docker container to allow users to access several
+    ensemble methods for generating districting plans on a state. In particular our Rust
+    implementation of our `gerrychain` library, `frcw`, the Julia implementation of 
+    [Forest Recom](https://arxiv.org/pdf/2008.08054.pdf), and the R/C++ implementation of
+    [Sequential Monte Carlo (SMC)](https://github.com/alarm-redist/redist) are available
+    through this module. 
+
+* **`gerrytools.plotting`** contains methods for generating extremely
+    high-quality Lab-standard data visualizations.
+
+* **`gerrytools.scoring`** provides a vast array of redistricting plan scores.
+    These can be used standalone _or_ as GerryChain 
+    [updaters](https://mggg.github.io/GerryChain/api.html#module-gerrychain.updaters).
+
+* **`gerrychain.utilities`** has ease-of-use methods for renaming
+    directories containing shapefiles (which comes in handy more often than you'd
+    think) and making JSON objects out of Python objects (useful when trying to
+    organize information for many districting plans in a standard format).
+
+<!-- ### Example
+
 GerryTools is easy to use and is designed to simplify data- and redistricting-related
 workflows. For example, let's say we want to analyze Alabama's citizen voting-age
 population data on 2020 Census tracts. First, we can download the geometric data
@@ -86,20 +127,25 @@ merged = geometric.merge(demographic, left_on="GEOID20", right_on="TRACT20")
 ```
 
 And there we are — what once took hours of setup and parsing now takes less than a
-minute.
+minute. -->
 
 ## Contributing
+
 GerryTools is an active project, and has multiple contributors. If you'd like to
 contribute, here are a few house rules:
 
 1. After cloning this repository, run `sh setup.sh` to download and install
 necessary git hooks and linting configurations.
+
 2. **Follow the [PEP8 style guide](https://peps.python.org/pep-0008/)**. After
 installing the above git hooks, linting is performed before every push. PEP8 errors can be automatically corrected by running `autopep8 --in-place --aggressive -r gerrytools` on the command line from the root directory.
+
 3. **Write tests.** All changes, major or minor, **must** be accompanied by testing
 code. Code and tests will be immediately reviewed by Lab maintainers.
+
 4. Test coverage must stay **at least** the same; this can be checked by running
 `pytest --cov=evaltools` after the tests are added to `tests/`.
+
 5. **Write documentation.** All changes should be documented via docstrings,
 and code should be repletely commented. It's much easier to decipher commented
 code! Docstring documentation is compiled on every commit via git hooks.

TODO.md

@@ -0,0 +1,57 @@
+- [ ] Make the `ben` and `mgrp` modes able to interact with the stdin of the docker
+  container so we don't always have to overwrite.
+
+
+## Major Rewrites and updates of the modules
+
+These need to be brought up-to-date with the current version of GerryChain and 
+Maup. They should also have redundant functionality trimmed.
+
+- [ ] Data
+  - [ ] ACS
+  - [ ] AssignmentCompressor (probably delete b/c of BEN)
+  - [ ] Census
+  - [ ] EstimateCVAP
+  - [ ] Fetch
+  - [ ] Geometries
+  - [ ] ReMap
+  - [ ] URLs
+
+- [ ] Geometry
+  - [ ] Compactness
+  - [ ] DataFrame
+  - [ ] Dissolve
+  - [ ] DualGraph
+  - [ ] Optimize
+  - [ ] Unit Map
+  - [ ] Updater
+
+- [ ] Plotting
+  - [ ] Annotation
+  - [ ] Bins
+  - [ ] BoxPlot
+  - [ ] Choropleth
+  - [ ] Colors
+  - [ ] DistrictNumbers
+  - [ ] DrawGraph
+  - [ ] DrawPlan
+  - [ ] Gifs
+  - [ ] Histogram
+  - [ ] MultiDimensional
+  - [ ] ScatterPlot
+  - [ ] SeaLevel
+  - [ ] Utils
+  - [ ] Violin
+
+- [ ] Scoring
+  - [ ] Contiguity
+  - [ ] Demogrpahics
+  - [ ] Partisan
+  - [ ] Population
+  - [ ] Scores
+  - [ ] Splits
+  - [ ] Types
+
+- [ ] Utilities
+  - [ ] JSON
+  - [ ] rename

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/docs_requirements.txt

@@ -0,0 +1,68 @@
+alabaster>=0.7.16,<1.0
+annotated-types>=0.6.0,<1.0
+attrs>=23.2.0,<24.0
+Babel>=2.14.0,<3.0
+CensusData>=1.15.post1,<2.0
+certifi>=2024.2.2,<2025.0
+charset-normalizer>=3.3.2,<4.0
+click>=8.1.7,<9.0
+click-plugins>=1.1.1,<2.0
+cligj>=0.7.2,<1.0
+contourpy>=1.2.0,<2.0
+cycler>=0.12.1,<1.0
+docutils>=0.20.1,<1.0
+docker==7.0.0,<8.0
+fiona>=1.9.5,<2.0
+fonttools>=4.49.0,<5.0
+geopandas>=0.12.2,<1.0
+gerrychain>=0.3.0,<1.0
+gurobipy>=11.0.0,<12.0
+idna>=3.6,<4.0
+imageio>=2.34.0,<3.0
+imagesize>=1.4.1,<2.0
+jellyfish>=0.11.2,<1.0
+Jinja2>=3.1.3,<4.0
+jsonlines>=4.0.0,<5.0
+kiwisolver>=1.4.5,<2.0
+markdown-it-py>=3.0.0,<4.0
+MarkupSafe>=2.1.5,<3.0
+matplotlib>=3.8.3,<4.0
+maup>=1.1.0,<2.0
+mdit-py-plugins>=0.4.0,<1.0
+mdurl>=0.1.2,<1.0
+myst-parser>=2.0.0,<3.0
+networkx>=3.2.1,<4.0
+numpy>=1.26.4,<2.0
+opencv-python-headless>=4.9.0.80,<5.0
+packaging>=23.2,<24.0
+pandas>=1.5.3,<2.0
+pillow>=10.2.0,<11.0
+pydantic>=2.6.1,<3.0
+pydantic_core>=2.16.2,<3.0
+Pygments>=2.17.2,<3.0
+pyparsing>=3.1.1,<4.0
+pyproj>=3.6.1,<4.0
+python-dateutil>=2.8.2,<3.0
+pytz>=2024.1,<2025.0
+PyYAML>=6.0.1,<7.0
+requests>=2.31.0,<3.0
+scipy>=1.12.0,<2.0
+seaborn>=0.13.2,<1.0
+shapely>=2.0.3,<3.0
+six>=1.16.0,<2.0
+snowballstemmer>=2.2.0,<3.0
+sortedcontainers>=2.4.0,<3.0
+Sphinx>=7.2.6,<8.0
+sphinx-copybutton>=0.5.2,<1.0
+sphinx-rtd-theme>=2.0.0,<3.0
+sphinxcontrib-applehelp>=1.0.8,<2.0
+sphinxcontrib-devhelp>=1.0.6,<2.0
+sphinxcontrib-htmlhelp>=2.0.5,<3.0
+sphinxcontrib-jquery>=4.1,<5.0
+sphinxcontrib-jsmath>=1.0.1,<2.0
+sphinxcontrib-qthelp>=1.0.7,<2.0
+sphinxcontrib-serializinghtml>=1.1.10,<2.0
+tqdm>=4.66.2,<5.0
+typing_extensions>=4.9.0,<5.0
+urllib3>=2.2.1,<3.0
+us>=3.1.1,<4.0