Skip to content

fix: global code update #225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 56 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
f0ac653
fix: update global soilid code
jjmaynard Dec 20, 2024
64cf160
fix: update global soilid code
jjmaynard Dec 20, 2024
93af38f
fix: postgres database integration
jjmaynard Jan 7, 2025
f5a11fb
Merge branch 'fix/global-code-update' of https://github.com/techmatte…
jjmaynard Jan 7, 2025
9ff26ea
fix: created .env file and updated .gitignore
jjmaynard Jan 10, 2025
f8bfc57
fix: revision of global functions
jjmaynard Feb 4, 2025
37561eb
fix: remove test files
jjmaynard Feb 4, 2025
b273b3d
fix: lint format changes
jjmaynard Feb 4, 2025
51cf8c8
fix: make format changes
jjmaynard Feb 4, 2025
ddab942
style: use LF line endings
paulschreiber Feb 4, 2025
d9ba6ab
Normalize line endings to LF
jjmaynard Feb 6, 2025
0b6c5c9
Update soil_id/db.py
jjmaynard Feb 12, 2025
418ea47
fix: new psql color tables
jjmaynard Feb 18, 2025
082568b
Merge branch 'fix/global-code-update' of https://github.com/techmatte…
jjmaynard Feb 18, 2025
1299c7e
fix: update psql functions
jjmaynard Feb 27, 2025
959add1
fix: HWSDv2 postgres integration
jjmaynard Mar 4, 2025
605b369
fix: global color calculation
jjmaynard Mar 4, 2025
587cc7e
feat: update tests for global algorithm
shrouxm Mar 11, 2025
2fc46f0
fix: global bugs
jjmaynard Mar 21, 2025
c24137a
test: update tests
shrouxm Mar 25, 2025
40261a0
fix: traceback errors-geo
jjmaynard Mar 28, 2025
3e70f5b
fix: no map data return
jjmaynard Mar 28, 2025
ac0a581
fix: make black/lint format
jjmaynard Mar 31, 2025
e86c5df
fix: sql code update
jjmaynard Apr 10, 2025
d26ba68
fix: update global soilid code
jjmaynard Dec 20, 2024
e7cd519
fix: postgres database integration
jjmaynard Jan 7, 2025
96f6c46
fix: update global soilid code
jjmaynard Dec 20, 2024
a429d2e
fix: created .env file and updated .gitignore
jjmaynard Jan 10, 2025
b03b133
fix: revision of global functions
jjmaynard Feb 4, 2025
1f91f2d
fix: remove test files
jjmaynard Feb 4, 2025
2ef351b
fix: lint format changes
jjmaynard Feb 4, 2025
d98bbb7
fix: make format changes
jjmaynard Feb 4, 2025
00a63f4
style: use LF line endings
paulschreiber Feb 4, 2025
05fe937
Normalize line endings to LF
jjmaynard Feb 6, 2025
c671b67
fix: new psql color tables
jjmaynard Feb 18, 2025
ea5f979
Update soil_id/db.py
jjmaynard Feb 12, 2025
8409988
fix: update psql functions
jjmaynard Feb 27, 2025
60d8d20
fix: HWSDv2 postgres integration
jjmaynard Mar 4, 2025
4f0d1c9
fix: global color calculation
jjmaynard Mar 4, 2025
26f1e84
feat: update tests for global algorithm
shrouxm Mar 11, 2025
f9f5cda
fix: global bugs
jjmaynard Mar 21, 2025
65152ab
test: update tests
shrouxm Mar 25, 2025
4e02a14
fix: traceback errors-geo
jjmaynard Mar 28, 2025
8d6e0b1
fix: no map data return
jjmaynard Mar 28, 2025
6a5ef1e
fix: make black/lint format
jjmaynard Mar 31, 2025
0951ee4
fix: sql code update
jjmaynard Apr 10, 2025
d54c625
build: update Makefile README.md pyproject.toml requirements-dev.txt …
paulschreiber Apr 14, 2025
ab9d207
style: run make format
paulschreiber Apr 14, 2025
caebd71
fix: fix imports in config
paulschreiber Apr 14, 2025
22eccb5
fix: remove unsued variables
paulschreiber Apr 14, 2025
6a947a2
fix: global test dataset and testing
jjmaynard Apr 22, 2025
e16183e
Merge branch 'fix/global-code-update' of https://github.com/techmatte…
jjmaynard Apr 22, 2025
d657f9a
test: update tests
shrouxm May 6, 2025
e727737
dx: add docker file for dev database
shrouxm May 6, 2025
a4fef55
fix: lint
shrouxm May 6, 2025
9cc216a
test: skip global test by default (no DB in CI)
shrouxm May 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,6 @@ prof/

# reference and output files
Data/
bulk_test_result*

get-pip.py*
16 changes: 14 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,24 @@ graphs:
gprof2dot -f pstats prof/test_soil_location.prof | dot -Tsvg -o prof/test_soil_location.svg
flameprof prof/test_soil_location.prof > prof/test_soil_location_flame.svg

generate_bulk_test_results:
generate_bulk_test_results_us:
python -m soil_id.tests.us.generate_bulk_test_results

process_bulk_test_results:
process_bulk_test_results_us:
python -m soil_id.tests.us.process_bulk_test_results $(RESULTS_FILE)

generate_bulk_test_results_global:
python -m soil_id.tests.global.generate_bulk_test_results

process_bulk_test_results_global:
python -m soil_id.tests.global.process_bulk_test_results $(RESULTS_FILE)

generate_bulk_test_results_legacy:
python -m soil_id.tests.legacy.generate_bulk_test_results

process_bulk_test_results_legacy:
python -m soil_id.tests.legacy.process_bulk_test_results $(RESULTS_FILE)

# Donwload Munsell CSV, SHX, SHP, SBX, SBN, PRJ, DBF
download-soil-data:
mkdir -p Data
Expand Down
16 changes: 16 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: soil_data

services:
db:
image: postgis/postgis
ports:
- "5432:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: soil_data # default database
volumes:
- pg_data:/var/lib/postgresql/data

volumes:
pg_data:
8 changes: 1 addition & 7 deletions soil_id/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,23 +12,19 @@
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see https://www.gnu.org/licenses/.

import os
import tempfile

from dotenv import load_dotenv
from platformdirs import user_cache_dir

load_dotenv()

DATA_PATH = os.environ.get("DATA_PATH", "Data")

# Numpy seeding
RANDOM_SEED = os.environ.get("RANDOM_SEED", 19)

# Output
APP_NAME = os.environ.get("APP_NAME", "org.terraso.soilid")
TEMP_DIR = tempfile.TemporaryDirectory(delete=False)
TEMP_DIR = tempfile.TemporaryDirectory()
CACHE_DIR = user_cache_dir(APP_NAME)
OUTPUT_PATH = TEMP_DIR.name
SOIL_ID_RANK_PATH = f"{OUTPUT_PATH}/soil_id_rank.csv"
Expand All @@ -45,8 +41,6 @@
# Global Soil ID
HWSD_PATH = f"{DATA_PATH}/HWSD_global_noWater_no_country.shp"
WISE_PATH = f"{DATA_PATH}/wise30sec_poly_simp_soil.shp"
NORM_DIST_1_PATH = f"{DATA_PATH}/NormDist1.csv"
NORM_DIST_2_PATH = f"{DATA_PATH}/NormDist2.csv"

# Database
DB_NAME = os.environ.get("DB_NAME", "terraso_backend")
Expand Down
Loading
Loading