list_soils failing on some coordinates

[knipec]

Description

• Some coordinates are giving failure results for soil ID
• This is causing both 1.2 and 1.3 backend endpoints to return ALGORITHM_FAILURE
• Seems similar to App returns "No Matches" but Soil Web provides matches #215
• This is currently deployed in prod. As far as I can tell, this behavior has been consistent between pre-global and post-global, so we are not considering it a shipblocker for global functionality.

Steps To Reproduce

Put these locations in the search bar to get their soil ID matches:

• (Iowa) us_soil.list_soils(lat=43.04364, lon=-92.30143) fails
  (Soil ID gave "Error" yesterday, but "No matches" today.. maybe something was down yesterday?)

web-1         | 0.31: https://soilmap4-1.lawr.ucdavis.edu/api/landPKS.php
web-1         | 2025-07-16T18:05:45.124352Z [error    ] Traceback (most recent call last):
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 417, in resolve_data_based_result
web-1         |     list_result = get_list_soils_output(latitude=latitude, longitude=longitude)
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 207, in get_list_soils_output
web-1         |     list_output = us_soil.list_soils(lat=latitude, lon=longitude)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/us_soil.py", line 1121, in list_soils
web-1         |     aws_PIW90, var_imp = soil_sim(muhorzdata_pd)
web-1         |                          ~~~~~~~~^^^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/soil_sim.py", line 528, in soil_sim
web-1         |     group_df1 = groups1.get_group(group_label).reset_index(drop=True)
web-1         |                 ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/groupby/groupby.py", line 1115, in get_group
web-1         |     raise KeyError(name)
web-1         | KeyError: 'klossner'

• (Massachusetts 1) us_soil.list_soils(lat=42.13097, lon=-73.27261) fails

web-1         | 0.26: https://soilmap4-1.lawr.ucdavis.edu/api/landPKS.php
web-1         | 2025-07-16T18:08:44.190380Z [error    ] Traceback (most recent call last):
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 417, in resolve_data_based_result
web-1         |     list_result = get_list_soils_output(latitude=latitude, longitude=longitude)
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 207, in get_list_soils_output
web-1         |     list_output = us_soil.list_soils(lat=latitude, lon=longitude)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/us_soil.py", line 1121, in list_soils
web-1         |     aws_PIW90, var_imp = soil_sim(muhorzdata_pd)
web-1         |                          ~~~~~~~~^^^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/soil_sim.py", line 528, in soil_sim
web-1         |     group_df1 = groups1.get_group(group_label).reset_index(drop=True)
web-1         |                 ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/groupby/groupby.py", line 1115, in get_group
web-1         |     raise KeyError(name)
web-1         | KeyError: 'natchaug'

• (Massachusetts 2) us_soil.list_soils(lat=42.5473 lon=-71.98151) fails:

web-1         | 0.29: https://soilmap4-1.lawr.ucdavis.edu/api/landPKS.php
web-1         | 2025-07-16T18:14:23.214681Z [error    ] Traceback (most recent call last):
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 417, in resolve_data_based_result
web-1         |     list_result = get_list_soils_output(latitude=latitude, longitude=longitude)
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 207, in get_list_soils_output
web-1         |     list_output = us_soil.list_soils(lat=latitude, lon=longitude)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/us_soil.py", line 1121, in list_soils
web-1         |     aws_PIW90, var_imp = soil_sim(muhorzdata_pd)
web-1         |                          ~~~~~~~~^^^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/soil_sim.py", line 528, in soil_sim
web-1         |     group_df1 = groups1.get_group(group_label).reset_index(drop=True)
web-1         |                 ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/groupby/groupby.py", line 1115, in get_group
web-1         |     raise KeyError(name)
web-1         | KeyError: 'wonsqueak'

• (Alaska) us_soil.list_soils(lat=61.41719 lon=-162.22394) fails:

web-1         | 0.35: https://soilmap4-1.lawr.ucdavis.edu/api/landPKS.php
web-1         | 2025-07-16T18:17:33.051780Z [error    ] Traceback (most recent call last):
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 417, in resolve_data_based_result
web-1         |     list_result = get_list_soils_output(latitude=latitude, longitude=longitude)
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 207, in get_list_soils_output
web-1         |     list_output = us_soil.list_soils(lat=latitude, lon=longitude)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/us_soil.py", line 1121, in list_soils
web-1         |     aws_PIW90, var_imp = soil_sim(muhorzdata_pd)
web-1         |                          ~~~~~~~~^^^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/soil_sim.py", line 528, in soil_sim
web-1         |     group_df1 = groups1.get_group(group_label).reset_index(drop=True)
web-1         |                 ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/groupby/groupby.py", line 1115, in get_group
web-1         |     raise KeyError(name)
web-1         | KeyError: 'e38-maritime scrub-organic frozen mounds'

• (Hawaii) us_soil.list_soils(lat=19.46499 lon=-155.107) fails:

web-1         | 0.31: https://soilmap4-1.lawr.ucdavis.edu/api/landPKS.php
web-1         | 2025-07-16T18:23:44.116766Z [error    ] Traceback (most recent call last):
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 417, in resolve_data_based_result
web-1         |     list_result = get_list_soils_output(latitude=latitude, longitude=longitude)
web-1         |   File "/app/terraso_backend/apps/soil_id/graphql/soil_id/resolvers.py", line 207, in get_list_soils_output
web-1         |     list_output = us_soil.list_soils(lat=latitude, lon=longitude)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/us_soil.py", line 1121, in list_soils
web-1         |     aws_PIW90, var_imp = soil_sim(muhorzdata_pd)
web-1         |                          ~~~~~~~~^^^^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/soil_sim.py", line 126, in soil_sim
web-1         |     sim = infill_soil_data(sim)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/soil_id/soil_sim.py", line 606, in infill_soil_data
web-1         |     result_df = pd.concat(processed_groups, ignore_index=True)
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/reshape/concat.py", line 382, in concat
web-1         |     op = _Concatenator(
web-1         |         objs,
web-1         |     ...<8 lines>...
web-1         |         sort=sort,
web-1         |     )
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/reshape/concat.py", line 445, in __init__
web-1         |     objs, keys = self._clean_keys_and_objs(objs, keys)
web-1         |                  ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
web-1         |   File "/home/terraso/venv/lib/python3.13/site-packages/pandas/core/reshape/concat.py", line 507, in _clean_keys_and_objs
web-1         |     raise ValueError("No objects to concatenate")
web-1         | ValueError: No objects to concatenate

Expected: Soil ID match results
Actual: Soil ID sometimes takes a long time and gives "Error" or "No matches"

[knipec]

Hi @jjmaynard, we found some more locations that seem to error in list_soils code. So far we've only seen it in US locations. Are you able to take a look? Thanks!

[jjmaynard]

@knipec @garobrik Thanks for finding these examples Carissa. These are all locations that have organic soils which is currently causing a bug in how we process the horizon data. I will work on a fix.

[garobrik]

FYI: some of these look like the same bug as #184, so we should validate and close together if it makes sense to do so!