fix(matrix): allow odd matrix format when importing #2305
Conversation
MartinBelthle
changed the title
antarest/study/service.py
Outdated
| df = df.dropna(axis=1, how="all") # We want to remove columns full of NaN at the import | ||
| matrix = df.to_numpy(dtype=np.float64) | ||
| return matrix | ||
| raise ValueError("Could not import the matrix") |
There was a problem hiding this comment.
Do we have more informations to make the error message more explicit ?
There was a problem hiding this comment.
Unfortunately, inside the calling method, I don't even know on which study I'm applied
There was a problem hiding this comment.
But I can perhaps catch it higher in the code to raise a different error. For the moment we have a HTTP 500 saying Unexpected server error: Could not import the matrix
There was a problem hiding this comment.
I added a home-made exception to raise a 422 with a clearer exception name (+ a test). So it's slightly better
| return np.zeros(shape=(0, 0)) | ||
| for delimiter in [",", ";", "\t"]: | ||
| with contextlib.suppress(Exception): | ||
| df = pd.read_csv(io.BytesIO(data), delimiter=delimiter, header=None).replace(",", ".", regex=True) |
There was a problem hiding this comment.
is there a way to load the matrix only once ? Maybe we can do something to "detect" the delimiter to prevent three openings
There was a problem hiding this comment.
I just realised that it is exactly what was done before ...
There was a problem hiding this comment.
Yep indeed. And it's because it wasn't reliable that I had to do this for loop
MartinBelthle
force-pushed
the
fix/add-test-case-for-matrix-import
branch
from
d374515 to
c4499d6
Compare
The csv sniffer is not reliable and can detect the wrong delimiter. So we can't trust it.
We have to check ourselves. It's probably a bit longer but at least we don't have failures anymore