Correct possible bug #29
Conversation
(1) Line 88: if data.iloc[:, j].dtype in num_dtypes and any(data.iloc[:, j] > 0): --> if data.iloc[:, j].dtype in num_dtypes and all(data.iloc[:, j] > 0): (2) over_sampling.py, Line 283-285, 377-379: synth_matrix[i * x_synth + j, (d - 1)] = data.iloc[i, (d - 1)] + data.iloc[ knn_matrix[i, neigh], (d - 1)] / 2 --> synth_matrix[i * x_synth + j, (d - 1)] = ( data.iloc[i, (d - 1)] + data.iloc[ knn_matrix[i, neigh], (d - 1)] ) / 2 (3) over_sampling.py, Line 267, 363: for z in feat_list_num: --> for z in feat_list_num[0:(d-1)]: (Because when z=d-1, the entry synth_matrix[x_synth * n + count, z] is not defined yet.)
| (d - 1)] = data.iloc[i, (d - 1)] + data.iloc[ | ||
| knn_matrix[i, neigh], (d - 1)] / 2 | ||
| (d - 1)] = (data.iloc[i, (d - 1)] + data.iloc[ | ||
| knn_matrix[i, neigh], (d - 1)]) / 2 |
| ## generate synthetic y response variable by | ||
| ## inverse distance weighted | ||
| for z in feat_list_num: | ||
| for z in feat_list_num[0:(d - 1)]: |
There was a problem hiding this comment.
Why is this preferred (genuinely curious)?
There was a problem hiding this comment.
I made this change because synth_matrix[:,z] was only defined for 0:(d-1).
|
|
||
| for j in range(d): | ||
| if data.iloc[:, j].dtype in num_dtypes and any(data.iloc[:, j] > 0): | ||
| if data.iloc[:, j].dtype in num_dtypes and all(data.iloc[:, j] > 0): |
There was a problem hiding this comment.
What is this doing that any() is not?
There was a problem hiding this comment.
all() assures all elements are positive.
any() only assures that some elements are positive (there may also be negative elements).
I made this change because I thought the former is expected here.
| ## generate synthetic y response variable by | ||
| ## inverse distance weighted | ||
| for z in feat_list_num: | ||
| for z in feat_list_num[0:(d - 1)]: |
There was a problem hiding this comment.
Why is this preferred (genuinely curious)?
There was a problem hiding this comment.
I made this change because synth_matrix[:,z] was only defined for 0:(d-1).
|
|
||
| if a == b: | ||
| synth_matrix[x_synth * n + count, (d - 1)] = data.iloc[ | ||
| synth_matrix[x_synth * n + count, (d - 1)] = (data.iloc[ |
|
@seazuma I hope that you might be able to address some of the questions I had in your PR. Thank you. |
|
@nickkunz I apologize for having failed to notice your comments before. Thank you for letting me know again. I have replied to them. |
2 participants
I appreciate your library smogn. I found some suspicious points in over_sampling.py and tried to fix them.
(1) Line 88:
if data.iloc[:, j].dtype in num_dtypes and any(data.iloc[:, j] > 0):
-->
if data.iloc[:, j].dtype in num_dtypes and all(data.iloc[:, j] > 0):
(2) over_sampling.py, Line 283-285, 377-379:
synth_matrix[i * x_synth + j,
(d - 1)] = data.iloc[i, (d - 1)] + data.iloc[
knn_matrix[i, neigh], (d - 1)] / 2
-->
synth_matrix[i * x_synth + j,
(d - 1)] = ( data.iloc[i, (d - 1)] + data.iloc[
knn_matrix[i, neigh], (d - 1)] ) / 2
(3) over_sampling.py, Line 267, 363:
for z in feat_list_num:
-->
for z in feat_list_num[0:(d-1)]:
(Because when z=d-1, the entry synth_matrix[x_synth * n + count, z] is not defined yet.)