Reported results #11
Description
Hi,
I ran the simulation using the provided configuration and observed results that appear to differ from the results reported in your paper. Despite my efforts to investigate the cause of this discrepancy, I have been unable to determine why these results differ.
I have used the following configuration:
p_alpha=[0.3]
p_lambda=[1e-4]
n_in=[3]
n_out=[3]
nonlin='elu'
lrate=[1e-3]
batch_size=[100]
dim_in=[200]
dim_out=[100]
batch_norm=[0]
normalization=['divide']
imb_fun=['wass']
experiments=1000
reweight_sample=[1]
split_output=[1]
rep_weight_decay=[0]
dropout_in=1.0
dropout_out=1.0
rbf_sigma=0.1
lrate_decay=0.97
decay=0.3
optimizer='Adam'
wass_lambda=10.0
wass_iterations=10
wass_bpt=1
use_p_correction=0
iterations=3000
weight_init=[0.1]
outdir='results/example_ihdp'
datadir='data/'
dataform='ihdp_npci_1-1000.train.npz'
data_test='ihdp_npci_1-1000.test.npz'
pred_output_delay=200
loss='l2'
sparse=0
varsel=0
repetitions=1
val_part=0.3
and I got the following results:
Mode: Train
| Pehe | Bias_ate | Rmse_fact | Rmse_cfact | Rmse_ite | Objective | Pehe_nn
0 | 0.785 +/- 0.052 | 0.219 +/- 0.031 | 1.135 +/- 0.045 | 1.188 +/- 0.020 | 1.164 +/- 0.021 | 0.994 +/- 0.036 | 4.895 +/- 0.613
Mode: Valid
| Pehe | Bias_ate | Rmse_fact | Rmse_cfact | Rmse_ite | Objective | Pehe_nn
0 | 0.884 +/- 0.084 | 0.224 +/- 0.031 | 1.363 +/- 0.091 | 1.206 +/- 0.032 | 1.201 +/- 0.032 | 1.995 +/- 0.402 | 5.661 +/- 0.799
Mode: Test
| Pehe | Bias_ate | Rmse_fact | Rmse_cfact | Rmse_ite | Pehe_nn
0 | 0.833 +/- 0.056 | 0.236 +/- 0.032 | 1.287 +/- 0.065 | 1.182 +/- 0.024 | 1.183 +/- 0.022 | 5.456 +/- 0.682
Thank you for your attention to this matter, and I look forward to your response.