imported

Author: musaatreadmesys

README

@@ -0,0 +1,51 @@
+1. The jobs below will produce all const-loc data for 3d-plots as well as all const-loc evaluation plots for osm2 dataset
+
+-Run error-delay and budget-delay jobs at first.
+./evaluator_jobs_sampler_osm2_errordelay_budgetdelay_constloc.sh 
+./process_evals_jobs_osm2_errordelay_budgetdelay_constloc.sh
+./scripts/process_osm2_errordelay_budgetdelay_constloc.sh
+
+- Then run error-budget jobs as error-budget depends on the data produced from error-delay sampler.
+./evaluator_jobs_sampler_osm2_errordelay_budgetdelay_constloc.sh 
+./process_evals_jobs_osm2_errordelay_budgetdelay_constloc.sh
+./scripts/process_osm2_errordelay_budgetdelay_constloc.sh
+
+-Then run following scripts to produce the plots
+./scripts/plot_osm2_constloc.sh
+
+
+2a. For *osm2* unified
+./evaluator_jobs_sampler_osm2_unified.sh
+./process_evals_jos_osm2_unified.sh
+./scrips/process_osm2_unified.sh
+./scripts/process_osm2_end2end.sh
+
+Then fill the table from this file:
+  data_new/error_delay_percentage_reduction_osm2_end2end
+
+2b. For *uic/msmls* unifield
+./evaluator_jobs_sampler_uic_unified.sh (or msmls)
+./evaluator_new_results_combine.sh (*** to combine the data from splits, for both uic and msmls)
+./process_evals_jos_uic_unified.sh (or msmls)
+./scrips/process_uic_unified.sh (or msmls)
+./process_new_end2end_unifiedonly.sh (for both uic and msmls) [for only unified data that fill the table in the paper]
+or                                     
+./scripts/process_uic_end2end.sh (or msmls) [for both const-loc and unified data] 
+
+Then fill the table from  these files:
+  data_new/error_delay_percentage_reduction_uic_end2end_unifiedonly 
+  data_new/budget_delay_percentage_reduction_p1800.0_uic_end2end_unifiedonly (or p3600.0)
+
+-Then run scripts/plot_new_end2end.sh to produce plot for all datasets
+
+3. to produce 3d data
+./scripts/produce_data_3d.sh
+./scripts/produce_data_3d_unified.sh
+
+to produce convergence plots for the paper
+./scripts/plot_new_convergence.sh 
+
+
+
+=========================
+all scripts with *test* are for small dataset

README_Sampler.txt

@@ -0,0 +1,42 @@
+Some directories are used locally to avoid committing too many files in the svn. The final data used for generating plots are only committed to svn. The plots are also kept locally as they will change a lot over time. All the local directories that the scripts will need are given below. Create them locally.
+
+evaluator_output_msmls
+evaluator_output_osm
+evaluator_output_uic
+processed_evals_output_msmls
+processed_evals_output_osm
+processed_evals_output_uic
+plots
+
+The overall evaluation process is given below. I run everything in the server that produces data/* and then commit to get them locally and produce the plots.
+Run 1 to 3 in the server. And 4 in the local machine.
+
+1. Run jobs using parallel for a specific sampler. In the job script, change data or extrapolator as necessary at the top. It might be better to test a single data-set (or even subset of that) with a particular extrapolator to run things quickly. Example command is given below. This will produce output files at evaluator_output_<category>
+
+   ./evaluator_jobs_sampler_error_delay.sh | parallel
+
+2. Process the evaluator output with parallel with the command below. Change data-set, extrapolator on the top of the script. Comment/uncomment a sampler as it is used/unused. This will produce results in the processed_evals_output_<category>
+
+   ./process_evals_jobs_new.sh | parallel
+
+3. Process the evaluation data using the command below. Similar to previous scripts comment/uncomment whatever necessary to produce the results quickly.
+
+   ./scripts/process_new.sh
+
+->> data_new/
+[] update parameters
+
+[COMMIT now. This will commit the data files]
+
+4. Generate the plots using:
+   
+   ./scripts/plot_new.sh
+
+-->> reads from data_new/
+plots_new/
+
+
+=================================================
+Producing statistical tables:
+-----------------------------
+./offline_table_exponential_jobs.sh | parallel