initial

Author: OnlyDeniko

report_generator/README.md

@@ -2,10 +2,14 @@
 
 Report generator produces Excel table file from json benchmark log files.
 
-Run `python report_generator.py --result-files bench_log_1.json,bench_log_2.json [--report-file new_report.xlsx --generation-config gen_config.json --merging none]` to launch report generation.
+Run `python report_generator.py --result-files bench_log_1.json,bench_log_2.json [--report-file new_report.xlsx --generation-config default_report_gen_config.json]` to launch report generation.
 
 runner options:
 * ``result-files`` : comma-separated benchmark json result file paths
 * ``report-file`` : report file path
 * ``generation-config`` : generation configuration file path
-* ``merging``: *full*, *none*, *sw_only*, *hw_only*. How to merge same cases in benchmark logs
+
+config parameters:
+* ``header`` : Column names in the table header. These parameters are also used to compare reports. If the name is compound, then it should be separated by the '':'' symbol
+* ``comparison_method`` : Comparison of two results. The options are: ``1 operation 2`` or ``2 operation 1``
+* ``aggregation_metrics`` : Metric applied to columns with comparisons of two reports. For each of these metrics, a separate sheet with a summary is compiled. It is important that the function is in Excel.

report_generator/default_report_gen_config.json

@@ -1,5 +1,5 @@
 {
-    "align": [
+    "header": [
         "algorithm",
         "stage",
         "input_data:data_order",
@@ -10,9 +10,22 @@
         "input_data:classes",
         "input_data:n_clusters"
     ],
-    "diff": [
-        "software_hash",
-        "hardware_hash",
-        "measurement_time"
+    "comparison_method": {
+        "time[s]": "2 / 1",
+        "davies_bouldin_score": "2 / 1",
+        "inertia": "2 / 1",
+        "iter": "2 / 1",
+        "noise_variance": "2 / 1",
+        "accuracy": "2 / 1",
+        "log_loss": "2 / 1",
+        "roc_auc": "2 / 1",
+        "rmse": "2 / 1",
+        "r2_score": "2 / 1",
+        "n_sv": "2 / 1",
+        "n_clusters": "2 / 1"
+    },
+    "aggregation_metrics": [
+        "geomean",
+        "average"
     ]
-}
+}