Skip to content

Add metrics testing #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Aug 17, 2021
Merged

Add metrics testing #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
facd58f
add configs
OnlyDeniko Aug 11, 2021
7078ca5
add report generator metrics config
OnlyDeniko Aug 11, 2021
202a11d
add configs
OnlyDeniko Aug 11, 2021
14e85b2
add report generator metrics config
OnlyDeniko Aug 11, 2021
2a90196
add knn_regr
OnlyDeniko Aug 11, 2021
a28a3d2
Merge branch 'add-metric_testing' of https://github.com/OnlyDeniko/sc…
OnlyDeniko Aug 11, 2021
98dbf28
add ci testing for knn_regr
OnlyDeniko Aug 11, 2021
8d6d76e
rename algos in cuml
OnlyDeniko Aug 11, 2021
b4cdd6c
rename algos in sklearn
OnlyDeniko Aug 11, 2021
e9a51c7
reworked kmeans config for test metrics
OnlyDeniko Aug 11, 2021
f96893d
add init to report config
OnlyDeniko Aug 11, 2021
2ca8190
add opportunity to write path to folder in configs
OnlyDeniko Aug 11, 2021
d8689a7
pep8
OnlyDeniko Aug 11, 2021
5d11996
change readme
OnlyDeniko Aug 12, 2021
b9babf4
apply comments, part1
OnlyDeniko Aug 12, 2021
618e7c4
apply comments, part2
OnlyDeniko Aug 12, 2021
57dd110
dfs strategy to take jsons from folders
OnlyDeniko Aug 12, 2021
4a21cd8
now you can --configs=folder,.json,folder
OnlyDeniko Aug 12, 2021
e4d6f94
Update README.md
OnlyDeniko Aug 12, 2021
6a126d8
Update sklearn_bench/knn_regr.py
OnlyDeniko Aug 12, 2021
ecb8938
Update runner.py
OnlyDeniko Aug 12, 2021
67cc941
EOF
OnlyDeniko Aug 13, 2021
3ab40f5
Merge branch 'add-metric_testing' of https://github.com/OnlyDeniko/sc…
OnlyDeniko Aug 13, 2021
ec05b86
apply comment
OnlyDeniko Aug 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul

Options:

- ``--configs``: specify the path to a configuration file.
- ``--configs``: specify the path to a configuration file or a folder that contains configuration files.
- ``--no-intel-optimized``: use Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). By default, the runner uses Intel(R) Extension for Scikit-learn.
- ``--output-file``: specify the name of the output file for the benchmark result. The default name is `result.json`
- ``--report``: create an Excel report based on benchmark results. The `openpyxl` library is required.
Expand Down
104 changes: 52 additions & 52 deletions configs/blogs/skl_2021_3.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
{
"common": {
"lib": ["sklearn"],
"data-format": ["pandas"],
"data-order": ["F"],
"dtype": ["float64"]
"lib": "sklearn",
"data-format": "pandas",
"data-order": "F",
"dtype": "float64"
},
"cases": [
{
Expand All @@ -19,11 +19,11 @@
}
}
],
"time-method": ["box_filter"],
"time-limit": [50],
"n-clusters": [1000],
"maxiter": [50],
"tol": [0.0]
"time-method": "box_filter",
"time-limit": 50,
"n-clusters": 1000,
"maxiter": 50,
"tol": 0.0
},
{
"algorithm": "kmeans",
Expand All @@ -38,12 +38,12 @@
}
}
],
"time-method": ["box_filter"],
"time-limit": [50],
"n-clusters": [5],
"maxiter": [50],
"init": ["k-means++"],
"tol": [0.0]
"time-method": "box_filter",
"time-limit": 50,
"n-clusters": 5,
"maxiter": 50,
"init": "k-means++",
"tol": 0.0
},
{
"algorithm": "kmeans",
Expand All @@ -58,11 +58,11 @@
}
}
],
"time-method": ["box_filter"],
"time-limit": [50],
"n-clusters": [20],
"maxiter": [50],
"tol": [0.0]
"time-method": "box_filter",
"time-limit": 50,
"n-clusters": 20,
"maxiter": 50,
"tol": 0.0
},
{
"algorithm": "pca",
Expand Down Expand Up @@ -107,12 +107,12 @@
}
}
],
"svd-solver": ["full"],
"n-components": [10]
"svd-solver": "full",
"n-components": 10
},
{
"algorithm": "df_clsf",
"dtype": ["float32"],
"dtype": "float32",
"dataset": [
{
"source": "npy",
Expand All @@ -129,10 +129,10 @@
}
}
],
"num-trees": [50],
"max-depth": [16],
"max-leaf-nodes": [131072],
"max-features": [0.2]
"num-trees": 50,
"max-depth": 16,
"max-leaf-nodes": 131072,
"max-features": 0.2
},
{
"algorithm": "ridge",
Expand All @@ -146,7 +146,7 @@
}
}
],
"alpha": [5]
"alpha": 5
},
{
"algorithm": "linear",
Expand Down Expand Up @@ -201,8 +201,8 @@
}
}
],
"maxiter": [100],
"tol": [0]
"maxiter": 100,
"tol": 0
},
{
"algorithm": "svm",
Expand All @@ -222,8 +222,8 @@
}
}
],
"C": [500.0],
"kernel": ["rbf"]
"C": 500.0,
"kernel": "rbf"
},
{
"algorithm": "svm",
Expand All @@ -243,8 +243,8 @@
}
}
],
"C": [1.5e-3],
"kernel": ["linear"]
"C": 1.5e-3,
"kernel": "linear"
},
{
"algorithm": "svm",
Expand All @@ -264,8 +264,8 @@
}
}
],
"C": [100.0],
"kernel": ["linear"]
"C": 100.0,
"kernel": "linear"
},
{
"algorithm": "svm",
Expand All @@ -285,8 +285,8 @@
}
}
],
"C": [50.0],
"kernel": ["rbf"]
"C": 50.0,
"kernel": "rbf"
},
{
"algorithm": "nusvc",
Expand All @@ -306,8 +306,8 @@
}
}
],
"nu": [0.25],
"kernel": ["poly"]
"nu": 0.25,
"kernel": "poly"
},
{
"algorithm": "svr",
Expand All @@ -327,8 +327,8 @@
}
}
],
"C": [0.1],
"kernel": ["poly"]
"C": 0.1,
"kernel": "poly"
},
{
"algorithm": "nusvr",
Expand All @@ -348,9 +348,9 @@
}
}
],
"nu": [0.8],
"C": [2.0],
"kernel": ["rbf"]
"nu": 0.8,
"C": 2.0,
"kernel": "rbf"
},
{
"algorithm": "dbscan",
Expand Down Expand Up @@ -386,7 +386,7 @@
},
{
"algorithm": "knn_clsf",
"dtype": ["float32"],
"dtype": "float32",
"dataset": [
{
"source": "synthetic",
Expand Down Expand Up @@ -437,11 +437,11 @@
}
}
],
"method": ["brute"]
"method": "brute"
},
{
"algorithm": "knn_clsf",
"dtype": ["float32"],
"dtype": "float32",
"dataset": [
{
"source": "synthetic",
Expand All @@ -468,7 +468,7 @@
}
}
],
"method": ["kd_tree"]
"method": "kd_tree"
},
{
"algorithm": "train_test_split",
Expand All @@ -483,9 +483,9 @@
}
}
],
"include-y": [""],
"train-size": [0.75],
"test-size": [0.25]
"include-y": "",
"train-size": 0.75,
"test-size": 0.25
}
]
}
Loading