From 672c8ccaef5e8e22fc2942ee0f83f256b05dda59 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Tue, 30 Mar 2021 15:05:27 +0300 Subject: [PATCH 01/10] Update readme --- README.md | 34 +++++++++---------- configs/README.md | 1 + ...ontext_config.json => skl_xpu_config.json} | 0 3 files changed, 17 insertions(+), 18 deletions(-) rename configs/{skl_with_context_config.json => skl_xpu_config.json} (100%) diff --git a/README.md b/README.md index 10ec2bc2d..63cc0f82d 100755 --- a/README.md +++ b/README.md @@ -26,9 +26,7 @@ We publish blogs on Medium, so [follow us](https://medium.com/intel-analytics-so ## Table of content -* [Prerequisites](#prerequisites) * [How to create conda environment for benchmarking](#how-to-create-conda-environment-for-benchmarking) -* [How to enable daal4py patching for scikit-learn benchmarks](#how-to-enable-daal4py-patching-for-scikit-learn-benchmarks) * [Running Python benchmarks with runner script](#running-python-benchmarks-with-runner-script) * [Supported algorithms](#supported-algorithms) * [Algorithms parameters](#algorithms-parameters) @@ -91,22 +89,22 @@ The configuration of benchmarks allows you to select the frameworks to run, sele ## Benchmark supported algorithms -| algorithm | benchmark name | sklearn | daal4py | cuml | xgboost | -|---|---|---|---|---|---| -|**[DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html)**|dbscan|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)**|df_clfs|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html)**|df_regr|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[pairwise_distances](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html)**|distances|:white_check_mark:|:white_check_mark:|:x:|:x:| -|**[KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html)**|kmeans|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[KNeighborsClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)**|knn_clsf|:white_check_mark:|:x:|:white_check_mark:|:x:| -|**[LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)**|linear|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)**|log_reg|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html)**|pca|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[Ridge](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html)**|ridge|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)**|svm|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)**|train_test_split|:white_check_mark:|:x:|:white_check_mark:|:x:| -|**[GradientBoostingClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| -|**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| +| algorithm | benchmark name | sklearn | sklearn on GPU | daal4py | cuml | xgboost | +|---|---|---|---|---|---|---| +|**[DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html)**|dbscan|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)**|df_clfs|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| +|**[RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html)**|df_regr|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| +|**[pairwise_distances](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html)**|distances|:white_check_mark:|:x:|:white_check_mark:|:x:|:x:| +|**[KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html)**|kmeans|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[KNeighborsClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)**|knn_clsf|:white_check_mark:|:x:|:x:|:white_check_mark:|:x:| +|**[LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)**|linear|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)**|log_reg|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html)**|pca|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| +|**[Ridge](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html)**|ridge|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| +|**[SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)**|svm|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| +|**[train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)**|train_test_split|:white_check_mark:|:x:|:x:|:white_check_mark:|:x:| +|**[GradientBoostingClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)**|gbt|:x:|:x:|:x:|:x:|:white_check_mark:| +|**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:x:|:white_check_mark:| ## Algorithms parameters diff --git a/configs/README.md b/configs/README.md index eaacb493f..8714895cc 100644 --- a/configs/README.md +++ b/configs/README.md @@ -27,6 +27,7 @@ Refer to the tables below for descriptions of all fields in the configuration fi |data-order| array[string] | **REQUIRED** input data order. Data order: *C* (row-major, default) or *F* (column-major) | |dtype| array[string] | **REQUIRED** input data type. Data type: *float64* (default) or *float32* | |check-finitness| array[] | Check finiteness in sklearn input check(disabled by default) | +|device| array[string] | For scikit-learn only. List of devices to run with sycl context. It can be *None* (without context, default), *cpu*, *gpu* or *host*| ### Case Object diff --git a/configs/skl_with_context_config.json b/configs/skl_xpu_config.json similarity index 100% rename from configs/skl_with_context_config.json rename to configs/skl_xpu_config.json From 53b4200ec8106913f5f86846d2f6653ee79aecd3 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Wed, 14 Apr 2021 01:12:09 +0300 Subject: [PATCH 02/10] Update GPU readme --- README.md | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index c76de2b57..45ed327ab 100755 --- a/README.md +++ b/README.md @@ -28,7 +28,8 @@ We publish blogs on Medium, so [follow us](https://medium.com/intel-analytics-so * [How to create conda environment for benchmarking](#how-to-create-conda-environment-for-benchmarking) * [Running Python benchmarks with runner script](#running-python-benchmarks-with-runner-script) -* [Supported algorithms](#supported-algorithms) +* [Benchmark supported algorithms](#benchmark-supported-algorithms) +* [Intel(R) Extension for Scikit-learn* support](#Intel(R)-Extension-for-Scikit-learn*-support) * [Algorithms parameters](#algorithms-parameters) ## How to create conda environment for benchmarking @@ -89,22 +90,25 @@ The configuration of benchmarks allows you to select the frameworks to run, sele ## Benchmark supported algorithms -| algorithm | benchmark name | sklearn | sklearn on GPU | daal4py | cuml | xgboost | -|---|---|---|---|---|---|---| -|**[DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html)**|dbscan|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)**|df_clfs|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| -|**[RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html)**|df_regr|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| -|**[pairwise_distances](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html)**|distances|:white_check_mark:|:x:|:white_check_mark:|:x:|:x:| -|**[KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html)**|kmeans|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[KNeighborsClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)**|knn_clsf|:white_check_mark:|:x:|:x:|:white_check_mark:|:x:| -|**[LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)**|linear|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)**|log_reg|:white_check_mark:|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| -|**[PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html)**|pca|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| -|**[Ridge](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html)**|ridge|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| -|**[SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)**|svm|:white_check_mark:|:x:|:white_check_mark:|:white_check_mark:|:x:| -|**[train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)**|train_test_split|:white_check_mark:|:x:|:x:|:white_check_mark:|:x:| -|**[GradientBoostingClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)**|gbt|:x:|:x:|:x:|:x:|:white_check_mark:| -|**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:x:|:white_check_mark:| +| algorithm | benchmark name | sklearn | daal4py | cuml | xgboost | +|---|---|---|---|---|---| +|**[DBSCAN](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html)**|dbscan|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)**|df_clfs|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html)**|df_regr|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[pairwise_distances](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html)**|distances|:white_check_mark:|:white_check_mark:|:x:|:x:| +|**[KMeans](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html)**|kmeans|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[KNeighborsClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html)**|knn_clsf|:white_check_mark:|:x:|:white_check_mark:|:x:| +|**[LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)**|linear|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)**|log_reg|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html)**|pca|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[Ridge](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html)**|ridge|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)**|svm|:white_check_mark:|:white_check_mark:|:white_check_mark:|:x:| +|**[train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)**|train_test_split|:white_check_mark:|:x:|:white_check_mark:|:x:| +|**[GradientBoostingClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| +|**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| + +## Intel(R) Extension for Scikit-learn* support +[Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) provides the ability to run scikit-learn on CPU and GPU with kernels optimized by [oneDAL](https://github.com/oneapi-src/oneDAL). The extension support GPU patching for algorihms: **DBSCAN**, **KMeans**, **LinearRegression**, **LogisticRegression**. You can launch benchmarks with GPU support using [skl_xpu_config.json](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_with_context_config.json) configuration file. ## Algorithms parameters From cbd99b1174424816565c6a36754e8679db9d53c2 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Wed, 14 Apr 2021 01:27:17 +0300 Subject: [PATCH 03/10] Add hyperlink --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 45ed327ab..5e58b098e 100755 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul runner options: * ``configs`` : configuration files paths -* ``no-intel-optimized`` : using Scikit-learn without Intel(R) Extension for Scikit-learn*. Now avalible for scikit-learn benchmarks. Default starts with using Intel(R) Extension for Scikit-learn*. +* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#Intel(R)-Extension-for-Scikit-learn*-support). Now avalible for scikit-learn benchmarks. Default starts with using Intel(R) Extension for Scikit-learn*. * ``output-file``: output file name for result benchmarks. Default is `result.json` * ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`. * ``dummy-run`` : run configuration parser and datasets generation without benchmarks running. From becfbde1babc67d0cb06b1e829f681c1dbc53c1b Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Wed, 14 Apr 2021 11:12:15 +0300 Subject: [PATCH 04/10] Fix comments --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 5e58b098e..a14b42c2b 100755 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ We publish blogs on Medium, so [follow us](https://medium.com/intel-analytics-so * [How to create conda environment for benchmarking](#how-to-create-conda-environment-for-benchmarking) * [Running Python benchmarks with runner script](#running-python-benchmarks-with-runner-script) * [Benchmark supported algorithms](#benchmark-supported-algorithms) -* [Intel(R) Extension for Scikit-learn* support](#Intel(R)-Extension-for-Scikit-learn*-support) +* [Intel(R) Extension for Scikit-learn* support](#intel(r)-extension-for-scikit-learn*-support) * [Algorithms parameters](#algorithms-parameters) ## How to create conda environment for benchmarking @@ -66,7 +66,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul runner options: * ``configs`` : configuration files paths -* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#Intel(R)-Extension-for-Scikit-learn*-support). Now avalible for scikit-learn benchmarks. Default starts with using Intel(R) Extension for Scikit-learn*. +* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intel(r)-extension-for-scikit-learn*-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench). Default running with using Intel(R) Extension for Scikit-learn*. * ``output-file``: output file name for result benchmarks. Default is `result.json` * ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`. * ``dummy-run`` : run configuration parser and datasets generation without benchmarks running. @@ -108,7 +108,9 @@ The configuration of benchmarks allows you to select the frameworks to run, sele |**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| ## Intel(R) Extension for Scikit-learn* support -[Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) provides the ability to run scikit-learn on CPU and GPU with kernels optimized by [oneDAL](https://github.com/oneapi-src/oneDAL). The extension support GPU patching for algorihms: **DBSCAN**, **KMeans**, **LinearRegression**, **LogisticRegression**. You can launch benchmarks with GPU support using [skl_xpu_config.json](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_with_context_config.json) configuration file. +[Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) providing drop-in patching speeds up scikit-learn on CPU and GPU. + +Scikit-learn benchmarks on GPU with Intel(R) Extension for Scikit-learn support algorithms: **DBSCAN**, **KMeans**, **LinearRegression**, **LogisticRegression**. Example config with GPU support [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). ## Algorithms parameters From 98749580174bc543b5f2b99a19862b719777a083 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Wed, 14 Apr 2021 23:48:54 +0300 Subject: [PATCH 05/10] Update readme --- README.md | 14 +++++++++++--- configs/README.md | 2 +- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a14b42c2b..c47946fcb 100755 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul runner options: * ``configs`` : configuration files paths -* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intel(r)-extension-for-scikit-learn*-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench). Default running with using Intel(R) Extension for Scikit-learn*. +* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](##intel(r)-extension-for-scikit-learn*-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench). Default running with using Intel(R) Extension for Scikit-learn*. * ``output-file``: output file name for result benchmarks. Default is `result.json` * ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`. * ``dummy-run`` : run configuration parser and datasets generation without benchmarks running. @@ -108,9 +108,17 @@ The configuration of benchmarks allows you to select the frameworks to run, sele |**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| ## Intel(R) Extension for Scikit-learn* support -[Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) providing drop-in patching speeds up scikit-learn on CPU and GPU. -Scikit-learn benchmarks on GPU with Intel(R) Extension for Scikit-learn support algorithms: **DBSCAN**, **KMeans**, **LinearRegression**, **LogisticRegression**. Example config with GPU support [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). +Every scikit-learn benchmark launched with default parameter values uses [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU. Some benchmarks have a GPU support: + +* **dbscan** +* **kmeans** +* **linear** +* **log_reg** + +A configuration file that contains all these benchmarks can be found [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). You can use the following command to run these benchmarks on both CPU and GPU. + + python runner.py --configs configs/skl_xpu_config.json ## Algorithms parameters diff --git a/configs/README.md b/configs/README.md index 8714895cc..48f0a0a2b 100644 --- a/configs/README.md +++ b/configs/README.md @@ -27,7 +27,7 @@ Refer to the tables below for descriptions of all fields in the configuration fi |data-order| array[string] | **REQUIRED** input data order. Data order: *C* (row-major, default) or *F* (column-major) | |dtype| array[string] | **REQUIRED** input data type. Data type: *float64* (default) or *float32* | |check-finitness| array[] | Check finiteness in sklearn input check(disabled by default) | -|device| array[string] | For scikit-learn only. List of devices to run with sycl context. It can be *None* (without context, default), *cpu*, *gpu* or *host*| +|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please reffer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details| ### Case Object From 0fe0ab4aed60fe7de5b836fd57b4eedd8ba4a17b Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Wed, 14 Apr 2021 23:50:38 +0300 Subject: [PATCH 06/10] Fix hyperlink --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c47946fcb..46a011b2d 100755 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul runner options: * ``configs`` : configuration files paths -* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](##intel(r)-extension-for-scikit-learn*-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench). Default running with using Intel(R) Extension for Scikit-learn*. +* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench). Default running with using Intel(R) Extension for Scikit-learn*. * ``output-file``: output file name for result benchmarks. Default is `result.json` * ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`. * ``dummy-run`` : run configuration parser and datasets generation without benchmarks running. From 9ef18ab91ce9d42e2910d2d073181aaffb4e321d Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Thu, 15 Apr 2021 00:47:22 +0300 Subject: [PATCH 07/10] Update comments --- README.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 46a011b2d..9bb3dce59 100755 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul runner options: * ``configs`` : configuration files paths -* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench). Default running with using Intel(R) Extension for Scikit-learn*. +* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). Default running with using Intel(R) Extension for Scikit-learn. * ``output-file``: output file name for result benchmarks. Default is `result.json` * ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`. * ``dummy-run`` : run configuration parser and datasets generation without benchmarks running. @@ -107,18 +107,16 @@ The configuration of benchmarks allows you to select the frameworks to run, sele |**[GradientBoostingClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| |**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:white_check_mark:| -## Intel(R) Extension for Scikit-learn* support +## Intel(R) Extension for Scikit-learn support -Every scikit-learn benchmark launched with default parameter values uses [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU. Some benchmarks have a GPU support: - -* **dbscan** -* **kmeans** -* **linear** -* **log_reg** +By default scikit-learn benchmark launches using [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU (use ``no-intel-optimized`` option to run without extention). Some benchmarks have a GPU support: -A configuration file that contains all these benchmarks can be found [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). You can use the following command to run these benchmarks on both CPU and GPU. +* dbscan +* kmeans +* linear +* log_reg - python runner.py --configs configs/skl_xpu_config.json +A configuration file that contains all these benchmarks can be found [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). You can use this file to run these benchmarks on both CPU and GPU. ## Algorithms parameters From ece84d9b76b134f7e053e93dbaba65bb9d0cb2d0 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Thu, 15 Apr 2021 00:52:15 +0300 Subject: [PATCH 08/10] Update link --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9bb3dce59..5efd4589e 100755 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ We publish blogs on Medium, so [follow us](https://medium.com/intel-analytics-so * [How to create conda environment for benchmarking](#how-to-create-conda-environment-for-benchmarking) * [Running Python benchmarks with runner script](#running-python-benchmarks-with-runner-script) * [Benchmark supported algorithms](#benchmark-supported-algorithms) -* [Intel(R) Extension for Scikit-learn* support](#intel(r)-extension-for-scikit-learn*-support) +* [Intel(R) Extension for Scikit-learn* support](#intelr-extension-for-scikit-learn-support) * [Algorithms parameters](#algorithms-parameters) ## How to create conda environment for benchmarking From 6568e48484679540ade7a4014e0d51789f039167 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Thu, 15 Apr 2021 13:09:46 +0300 Subject: [PATCH 09/10] Fix comments --- README.md | 2 +- configs/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5efd4589e..30b1dab86 100755 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ The configuration of benchmarks allows you to select the frameworks to run, sele ## Intel(R) Extension for Scikit-learn support -By default scikit-learn benchmark launches using [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU (use ``no-intel-optimized`` option to run without extention). Some benchmarks have a GPU support: +The runs of Scikit-learn benchmark use [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU by default (use ``no-intel-optimized`` option to run without extention). Some benchmarks have a GPU support: * dbscan * kmeans diff --git a/configs/README.md b/configs/README.md index 48f0a0a2b..e19b1d49b 100644 --- a/configs/README.md +++ b/configs/README.md @@ -27,7 +27,7 @@ Refer to the tables below for descriptions of all fields in the configuration fi |data-order| array[string] | **REQUIRED** input data order. Data order: *C* (row-major, default) or *F* (column-major) | |dtype| array[string] | **REQUIRED** input data type. Data type: *float64* (default) or *float32* | |check-finitness| array[] | Check finiteness in sklearn input check(disabled by default) | -|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please reffer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details| +|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please refer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details| ### Case Object From 9f133ef548542254a88097930f3f2ea4d0b962c7 Mon Sep 17 00:00:00 2001 From: Vladislav Nazarov Date: Thu, 15 Apr 2021 14:49:37 +0300 Subject: [PATCH 10/10] Apply comments --- README.md | 19 ++++++++++--------- configs/README.md | 2 +- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 30b1dab86..d2f2ebe34 100755 --- a/README.md +++ b/README.md @@ -64,13 +64,13 @@ conda create -n bench -c conda-forge python=3.7 xgboost pandas Run `python runner.py --configs configs/config_example.json [--output-file result.json --verbose INFO --report]` to launch benchmarks. -runner options: -* ``configs`` : configuration files paths -* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). Default running with using Intel(R) Extension for Scikit-learn. -* ``output-file``: output file name for result benchmarks. Default is `result.json` -* ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`. -* ``dummy-run`` : run configuration parser and datasets generation without benchmarks running. -* ``verbose`` : *WARNING*, *INFO*, *DEBUG*. print additional information during benchmarks running. Default is *INFO* +Options: +* ``--configs``: specify the path to a configuration file. +* ``--no-intel-optimized``: use Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). By default, the runner uses Intel(R) Extension for Scikit-learn. +* ``--output-file``: output file name for the benchmark result. The default name is `result.json` +* ``--report``: create an Excel report based on benchmark results. The `openpyxl` library is required. +* ``--dummy-run``: run configuration parser and dataset generation without benchmarks running. +* ``--verbose``: *WARNING*, *INFO*, *DEBUG*. print additional information during benchmarks running. Default is *INFO*. | Level | Description | |-----------|---------------| @@ -109,14 +109,15 @@ The configuration of benchmarks allows you to select the frameworks to run, sele ## Intel(R) Extension for Scikit-learn support -The runs of Scikit-learn benchmark use [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU by default (use ``no-intel-optimized`` option to run without extention). Some benchmarks have a GPU support: +When you run scikit-learn benchmarks on CPU, [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) is used by default. Use the ``--no-intel-optimized`` option to run the benchmarks without the extension. +The following benchmarks have a GPU support: * dbscan * kmeans * linear * log_reg -A configuration file that contains all these benchmarks can be found [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). You can use this file to run these benchmarks on both CPU and GPU. +You may use the [configuration file for these benchmarks](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json) to run them on both CPU and GPU. ## Algorithms parameters diff --git a/configs/README.md b/configs/README.md index e19b1d49b..44ce2ae21 100644 --- a/configs/README.md +++ b/configs/README.md @@ -27,7 +27,7 @@ Refer to the tables below for descriptions of all fields in the configuration fi |data-order| array[string] | **REQUIRED** input data order. Data order: *C* (row-major, default) or *F* (column-major) | |dtype| array[string] | **REQUIRED** input data type. Data type: *float64* (default) or *float32* | |check-finitness| array[] | Check finiteness in sklearn input check(disabled by default) | -|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please refer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details| +|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Refer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details| ### Case Object