Skip to content

Commit 5e00701

Browse files
committed
fix: readme
1 parent dcd1a3d commit 5e00701

File tree

2 files changed

+8
-141
lines changed

2 files changed

+8
-141
lines changed

benchmarks/README.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,20 +4,16 @@ Benchmarking dpctl using Airspeed Velocity
44
Read more about ASV [here](https://asv.readthedocs.io/en/stable/index.html)
55

66
## Usage
7-
This was made for use within our CI in mind. As such, you will see the `asv.conf.json` is minimal without any environmental information supplied.
7+
The benchmarks were made with using an existing environment in-mind before execution. You will see the `asv.conf.json` is minimal without any environmental information supplied.
88
The expectation is for users to execute `asv run` with an existing environment.
99

10-
As such, you should have conda or mamba installed, and create a `dpctl-benchmarking` environment as so:
11-
```
12-
conda create --name dpctl-benchmarking python=$PYTHON_VERSION dpctl asv libmambapy conda dpcpp_linux-64 --override-channels \
13-
-c https://software.repos.intel.com/python/conda \
14-
-c conda-forge -y
15-
```
10+
As such, you should have conda or mamba installed, and create an environment [following these instructions](https://intelpython.github.io/dpctl/latest/beginners_guides/installation.html#dpctl-installation)
11+
Additionally, install `asv` and `libmambapy` to the environment.
1612

17-
Then, you may activate the environment and instruct `asv run` to use this existing environment for the benchmarks by pointing it to thhe environment's python binary, like so:
13+
Then, you may activate the environment and instruct `asv run` to use this existing environment for the benchmarks by pointing it to the environment's python binary, like so:
1814
```
19-
conda activate dpctl-benchmarking
20-
asv run --environment existing:/full/mamba/path/envs/dpctl-benchmarking/bin/python
15+
conda activate dpctl_env
16+
asv run --environment existing:/full/mamba/path/envs/dpctl_env/bin/python
2117
```
2218

2319
## Writing new benchmarks

benchmarks/asv.conf.json

Lines changed: 2 additions & 131 deletions
Original file line numberDiff line numberDiff line change
@@ -13,24 +13,13 @@
1313
// project being benchmarked
1414
"repo": "..",
1515

16-
// The Python project's subdirectory in your repo. If missing or
17-
// the empty string, the project is assumed to be located at the root
18-
// of the repository.
19-
// "repo_subdir": "",
20-
2116
// Customizable commands for building the project.
2217
// See asv.conf.json documentation.
2318
"build_command": [],
2419

25-
// Customizable commands for installing and uninstalling the project.
26-
// See asv.conf.json documentation.
27-
// "install_command": ["in-dir={env_dir} conda install dpctl --yes"],
28-
// "uninstall_command": ["return-code=any conda uninstall dpctl --yes"],
29-
3020
// List of branches to benchmark. If not provided, defaults to "main"
3121
// (for git) or "default" (for mercurial).
32-
"branches": ["HEAD"], // for git
33-
// "branches": ["default"], // for mercurial
22+
"branches": ["HEAD"],
3423

3524
// The DVCS being used. If not set, it will be automatically
3625
// determined from "repo" by looking at the protocol in the URL
@@ -46,92 +35,6 @@
4635
// variable.
4736
"environment_type": "conda",
4837

49-
// timeout in seconds for installing any dependencies in environment
50-
// defaults to 10 min
51-
//"install_timeout": 600,
52-
53-
// the base URL to show a commit for the project.
54-
// "show_commit_url": "http://github.com/owner/project/commit/",
55-
56-
// The Pythons you'd like to test against. If not provided, defaults
57-
// to the current version of Python used to run `asv`.
58-
// "pythons": ["3.8", "3.12"],
59-
60-
// The list of conda channel names to be searched for benchmark
61-
// dependency packages in the specified order
62-
// "conda_channels": [
63-
// "https://af01p-igk.devtools.intel.com/artifactory/api/conda/idp-conda-pkgserver-igk-local/gold",
64-
// "https://af01p-igk.devtools.intel.com/artifactory/api/conda/idp-conda-pkgserver-igk-local/tools",
65-
// "conda-forge"
66-
// ],
67-
68-
// A conda environment file that is used for environment creation.
69-
// "conda_environment_file": "environment.yml",
70-
71-
// The matrix of dependencies to test. Each key of the "req"
72-
// requirements dictionary is the name of a package (in PyPI) and
73-
// the values are version numbers. An empty list or empty string
74-
// indicates to just test against the default (latest)
75-
// version. null indicates that the package is to not be
76-
// installed. If the package to be tested is only available from
77-
// PyPi, and the 'environment_type' is conda, then you can preface
78-
// the package name by 'pip+', and the package will be installed
79-
// via pip (with all the conda available packages installed first,
80-
// followed by the pip installed packages).
81-
//
82-
// The ``@env`` and ``@env_nobuild`` keys contain the matrix of
83-
// environment variables to pass to build and benchmark commands.
84-
// An environment will be created for every combination of the
85-
// cartesian product of the "@env" variables in this matrix.
86-
// Variables in "@env_nobuild" will be passed to every environment
87-
// during the benchmark phase, but will not trigger creation of
88-
// new environments. A value of ``null`` means that the variable
89-
// will not be set for the current combination.
90-
//
91-
// "matrix": {
92-
// "env": {"PACKAGE_PATH": [""]}
93-
// },
94-
95-
// Combinations of libraries/python versions can be excluded/included
96-
// from the set to test. Each entry is a dictionary containing additional
97-
// key-value pairs to include/exclude.
98-
//
99-
// An exclude entry excludes entries where all values match. The
100-
// values are regexps that should match the whole string.
101-
//
102-
// An include entry adds an environment. Only the packages listed
103-
// are installed. The 'python' key is required. The exclude rules
104-
// do not apply to includes.
105-
//
106-
// In addition to package names, the following keys are available:
107-
//
108-
// - python
109-
// Python version, as in the *pythons* variable above.
110-
// - environment_type
111-
// Environment type, as above.
112-
// - sys_platform
113-
// Platform, as in sys.platform. Possible values for the common
114-
// cases: 'linux2', 'win32', 'cygwin', 'darwin'.
115-
// - req
116-
// Required packages
117-
// - env
118-
// Environment variables
119-
// - env_nobuild
120-
// Non-build environment variables
121-
//
122-
// "exclude": [
123-
// {"python": "3.2", "sys_platform": "win32"}, // skip py3.2 on windows
124-
// {"environment_type": "conda", "req": {"six": null}}, // don't run without six on conda
125-
// {"env": {"ENV_VAR_1": "val2"}}, // skip val2 for ENV_VAR_1
126-
// ],
127-
//
128-
// "include": [
129-
// // additional env for python3.12
130-
// {"python": "3.12", "req": {"numpy": "1.26"}, "env_nobuild": {"FOO": "123"}},
131-
// // additional env if run on windows+conda
132-
// {"platform": "win32", "environment_type": "conda", "python": "3.12", "req": {"libpython": ""}},
133-
// ],
134-
13538
// The directory (relative to the current directory) that benchmarks are
13639
// stored in. If not provided, defaults to "benchmarks"
13740
"benchmark_dir": "benchmarks",
@@ -146,37 +49,5 @@
14649

14750
// The directory (relative to the current directory) that the html tree
14851
// should be written to. If not provided, defaults to "html".
149-
"html_dir": ".asv/html",
150-
151-
// The number of characters to retain in the commit hashes.
152-
// "hash_length": 8,
153-
154-
// `asv` will cache results of the recent builds in each
155-
// environment, making them faster to install next time. This is
156-
// the number of builds to keep, per environment.
157-
// "build_cache_size": 2,
158-
159-
// The commits after which the regression search in `asv publish`
160-
// should start looking for regressions. Dictionary whose keys are
161-
// regexps matching to benchmark names, and values corresponding to
162-
// the commit (exclusive) after which to start looking for
163-
// regressions. The default is to start from the first commit
164-
// with results. If the commit is `null`, regression detection is
165-
// skipped for the matching benchmark.
166-
//
167-
// "regressions_first_commits": {
168-
// "some_benchmark": "352cdf", // Consider regressions only after this commit
169-
// "another_benchmark": null, // Skip regression detection altogether
170-
// },
171-
172-
// The thresholds for relative change in results, after which `asv
173-
// publish` starts reporting regressions. Dictionary of the same
174-
// form as in ``regressions_first_commits``, with values
175-
// indicating the thresholds. If multiple entries match, the
176-
// maximum is taken. If no entry matches, the default is 5%.
177-
//
178-
// "regressions_thresholds": {
179-
// "some_benchmark": 0.01, // Threshold of 1%
180-
// "another_benchmark": 0.5, // Threshold of 50%
181-
// },
52+
"html_dir": ".asv/html"
18253
}

0 commit comments

Comments
 (0)