Skip to content

Rename "xptests" back to "array_api_tests" #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 10, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions .github/workflows/numpy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,30 +23,30 @@ jobs:
python -m pip install -r requirements.txt
- name: Run the test suite
env:
XPTESTS_MODULE: numpy.array_api
ARRAY_API_TESTS_MODULE: numpy.array_api
run: |
# Mark some known issues as XFAIL
cat << EOF >> xfails.txt

# https://github.com/numpy/numpy/issues/18881
xptests/test_creation_functions.py::test_linspace
array_api_tests/test_creation_functions.py::test_linspace
# einsum is not yet completed in the spec
xptests/test_signatures.py::test_has_names[einsum]
array_api_tests/test_signatures.py::test_has_names[einsum]
# dlpack support is not yet implemented in NumPy
# See https://github.com/numpy/numpy/pull/19083
xptests/test_signatures.py::test_function_positional_args[__dlpack__]
xptests/test_signatures.py::test_function_positional_args[__dlpack_device__]
xptests/test_signatures.py::test_function_positional_args[from_dlpack]
xptests/test_signatures.py::test_function_positional_args[to_device]
xptests/test_signatures.py::test_function_keyword_only_args[__dlpack__]
array_api_tests/test_signatures.py::test_function_positional_args[__dlpack__]
array_api_tests/test_signatures.py::test_function_positional_args[__dlpack_device__]
array_api_tests/test_signatures.py::test_function_positional_args[from_dlpack]
array_api_tests/test_signatures.py::test_function_positional_args[to_device]
array_api_tests/test_signatures.py::test_function_keyword_only_args[__dlpack__]
# floor_divide has an issue related to https://github.com/data-apis/array-api/issues/264
xptests/test_elementwise_functions.py::test_floor_divide
array_api_tests/test_elementwise_functions.py::test_floor_divide
# mesgrid doesn't return all arrays as the promoted dtype
xptests/test_type_promotion.py::test_meshgrid
array_api_tests/test_type_promotion.py::test_meshgrid
# https://github.com/numpy/numpy/pull/20066#issuecomment-947056094
xptests/test_type_promotion.py::test_where
array_api_tests/test_type_promotion.py::test_where
# shape mismatches are not handled
xptests/test_type_promotion.py::test_tensordot
array_api_tests/test_type_promotion.py::test_tensordot

EOF

Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ Ensure you have the array library that you want to test installed.
### Specifying the array module

You need to specify the array library to test. It can be specified via the
`XPTESTS_MODULE` environment variable, e.g.
`ARRAY_API_TESTS_MODULE` environment variable, e.g.

```bash
$ export XPTESTS_MODULE=numpy.array_api
$ export ARRAY_API_TESTS_MODULE=numpy.array_api
```

Alternately, change the `array_module` variable in `xptests/_array_module.py`
Alternately, change the `array_module` variable in `array_api_tests/_array_module.py`
line, e.g.

```diff
Expand All @@ -37,17 +37,17 @@ line, e.g.

### Run the suite

Simply run `pytest` against the `xptests/` folder to run the full suite.
Simply run `pytest` against the `array_api_tests/` folder to run the full suite.

```bash
$ pytest xptests/
$ pytest array_api_tests/
```

The suite tries to logically organise its tests. `pytest` allows you to only run
a specific test case, which is useful when developing functions.

```bash
$ pytest xptests/test_creation_functions.py::test_zeros
$ pytest array_api_tests/test_creation_functions.py::test_zeros
```

## What the test suite covers
Expand Down Expand Up @@ -104,7 +104,7 @@ of the functions and some miscellaneous things.
[constants](https://data-apis.org/array-api/latest/API_specification/constants.html)
behave expectedly, are roughly the expected value, and that any related
functions interact with them correctly.

Be aware that some aspects of the spec are impractical or impossible to actually
test, so they are not covered in the suite <!-- TODO: note what these are -->

Expand Down
File renamed without changes.
6 changes: 3 additions & 3 deletions xptests/_array_module.py → array_api_tests/_array_module.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
array_module = None

if array_module is None:
if 'XPTESTS_MODULE' in os.environ:
mod_name = os.environ['XPTESTS_MODULE']
if 'ARRAY_API_TESTS_MODULE' in os.environ:
mod_name = os.environ['ARRAY_API_TESTS_MODULE']
_module, _sub = mod_name, None
if '.' in mod_name:
_module, _sub = mod_name.split('.', 1)
Expand All @@ -24,7 +24,7 @@
# submodules that can be imported (like mxnet.nd).
mod = import_module(mod_name)
else:
raise RuntimeError("No array module specified. Either edit _array_module.py or set the XPTESTS_MODULE environment variable")
raise RuntimeError("No array module specified. Either edit _array_module.py or set the ARRAY_API_TESTS_MODULE environment variable")
else:
mod = array_module
mod_name = mod.__name__
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
from hypothesis import strategies as st
from hypothesis.control import assume

from xptests.typing import Scalar, ScalarType, Shape
from .typing import Scalar, ScalarType, Shape
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An aside: we need to avoid absolute imports like this, so that the module can easily be vendored. Maybe we should add a simple meta-test for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep this was just an unintentional mistake


from . import _array_module as xp
from . import dtype_helpers as dh
Expand Down
File renamed without changes.
File renamed without changes.
6 changes: 3 additions & 3 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
from pytest import mark
from hypothesis import settings

from xptests import _array_module as xp
from xptests._array_module import _UndefinedStub
from array_api_tests import _array_module as xp
from array_api_tests._array_module import _UndefinedStub


settings.register_profile('xp_default', deadline=800)
Expand Down Expand Up @@ -77,7 +77,7 @@ def xp_has_ext(ext: str) -> bool:
if xfails_path.exists():
with open(xfails_path) as f:
for line in f:
if line.startswith('xptests'):
if line.startswith('array_api_tests'):
id_ = line.strip('\n')
xfail_ids.append(id_)

Expand Down
16 changes: 8 additions & 8 deletions generate_stubs.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

./generate_stubs.py path/to/clone/of/array-api

This will update the stub files in xptests/function_stubs/
This will update the stub files in array_api_tests/function_stubs/
"""
from __future__ import annotations

Expand Down Expand Up @@ -218,12 +218,12 @@ def main():
parser.add_argument('-v', '--verbose', help="Print verbose output to the terminal", action='store_true')
args = parser.parse_args()

types_path = os.path.join('xptests', 'function_stubs', '_types.py')
types_path = os.path.join('array_api_tests', 'function_stubs', '_types.py')
if args.write:
with open(types_path, 'w') as f:
f.write(TYPES_HEADER)

special_cases_dir = Path('xptests/special_cases')
special_cases_dir = Path('array_api_tests/special_cases')
special_cases_dir.mkdir(exist_ok=True)
(special_cases_dir / '__init__.py').touch()

Expand Down Expand Up @@ -256,7 +256,7 @@ def main():
title += " (Extension)"
else:
py_file = filename.replace('.md', '.py')
py_path = os.path.join('xptests', 'function_stubs', py_file)
py_path = os.path.join('array_api_tests', 'function_stubs', py_file)
module_name = py_file.replace('.py', '')
modules[module_name] = []
if args.verbose:
Expand Down Expand Up @@ -342,7 +342,7 @@ def {annotated_sig}:{doc}
if filename == 'elementwise_functions.md':
special_cases = parse_special_cases(text, verbose=args.verbose)
for func in special_cases:
py_path = os.path.join('xptests', 'special_cases', f'test_{func}.py')
py_path = os.path.join('array_api_tests', 'special_cases', f'test_{func}.py')
tests = make_special_case_tests(func, special_cases, sigs)
if tests:
code = SPECIAL_CASES_HEADER.format(func=func) + '\n'.join(tests)
Expand All @@ -354,7 +354,7 @@ def {annotated_sig}:{doc}
elif filename == 'array_object.md':
op_special_cases = parse_special_cases(text, verbose=args.verbose)
for func in op_special_cases:
py_path = os.path.join('xptests', 'special_cases', f'test_dunder_{func[2:-2]}.py')
py_path = os.path.join('array_api_tests', 'special_cases', f'test_dunder_{func[2:-2]}.py')
tests = make_special_case_tests(func, op_special_cases, sigs)
if tests:
code = OP_SPECIAL_CASES_HEADER.format(func=func) + '\n'.join(tests)
Expand All @@ -368,7 +368,7 @@ def {annotated_sig}:{doc}
iop = f"__i{name}__"
iop_special_cases[iop] = op_special_cases[op]
for func in iop_special_cases:
py_path = os.path.join('xptests', 'special_cases', f'test_dunder_{func[2:-2]}.py')
py_path = os.path.join('array_api_tests', 'special_cases', f'test_dunder_{func[2:-2]}.py')
tests = make_special_case_tests(func, iop_special_cases, sigs)
if tests:
code = IOP_SPECIAL_CASES_HEADER.format(func=func, operator=func[2:-2]) + '\n'.join(tests)
Expand All @@ -377,7 +377,7 @@ def {annotated_sig}:{doc}
with open(py_path, 'w') as f:
f.write(code)

init_path = os.path.join('xptests', 'function_stubs', '__init__.py')
init_path = os.path.join('array_api_tests', 'function_stubs', '__init__.py')
if args.write:
with open(init_path, 'w') as f:
f.write(INIT_HEADER)
Expand Down