Aarch64 support

[NicolasHug]

Opening this issue to track requests for support of aarch64 (Linux ARM).

I think our build/test/release infra should be able to support those fairly naturally. We'd need to build and push more non-GPL FFmpeg binaries as well.

[imstevenpmwork]

We, at the lerobot OSS project from Hugging Face, recently decided to use torchcode as our default codec. However, some of our 'deployed' robots are running on aarch64, so the installation is currently broken for some users. To address this, we’ve implemented a platform check: if torchcode is available, we use it; otherwise, we gracefully fall back to another one.
Happy to test it once it is packaged for aarch64 🙂
Here are the logs:

(lerobot) raspberrypi@raspberrypi:~/lerobot $ uname -a
Linux raspberrypi 6.6.74+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.74-1+rpt1 (2025-01-27) aarch64 GNU/Linux
(lerobot) raspberrypi@raspberrypi:~/lerobot $ pip install torchcodec==0.2.1
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: Could not find a version that satisfies the requirement torchcodec==0.2.1 (from versions: 0.0.0.dev0)
ERROR: No matching distribution found for torchcodec==0.2.1

[traversaro]

While packaging torchcodec on conda-forge, it was trivial to also add the aarch64 build. However, it seems that both C++ and Python tests are failing on linux-aarch64 . I initially tought this may be a problem of conda-forge's pytorch package on linux-aarch64, but I was able to reproduce the same failures even linking PyTorch installed via PyPI.

Log of tests failures: https://github.com/traversaro/torchcodec-test-debug/actions/runs/14695881468/job/41237423866 .

Output C++ test failing on linux-aarch64

 Internal ctest changing into directory: /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi
Test project /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi
      Start  1: SingleStreamDecoderTest.MissingVideoFileThrowsException
 1/23 Test  #1: SingleStreamDecoderTest.MissingVideoFileThrowsException ...................................   Passed    0.31 sec
      Start  2: SingleStreamDecoderTest.RespectsWidthAndHeightFromOptions
 2/23 Test  #2: SingleStreamDecoderTest.RespectsWidthAndHeightFromOptions .................................   Passed    0.32 sec
      Start  3: SingleStreamDecoderTest.RespectsOutputTensorDimensionOrderFromOptions
 3/23 Test  #3: SingleStreamDecoderTest.RespectsOutputTensorDimensionOrderFromOptions .....................   Passed    0.32 sec
      Start  4: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFpsAndDurationForVideoInMetadata/false
 4/23 Test  #4: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFpsAndDurationForVideoInMetadata/false ...   Passed    0.31 sec
      Start  5: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFpsAndDurationForVideoInMetadata/true
 5/23 Test  #5: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFpsAndDurationForVideoInMetadata/true ....   Passed    0.31 sec
      Start  6: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/false
 6/23 Test  #6: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/false ..............***Failed    0.32 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/0
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/0
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:190: Failure
Value of: torch::equal(tensor0FromOurDecoder, tensor0FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:191: Failure
Value of: torch::equal(tensor1FromOurDecoder, tensor1FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/0, where GetParam() = false (17 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (17 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (17 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/0, where GetParam() = false

 1 FAILED TEST

      Start  7: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/true
 7/23 Test  #7: FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/true ...............***Failed    0.33 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/1
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/1
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:190: Failure
Value of: torch::equal(tensor0FromOurDecoder, tensor0FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:191: Failure
Value of: torch::equal(tensor1FromOurDecoder, tensor1FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/1, where GetParam() = true (19 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (19 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (19 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/1, where GetParam() = true

 1 FAILED TEST

      Start  8: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/false
 8/23 Test  #8: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/false ...............***Failed    0.34 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/0
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/0
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:224: Failure
Value of: torch::equal(tensor[0], tensor0FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:225: Failure
Value of: torch::equal(tensor[1], tensorTime6FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/0, where GetParam() = false (30 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (30 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (30 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/0, where GetParam() = false

 1 FAILED TEST

      Start  9: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/true
 9/23 Test  #9: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/true ................***Failed    0.35 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/1
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/1
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:224: Failure
Value of: torch::equal(tensor[0], tensor0FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:225: Failure
Value of: torch::equal(tensor[1], tensorTime6FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/1, where GetParam() = true (31 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (31 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (31 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/1, where GetParam() = true

 1 FAILED TEST

      Start 10: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/false
10/23 Test #10: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/false ...............***Failed    0.34 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/0
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/0
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:249: Failure
Value of: torch::equal(tensor[0], tensor0FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:250: Failure
Value of: torch::equal(tensor[1], tensorTime6FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/0, where GetParam() = false (29 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (29 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (29 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/0, where GetParam() = false

 1 FAILED TEST

      Start 11: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/true
11/23 Test #11: FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/true ................***Failed    0.33 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/1
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/1
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:249: Failure
Value of: torch::equal(tensor[0], tensor0FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:250: Failure
Value of: torch::equal(tensor[1], tensorTime6FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/1, where GetParam() = true (29 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (29 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (29 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/1, where GetParam() = true

 1 FAILED TEST

      Start 12: FromFileAndMemory/SingleStreamDecoderTest.SeeksCloseToEof/false
12/23 Test #12: FromFileAndMemory/SingleStreamDecoderTest.SeeksCloseToEof/false ...........................   Passed    0.33 sec
      Start 13: FromFileAndMemory/SingleStreamDecoderTest.SeeksCloseToEof/true
13/23 Test #13: FromFileAndMemory/SingleStreamDecoderTest.SeeksCloseToEof/true ............................   Passed    0.33 sec
      Start 14: FromFileAndMemory/SingleStreamDecoderTest.GetsFramePlayedAtTimestamp/false
14/23 Test #14: FromFileAndMemory/SingleStreamDecoderTest.GetsFramePlayedAtTimestamp/false ................   Passed    0.37 sec
      Start 15: FromFileAndMemory/SingleStreamDecoderTest.GetsFramePlayedAtTimestamp/true
15/23 Test #15: FromFileAndMemory/SingleStreamDecoderTest.GetsFramePlayedAtTimestamp/true .................   Passed    0.38 sec
      Start 16: FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/false
16/23 Test #16: FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/false ...............***Failed    0.43 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/0
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/0
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:307: Failure
Value of: torch::equal(tensor6FromOurDecoder, tensor6FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:323: Failure
Value of: torch::equal(tensor61FromOurDecoder, tensor61FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:343: Failure
Value of: torch::equal(tensor10FromOurDecoder, tensor10FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:358: Failure
Value of: torch::equal(tensor6FromOurDecoder, tensor6FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:375: Failure
Value of: torch::equal(tensor7FromOurDecoder, tensor7FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/0, where GetParam() = false (118 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (118 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (118 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/0, where GetParam() = false

 1 FAILED TEST

      Start 17: FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/true
17/23 Test #17: FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/true ................***Failed    0.45 sec
Running main() from /home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/buildpypi/_deps/googletest-src/googletest/src/gtest_main.cc
Note: Google Test filter = FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/1
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest
[ RUN      ] FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/1
/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:307: Failure
Value of: torch::equal(tensor6FromOurDecoder, tensor6FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:323: Failure
Value of: torch::equal(tensor61FromOurDecoder, tensor61FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:343: Failure
Value of: torch::equal(tensor10FromOurDecoder, tensor10FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:358: Failure
Value of: torch::equal(tensor6FromOurDecoder, tensor6FromFFMPEG)
  Actual: false
Expected: true

/home/runner/work/torchcodec-test-debug/torchcodec-test-debug/.pixi_ws/torchcodec/test/VideoDecoderTest.cpp:375: Failure
Value of: torch::equal(tensor7FromOurDecoder, tensor7FromFFMPEG)
  Actual: false
Expected: true

[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/1, where GetParam() = true (137 ms)
[----------] 1 test from FromFileAndMemory/SingleStreamDecoderTest (137 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (137 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/1, where GetParam() = true

 1 FAILED TEST

      Start 18: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorFilterGraph/false
18/23 Test #18: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorFilterGraph/false .............   Passed    0.36 sec
      Start 19: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorFilterGraph/true
19/23 Test #19: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorFilterGraph/true ..............   Passed    0.32 sec
      Start 20: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorSwscale/false
20/23 Test #20: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorSwscale/false .................   Passed    0.32 sec
      Start 21: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorSwscale/true
21/23 Test #21: FromFileAndMemory/SingleStreamDecoderTest.PreAllocatedTensorSwscale/true ..................   Passed    0.31 sec
      Start 22: FromFileAndMemory/SingleStreamDecoderTest.GetAudioMetadata/false
22/23 Test #22: FromFileAndMemory/SingleStreamDecoderTest.GetAudioMetadata/false ..........................   Passed    0.30 sec
      Start 23: FromFileAndMemory/SingleStreamDecoderTest.GetAudioMetadata/true
23/23 Test #23: FromFileAndMemory/SingleStreamDecoderTest.GetAudioMetadata/true ...........................   Passed    0.30 sec

65% tests passed, 8 tests failed out of 23

Total Test time (real) =   8.07 sec

The following tests FAILED:
Errors while running CTest
	  6 - FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/false (Failed)
	  7 - FromFileAndMemory/SingleStreamDecoderTest.ReturnsFirstTwoFramesOfVideo/true (Failed)
	  8 - FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/false (Failed)
	  9 - FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNCHW/true (Failed)
	 10 - FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/false (Failed)
	 11 - FromFileAndMemory/SingleStreamDecoderTest.DecodesFramesInABatchInNHWC/true (Failed)
	 16 - FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/false (Failed)
	 17 - FromFileAndMemory/SingleStreamDecoderTest.SeeksToFrameWithSpecificPts/true (Failed)
Error: Process completed with exit code 8.

[NicolasHug]

Thank you for the logs @traversaro . It looks like most of the errors look like

Mismatched elements: 170001 / 388800 (43.7%)
Greatest absolute difference: 3 at index (0, 27, 274)
Greatest relative difference: inf at index (0, 0, 104)
FAILED decoders/test_decoders.py::TestVideoDecoder::test_getitem_int[exact-cpu-4] - AssertionError: Tensor-likes are not equal!

which don't seem too bad. We do exact checks against linux x86, but we do have to relax the tolerances on MacOS and on CUDA already:

torchcodec/test/utils.py

Lines 38 to 51 in ed13ac5

	def assert_frames_equal(*args, **kwargs):
	if sys.platform == "linux":
	if args[0].device.type == "cuda":
	atol = 2
	if get_ffmpeg_major_version() == 4:
	assert_tensor_close_on_at_least(
	args[0], args[1], percentage=95, atol=atol
	)
	else:
	torch.testing.assert_close(*args, **kwargs, atol=atol, rtol=0)
	else:
	torch.testing.assert_close(*args, **kwargs, atol=0, rtol=0)
	else:
	torch.testing.assert_close(*args, **kwargs, atol=3, rtol=0)

So relaxing the tolerances on aarch64 seems reasonable. I'm not sure how to best do that robustly though, especially if those new architectures are supported out-of-core. This is something we may have to enable soon for XPU support as well: CC @scotts @dvrogozh for ideas?

[traversaro]

Thanks, that is clear! So if I understood correctly the C++ tests are only expected to pass on Linux on x86_64?

[NicolasHug]

They are ideally meant to pass on all platforms, but if they don't, it's safe to disable them. Most of the C++ tests are legacy at this point anyway. As long as the Python tests pass (possibly slightly increasing the tolerance), an architecture can be considered safe to release.

[scotts]

@traversaro, thanks for digging into this! Did you need to make any changes outside of testing?

[traversaro]

@traversaro, thanks for digging into this! Did you need to make any changes outside of testing?

No, with just the test-related patch suggested by @NicolasHug (https://github.com/conda-forge/torchcodec-feedstock/blob/fe701d8f1f535381f4779310bc158584b8420c7c/recipe/use_strict_threshold_only_for_linux_x86_64.patch) all the Python test run fine on CI on linux-aarch64: conda-forge/torchcodec-feedstock#12 .

[dvrogozh]

So relaxing the tolerances on aarch64 seems reasonable. I'm not sure how to best do that robustly though<...>: CC @scotts @dvrogozh for ideas?

Just relaxing the tolerance won't work actually. Problem is with color conversion algorithms which might significantly differ per implementation and per-platform. You can't expect bit-to-bit matches. And that's not something you can overcome with the current threshold which expects difference in no more than N pixels. There could be bigger substantial differences. I did step into that working on XPU support for torchcodec.

For the media domain that's known problem. It's being solved by comparing frames by metrics such as PSNR, SSIM, etc. That was the part of my initial #558, see change in test/utils.py:

        elif args[0].device.type == "xpu":
            if not torch.allclose(*args, atol=0, rtol=0):
                from torcheval.metrics import PeakSignalNoiseRatio

                metric = PeakSignalNoiseRatio()
                metric.update(args[0], args[1])
                assert metric.compute() >= 40

As you can see I did that for XPU, but we can reuse the idea and code for other cases. I will extract this to separate PR today, but I will need help to test this on aarch64 since I don't have this platform at hand.