Skip to content

Trigger JIT tracing&compilation more often in tests #12250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 29 commits into from
Closed
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
ef6db9f
Trigger JIT tracing&compilation more often, enable JIT for ASAN
danog Sep 18, 2023
4e504e1
Revert some changes
danog Sep 20, 2023
2c965c1
Fix
danog Sep 20, 2023
ca53391
Increase JIT buffer size since Psalm get dangerously close to the 16M…
danog Sep 20, 2023
88cf395
Add test for https://github.com/php/php-src/issues/11917
danog Sep 20, 2023
b3a2b3e
Remove jit_prof_threshold
danog Sep 20, 2023
a979682
Run opcache tests with JIT on windows
danog Sep 20, 2023
94714eb
Unify JIT settings and explicitly disable JIT where needed
danog Sep 20, 2023
f397961
Move new JIT test, fixup mac OS builds too
danog Sep 20, 2023
b4ac0e2
Explicitly disable JIT for ASAN builds
danog Sep 20, 2023
8c92c0e
Remove mistakenly added additional 0
danog Sep 20, 2023
8701969
Add phpseclib, psalm, phpstan nightly JIT+ASAN tests
danog Sep 20, 2023
104f03f
Change config in all other CIs
danog Sep 20, 2023
c5af745
Change config in all other CIs
danog Sep 20, 2023
c30b9b3
Fixup config
danog Sep 20, 2023
e11a67a
Improve
danog Sep 20, 2023
665e09c
Improve tests
danog Sep 21, 2023
f822a56
Parallelize
danog Sep 21, 2023
4aa113f
Remove RAM limitations
danog Sep 21, 2023
543b52b
Update
danog Sep 21, 2023
a06eea9
Fix deps
danog Sep 21, 2023
8de7066
Fix deps
danog Sep 21, 2023
1a4d450
Fixup opcache
danog Sep 21, 2023
96f7b74
Reorder
danog Sep 21, 2023
84373f0
fix
danog Sep 21, 2023
f837597
Update to upstream phpseclib
danog Sep 22, 2023
4caf2d6
Improve workflow
danog Sep 22, 2023
12281a4
Remove new tests, move to separate PR
danog Sep 22, 2023
dbd4005
Remove stray line
danog Oct 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .cirrus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ arm_task:
sapi/cli/php run-tests.php
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit_buffer_size=16M
-d opcache.jit_buffer_size=64M
-d opcache.jit=function
-P -q -x -j2
-g FAIL,BORK,LEAK,XLEAK
Expand All @@ -217,7 +217,14 @@ arm_task:
sapi/cli/php run-tests.php
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit_buffer_size=16M
-d opcache.jit_buffer_size=64M
-d opcache.jit_max_root_traces=1000000
-d opcache.jit_max_side_traces=1000000
-d opcache.jit_max_exit_counters=1000000
-d opcache.jit_hot_loop=1
-d opcache.jit_hot_func=1
-d opcache.jit_hot_return=1
-d opcache.jit_hot_side_exit=1
-d opcache.jit=tracing
-P -q -x -j2
-g FAIL,BORK,LEAK,XLEAK
Expand Down
9 changes: 9 additions & 0 deletions .github/actions/test-linux/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,15 @@ runs:
export TEST_PHP_JUNIT=junit.out.xml
export STACK_LIMIT_DEFAULTS_CHECK=1
sapi/cli/php run-tests.php -P -q ${{ inputs.runTestsParameters }} \
-d opcache.protect_memory=1 \
-d opcache.jit_buffer_size=64M \
-d opcache.jit_max_root_traces=1000000 \
-d opcache.jit_max_side_traces=1000000 \
-d opcache.jit_max_exit_counters=1000000 \
-d opcache.jit_hot_loop=1 \
-d opcache.jit_hot_func=1 \
-d opcache.jit_hot_return=1 \
-d opcache.jit_hot_side_exit=1 \
-j$(/usr/bin/nproc) \
-g FAIL,BORK,LEAK,XLEAK \
--no-progress \
Expand Down
9 changes: 9 additions & 0 deletions .github/actions/test-macos/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@ runs:
export TEST_PHP_JUNIT=junit.out.xml
export STACK_LIMIT_DEFAULTS_CHECK=1
sapi/cli/php run-tests.php -P -q ${{ inputs.runTestsParameters }} \
-d opcache.protect_memory=1 \
-d opcache.jit_buffer_size=64M \
-d opcache.jit_max_root_traces=1000000 \
-d opcache.jit_max_side_traces=1000000 \
-d opcache.jit_max_exit_counters=1000000 \
-d opcache.jit_hot_loop=1 \
-d opcache.jit_hot_func=1 \
-d opcache.jit_hot_return=1 \
-d opcache.jit_hot_side_exit=1 \
-j$(sysctl -n hw.ncpu) \
-g FAIL,BORK,LEAK,XLEAK \
--no-progress \
Expand Down
2 changes: 1 addition & 1 deletion .github/scripts/windows/test_task.bat
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ set OPENSSL_CONF=
rem set SSLEAY_CONF=

rem prepare for OPcache
if "%OPCACHE%" equ "1" set OPCACHE_OPTS=-d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.protect_memory=1 -d opcache.jit_buffer_size=16M
if "%OPCACHE%" equ "1" set OPCACHE_OPTS=-d opcache.enable=1 -d opcache.enable_cli=1 -d opcache.protect_memory=1 -d opcache.jit_buffer_size=64M -d opcache.jit_max_root_traces=1000000 -d opcache.jit_max_side_traces=1000000 -d opcache.jit_max_exit_counters=1000000 -d opcache.jit_hot_loop=1 -d opcache.jit_hot_func=1 -d opcache.jit_hot_return=1 -d opcache.jit_hot_side_exit=1 -d opcache.jit=tracing
rem work-around for failing to dl(mysqli) with OPcache (https://github.com/php/php-src/issues/8508)
if "%OPCACHE%" equ "1" set OPCACHE_OPTS=%OPCACHE_OPTS% -d extension=mysqli

Expand Down
75 changes: 63 additions & 12 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ jobs:
${{ matrix.run_tests_parameters }}
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=tracing
- name: Test OpCache
uses: ./.github/actions/test-linux
with:
Expand All @@ -116,6 +116,7 @@ jobs:
${{ matrix.run_tests_parameters }}
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
- name: Test Function JIT
# ASAN frequently timeouts. Each test run takes ~90 minutes, we can
# avoid running into the 6 hour timeout by skipping the function JIT.
Expand All @@ -127,8 +128,7 @@ jobs:
${{ matrix.run_tests_parameters }}
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=1205
-d opcache.jit=function
- name: Verify generated files are up to date
uses: ./.github/actions/verify-generated-files
- name: Notify Slack
Expand Down Expand Up @@ -198,23 +198,23 @@ jobs:
${{ matrix.run_tests_parameters }}
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=tracing
- name: Test OpCache
uses: ./.github/actions/test-linux
with:
runTestsParameters: >-
${{ matrix.run_tests_parameters }}
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
- name: Test Function JIT
uses: ./.github/actions/test-linux
with:
runTestsParameters: >-
${{ matrix.run_tests_parameters }}
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=1205
-d opcache.jit=function
- name: Notify Slack
if: failure()
uses: ./.github/actions/notify-slack
Expand Down Expand Up @@ -261,26 +261,23 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.protect_memory=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=tracing
- name: Test OpCache
uses: ./.github/actions/test-macos
with:
testArtifacts: ${{ matrix.branch.name }}_${{ matrix.debug && 'DEBUG' || 'RELEASE' }}_${{ matrix.zts && 'ZTS' || 'NTS' }} OpCache
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.protect_memory=1
-d opcache.jit=disable
- name: Test Function JIT
uses: ./.github/actions/test-macos
with:
testArtifacts: ${{ matrix.branch.name }}_${{ matrix.debug && 'DEBUG' || 'RELEASE' }}_${{ matrix.zts && 'ZTS' || 'NTS' }} Function JIT
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.protect_memory=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=1205
-d opcache.jit=function
- name: Verify generated files are up to date
uses: ./.github/actions/verify-generated-files
- name: Notify Slack
Expand Down Expand Up @@ -324,6 +321,7 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=tracing
- name: Upload Test Coverage to Codecov.io
if: always()
run: bash <(curl -s https://codecov.io/bash)
Expand Down Expand Up @@ -375,9 +373,56 @@ jobs:
echo opcache.enable_cli=1 >> /etc/php.d/opcache.ini
echo opcache.protect_memory=1 >> /etc/php.d/opcache.ini
echo opcache.jit_buffer_size=1G >> /etc/php.d/opcache.ini
echo opcache.jit_max_root_traces=1000000 >> /etc/php.d/opcache.ini
echo opcache.jit_max_side_traces=1000000 >> /etc/php.d/opcache.ini
echo opcache.jit_max_exit_counters=1000000 >> /etc/php.d/opcache.ini
echo opcache.jit_hot_loop=1 >> /etc/php.d/opcache.ini
echo opcache.jit_hot_func=1 >> /etc/php.d/opcache.ini
echo opcache.jit_hot_return=1 >> /etc/php.d/opcache.ini
echo opcache.jit_hot_side_exit=1 >> /etc/php.d/opcache.ini
echo memory_limit=-1 >> /etc/php.d/opcache.ini
php -v
- name: Test PHPSeclib
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please split this into a separate PR? Can you also temporarily copy this to push.yml to show that the tests are succeeding? It would be great if we could trigger this on-demand, but we don't have that currently...

Copy link
Contributor Author

@danog danog Sep 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will split this to a separate PR, actually most of the new nightly tests are failing under JIT, might be worth pinging @dstogov about this:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split to #12270

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will split this to a separate PR, actually most of the new nightly tests are failing under JIT, might be worth pinging @dstogov about this:

Thanks. I've assigned these three issues to myself.

if: always()
run: |
git clone https://github.com/phpseclib/phpseclib --branch=master
cd phpseclib
git checkout 259bd9f1e8af11726ed74acf527c2c046549061b
export ASAN_OPTIONS=exitcode=139
export PHPSECLIB_ALLOW_JIT=1
php /usr/bin/composer install --no-progress --ignore-platform-reqs
php vendor/bin/paratest --verbose --configuration=tests/phpunit.xml --runner=WrapperRunner || exit $?
git clean -ffdx
- name: Test Psalm
if: matrix.branch.ref != 'PHP-8.0'
run: |
git clone https://github.com/vimeo/psalm --branch=master
cd psalm
git checkout 7428e49b115a2a837aa29cf0fafd0ca902fe2457
export ASAN_OPTIONS=exitcode=139
export PSALM_ALLOW_XDEBUG=1
php /usr/bin/composer install --no-progress --ignore-platform-reqs
php ./psalm --no-cache || exit $?
- name: Test PHPStan
if: matrix.branch.ref != 'PHP-8.0'
run: |
git clone https://github.com/phpstan/phpstan-src
cd phpstan-src
git checkout 1f608dc6a560f4a5accc854add8e005da0e7ceea
sed 's/80299/89999/g' -i conf/config.neon
php /usr/bin/composer install --no-progress --ignore-platform-reqs
export ASAN_OPTIONS=exitcode=139
php bin/phpstan clear-result-cache
php bin/phpstan || exit $?
- name: Test AMPHP
if: matrix.branch.ref != 'PHP-8.0'
run: |
sudo apt-get update && sudo apt-get install -y libuv1-dev
git clone https://github.com/amphp/ext-uv --depth 1
cd ext-uv
phpize && ./configure --enable-debug && make -j$(nproc) && sudo make install
cd ..
echo extension=uv >> /etc/php.d/uv.ini
repositories="amp cache dns file http parallel parser pipeline process serialization socket sync websocket-client websocket-server"
X=0
for repository in $repositories; do
Expand Down Expand Up @@ -546,20 +591,23 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
--file-cache-prime
- name: Test File Cache (prime shm, use shm)
uses: ./.github/actions/test-linux
with:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
--file-cache-use
- name: Test File Cache (prime shm, use file)
uses: ./.github/actions/test-linux
with:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
--file-cache-use
-d opcache.file_cache_only=1
- name: Test File Cache Only (prime)
Expand All @@ -568,6 +616,7 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
--file-cache-prime
-d opcache.file_cache_only=1
- name: Test File Cache Only (use)
Expand All @@ -576,6 +625,7 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
--file-cache-use
-d opcache.file_cache_only=1
- name: Verify generated files are up to date
Expand Down Expand Up @@ -676,6 +726,7 @@ jobs:
--msan
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.jit=disable
- name: Verify generated files are up to date
uses: ./.github/actions/verify-generated-files
- name: Notify Slack
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
${{ !matrix.asan && '-d opcache.jit_buffer_size=16M' || '' }}
${{ matrix.asan && '-d opcache.jit=disable' || '-d opcache.jit=tracing' }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would result in opcache.jit=tracing twice for non-asan, as it's already added in the test action.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, where? I see only two instances of opcache.jit=tracing in this workflow, one here and the other in the mac OS tests...

${{ matrix.asan && '--asan -x' || '' }}
- name: Verify generated files are up to date
if: ${{ !matrix.asan }}
Expand Down Expand Up @@ -163,8 +163,7 @@ jobs:
runTestsParameters: >-
-d zend_extension=opcache.so
-d opcache.enable_cli=1
-d opcache.protect_memory=1
-d opcache.jit_buffer_size=16M
-d opcache.jit=tracing
- name: Verify generated files are up to date
uses: ./.github/actions/verify-generated-files
WINDOWS:
Expand Down
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ before_script:

# Run PHPs run-tests.php
script:
- travis_wait ./travis/test.sh -d opcache.jit_buffer_size=16M -d opcache.jit=tracing
- travis_wait 60 ./travis/test.sh -d opcache.jit=tracing -d opcache.jit_buffer_size=64M -d opcache.jit_max_root_traces=1000000 -d opcache.jit_max_side_traces=1000000 -d opcache.jit_max_exit_counters=1000000 -d opcache.jit_hot_loop=1 -d opcache.jit_hot_func=1 -d opcache.jit_hot_return=1 -d opcache.jit_hot_side_exit=1
- sapi/cli/php -d extension_dir=`pwd`/modules -r 'dl("zend_test");'

after_success:
Expand Down
73 changes: 73 additions & 0 deletions ext/opcache/tests/jit/bug11917.phpt
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
--TEST--
Bug #11917 (primitives seem to be passed via reference instead of by value under some conditions when JIT is enabled on windows)
--INI--
opcache.enable=1
opcache.enable_cli=1
opcache.protect_memory=1
opcache.jit_buffer_size=64M
opcache.jit=1255
opcache.jit_max_root_traces=1000000
opcache.jit_max_side_traces=1000000
opcache.jit_max_exit_counters=1000000
opcache.jit_hot_loop=1
opcache.jit_hot_func=1
opcache.jit_hot_return=1
opcache.jit_hot_side_exit=1
--EXTENSIONS--
opcache
--FILE--
<?php
$a = [2147483647,2147483647,2147483647,3,0,0,32,2147483584,127];
echo crc32(json_encode(bitwise_small_split($a))) . "\n";
echo crc32(json_encode(bitwise_small_split($a))) . "\n";
echo crc32(json_encode(bitwise_small_split($a))) . "\n";
echo crc32(json_encode(bitwise_small_split($a))) . "\n";

function bitwise_small_split($val)
{
$split = 8;
$vals = [];

$mask = (1 << $split) - 1;

$i = $overflow = 0;
$len = count($val);
$val[] = 0;
$remaining = 31;

while ($i != $len) {
$digit = $val[$i] & $mask;

$val[$i] >>= $split;
if (!$overflow) {
$remaining -= $split;
$overflow = $split <= $remaining ? 0 : $split - $remaining;

if (!$remaining) {
$i++;
$remaining = 31;
$overflow = 0;
}
} elseif (++$i != $len) {
$tempmask = (1 << $overflow) - 1;
$digit |= ($val[$i] & $tempmask) << $remaining;
$val[$i] >>= $overflow;
$remaining = 31 - $overflow;
$overflow = $split <= $remaining ? 0 : $split - $remaining;
}

$vals[] = $digit;
}

while ($vals[count($vals) - 1] == 0) {
unset($vals[count($vals) - 1]);
}

return array_reverse($vals);
}
?>
--EXPECT--
48207660
48207660
48207660
48207660