[ARM]: codegen `llvm.roundeven.v*` #141786

folkertdev · 2025-05-28T15:05:44Z

The aarch64 version of frintn.ll notes the intention to auto-upgrade frintn to roundeven. I haven't been able to figure out how to make that happen though (either for arm or aarch64).

The original issue came up in rust-lang/stdarch#1807

github-actions · 2025-05-28T15:06:08Z

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

llvmbot · 2025-05-28T15:06:42Z

@llvm/pr-subscribers-backend-arm

Author: Folkert de Vries (folkertdev)

Changes

fixes #73588

The aarch64 version of frintn.ll notes the intention to auto-upgrade frintn to roundeven. I haven't been able to figure out how to make that happen though (either for arm or aarch64).

Full diff: https://github.com/llvm/llvm-project/pull/141786.diff

4 Files Affected:

(modified) llvm/lib/Target/ARM/ARMISelLowering.cpp (+6)
(modified) llvm/lib/Target/ARM/ARMInstrVFP.td (+1-1)
(modified) llvm/test/CodeGen/ARM/arm32-rounding.ll (+18)
(added) llvm/test/CodeGen/ARM/frintn.ll (+78)

diff --git a/llvm/lib/Target/ARM/ARMISelLowering.cpp b/llvm/lib/Target/ARM/ARMISelLowering.cpp
index afbf1b4c55e70..8b65d3eeafa06 100644
--- a/llvm/lib/Target/ARM/ARMISelLowering.cpp
+++ b/llvm/lib/Target/ARM/ARMISelLowering.cpp
@@ -905,6 +905,7 @@ ARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
     setOperationAction(ISD::FCEIL, MVT::v2f64, Expand);
     setOperationAction(ISD::FTRUNC, MVT::v2f64, Expand);
     setOperationAction(ISD::FRINT, MVT::v2f64, Expand);
+    setOperationAction(ISD::FROUNDEVEN, MVT::v2f64, Expand);
     setOperationAction(ISD::FNEARBYINT, MVT::v2f64, Expand);
     setOperationAction(ISD::FFLOOR, MVT::v2f64, Expand);
     setOperationAction(ISD::FMA, MVT::v2f64, Expand);
@@ -927,6 +928,7 @@ ARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
     setOperationAction(ISD::FCEIL, MVT::v4f32, Expand);
     setOperationAction(ISD::FTRUNC, MVT::v4f32, Expand);
     setOperationAction(ISD::FRINT, MVT::v4f32, Expand);
+    setOperationAction(ISD::FROUNDEVEN, MVT::v4f32, Expand);
     setOperationAction(ISD::FNEARBYINT, MVT::v4f32, Expand);
     setOperationAction(ISD::FFLOOR, MVT::v4f32, Expand);
 
@@ -945,6 +947,7 @@ ARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
     setOperationAction(ISD::FCEIL, MVT::v2f32, Expand);
     setOperationAction(ISD::FTRUNC, MVT::v2f32, Expand);
     setOperationAction(ISD::FRINT, MVT::v2f32, Expand);
+    setOperationAction(ISD::FROUNDEVEN, MVT::v2f32, Expand);
     setOperationAction(ISD::FNEARBYINT, MVT::v2f32, Expand);
     setOperationAction(ISD::FFLOOR, MVT::v2f32, Expand);
 
@@ -1087,6 +1090,7 @@ ARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
     setOperationAction(ISD::FCEIL,      MVT::f64, Expand);
     setOperationAction(ISD::FTRUNC,     MVT::f64, Expand);
     setOperationAction(ISD::FRINT,      MVT::f64, Expand);
+    setOperationAction(ISD::FROUNDEVEN, MVT::f64, Expand);
     setOperationAction(ISD::FNEARBYINT, MVT::f64, Expand);
     setOperationAction(ISD::FFLOOR,     MVT::f64, Expand);
     setOperationAction(ISD::SINT_TO_FP, MVT::i32, Custom);
@@ -1534,6 +1538,7 @@ ARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
     setOperationAction(ISD::FTRUNC, MVT::f32, Legal);
     setOperationAction(ISD::FNEARBYINT, MVT::f32, Legal);
     setOperationAction(ISD::FRINT, MVT::f32, Legal);
+    setOperationAction(ISD::FROUNDEVEN, MVT::f32, Legal);
     setOperationAction(ISD::FMINNUM, MVT::f32, Legal);
     setOperationAction(ISD::FMAXNUM, MVT::f32, Legal);
     if (Subtarget->hasNEON()) {
@@ -1550,6 +1555,7 @@ ARMTargetLowering::ARMTargetLowering(const TargetMachine &TM,
       setOperationAction(ISD::FTRUNC, MVT::f64, Legal);
       setOperationAction(ISD::FNEARBYINT, MVT::f64, Legal);
       setOperationAction(ISD::FRINT, MVT::f64, Legal);
+      setOperationAction(ISD::FROUNDEVEN, MVT::f64, Legal);
       setOperationAction(ISD::FMINNUM, MVT::f64, Legal);
       setOperationAction(ISD::FMAXNUM, MVT::f64, Legal);
     }
diff --git a/llvm/lib/Target/ARM/ARMInstrVFP.td b/llvm/lib/Target/ARM/ARMInstrVFP.td
index 2ee2cb2fda4a0..483786a88d28c 100644
--- a/llvm/lib/Target/ARM/ARMInstrVFP.td
+++ b/llvm/lib/Target/ARM/ARMInstrVFP.td
@@ -1095,7 +1095,7 @@ multiclass vrint_inst_zrx<string opc, bit op, bit op2, SDPatternOperator node> {
 
 defm VRINTZ : vrint_inst_zrx<"z", 0, 1, ftrunc>;
 defm VRINTR : vrint_inst_zrx<"r", 0, 0, fnearbyint>;
-defm VRINTX : vrint_inst_zrx<"x", 1, 0, frint>;
+defm VRINTX : vrint_inst_zrx<"x", 1, 0, any_froundeven>;
 
 multiclass vrint_inst_anpm<string opc, bits<2> rm,
                            SDPatternOperator node = null_frag> {
diff --git a/llvm/test/CodeGen/ARM/arm32-rounding.ll b/llvm/test/CodeGen/ARM/arm32-rounding.ll
index b0a9f54e42404..3b9aced91143e 100644
--- a/llvm/test/CodeGen/ARM/arm32-rounding.ll
+++ b/llvm/test/CodeGen/ARM/arm32-rounding.ll
@@ -104,6 +104,22 @@ entry:
   ret double %call
 }
 
+; CHECK-LABEL: test13
+; CHECK: b roundevenf
+define float @test13(float %a) {
+entry:
+  %round = call float @llvm.roundeven.f32(float %a)
+  ret float %round
+}
+
+; CHECK-LABEL: test14
+; CHECK: b roundeven
+define double @test14(double %a) {
+entry:
+  %round = call double @llvm.roundeven.f64(double %a)
+  ret double %round
+}
+
 declare float @floorf(float) nounwind readnone
 declare double @floor(double) nounwind readnone
 declare float @ceilf(float) nounwind readnone
@@ -116,3 +132,5 @@ declare float @nearbyintf(float) nounwind readnone
 declare double @nearbyint(double) nounwind readnone
 declare float @rintf(float) nounwind readnone
 declare double @rint(double) nounwind readnone
+declare float @llvm.roundeven.f32(float)
+declare double @llvm.roundeven.f64(double)
diff --git a/llvm/test/CodeGen/ARM/frintn.ll b/llvm/test/CodeGen/ARM/frintn.ll
new file mode 100644
index 0000000000000..e801b8ea0a707
--- /dev/null
+++ b/llvm/test/CodeGen/ARM/frintn.ll
@@ -0,0 +1,78 @@
+; RUN: llc -mtriple=armv8 -mattr=+neon %s -o - | FileCheck %s
+
+; The llvm.arm.neon.frintn intrinsic should be auto-upgraded to the
+; target-independent roundeven intrinsic.
+
+define <4 x half> @frintn_4h(<4 x half> %A) nounwind {
+;CHECK-LABEL: frintn_4h:
+;CHECK: bl llvm.arm.neon.frintn.v4f16
+	%tmp3 = call <4 x half> @llvm.arm.neon.frintn.v4f16(<4 x half> %A)
+	ret <4 x half> %tmp3
+}
+
+define <2 x float> @frintn_2s(<2 x float> %A) nounwind {
+;CHECK-LABEL: frintn_2s:
+;CHECK: bl llvm.arm.neon.frintn.v2f32
+	%tmp3 = call <2 x float> @llvm.arm.neon.frintn.v2f32(<2 x float> %A)
+	ret <2 x float> %tmp3
+}
+
+define <4 x float> @frintn_4s(<4 x float> %A) nounwind {
+;CHECK-LABEL: frintn_4s:
+;CHECK: bl llvm.arm.neon.frintn.v4f32
+	%tmp3 = call <4 x float> @llvm.arm.neon.frintn.v4f32(<4 x float> %A)
+	ret <4 x float> %tmp3
+}
+
+define <2 x double> @frintn_2d(<2 x double> %A) nounwind {
+;CHECK-LABEL: frintn_2d:
+;CHECK: bl llvm.arm.neon.frintn.v2f64
+	%tmp3 = call <2 x double> @llvm.arm.neon.frintn.v2f64(<2 x double> %A)
+	ret <2 x double> %tmp3
+}
+
+declare <4 x half> @llvm.arm.neon.frintn.v4f16(<4 x half>) nounwind readnone
+declare <2 x float> @llvm.arm.neon.frintn.v2f32(<2 x float>) nounwind readnone
+declare <4 x float> @llvm.arm.neon.frintn.v4f32(<4 x float>) nounwind readnone
+declare <2 x double> @llvm.arm.neon.frintn.v2f64(<2 x double>) nounwind readnone
+
+define <4 x half> @roundeven_4h(<4 x half> %A) nounwind {
+;CHECK-LABEL: roundeven_4h:
+;CHECK: vcvtb.f16.f32 s0, s0
+;CHECK: vrintx.f32 s2, s2
+;CHECK: vmov r0, s0
+;CHECK: vcvtb.f16.f32 s2, s2
+	%tmp3 = call <4 x half> @llvm.roundeven.v4f16(<4 x half> %A)
+	ret <4 x half> %tmp3
+}
+
+define <2 x float> @roundeven_2s(<2 x float> %A) nounwind {
+;CHECK-LABEL: roundeven_2s:
+;CHECK: vrintx.f32 s3, s1
+;CHECK: vrintx.f32 s2, s0
+	%tmp3 = call <2 x float> @llvm.roundeven.v2f32(<2 x float> %A)
+	ret <2 x float> %tmp3
+}
+
+define <4 x float> @roundeven_4s(<4 x float> %A) nounwind {
+;CHECK-LABEL: roundeven_4s:
+;CHECK: vrintx.f32 s7, s3
+;CHECK: vrintx.f32 s6, s2
+;CHECK: vrintx.f32 s5, s1
+;CHECK: vrintx.f32 s4, s0
+	%tmp3 = call <4 x float> @llvm.roundeven.v4f32(<4 x float> %A)
+	ret <4 x float> %tmp3
+}
+
+define <2 x double> @roundeven_2d(<2 x double> %A) nounwind {
+;CHECK-LABEL: roundeven_2d:
+;CHECK: vrintx.f64 d16, d16
+;CHECK: vrintx.f64 d17, d17
+	%tmp3 = call <2 x double> @llvm.roundeven.v2f64(<2 x double> %A)
+	ret <2 x double> %tmp3
+}
+
+declare <4 x half> @llvm.roundeven.v4f16(<4 x half>) nounwind readnone
+declare <2 x float> @llvm.roundeven.v2f32(<2 x float>) nounwind readnone
+declare <4 x float> @llvm.roundeven.v4f32(<4 x float>) nounwind readnone
+declare <2 x double> @llvm.roundeven.v2f64(<2 x double>) nounwind readnone

folkertdev · 2025-05-28T23:05:23Z

Now froundeven has the same behavior as frint, as desired. I'm not sure if I've implemented that in an idiomatic way though, so eh, please let me know?

davemgreen

Hi - thanks for looking into this. I think it will want to generate a vrintn, which is round-to-even. vrintx uses the current rounding mode (and sets inexact flags).

There is an example in https://reviews.llvm.org/D98487, where old llvm.aarch64.neon.frintn intrinsics were upgraded to roundeven.

folkertdev · 2025-05-29T10:04:00Z

Allright, roundeven generates vrintn now. Are there any other parts of the PR you linked that are important (it's large and from a while ago, and the arm backend is different, so I'm having a hard time figuring out what is relevant and what is not).

davemgreen

Can you add test cases to
llvm/test/CodeGen/ARM/fp16-fullfp16.ll
llvm/test/CodeGen/ARM/fp16-promote.ll
llvm/test/CodeGen/Thumb2/float-intrinsics-double.ll
llvm/test/CodeGen/Thumb2/float-intrinsics-float.ll
Just like the existing llvm.nearbyint tests. That should cover the scalar types. MVE I can add separately, it should be relatively simple once you have added the scalars in.

There is also a test in llvm/test/CodeGen/ARM/vrint.ll, but it looks almost useless. It should be testing that vrintn.f32 s0, s0 gets generated, but that might now stop working. The idea is that either llvm.roundeven and llvm.arm.neon.vrintn both produce a vrintn, or (preferably) the llvm.arm.neon.vrintn is auto-upgraded to roundeven, like the aarch64 patch above.

llvm/test/CodeGen/ARM/frintn.ll

davemgreen · 2025-05-30T12:08:59Z

Can you add tests into these files to make sure they are generating correctly in the various configs:

llvm/test/CodeGen/ARM/fp16-fullfp16.ll
llvm/test/CodeGen/ARM/fp16-promote.ll
llvm/test/CodeGen/Thumb2/float-intrinsics-double.ll
llvm/test/CodeGen/Thumb2/float-intrinsics-float.ll

There are still some oddities on Neon for all the rounding intrinsics, like you mention. I will try and look into those separately but otherwise this looks OK to me.

llvm/lib/Target/ARM/ARMInstrVFP.td

folkertdev · 2025-05-30T16:28:09Z

Cool, I've added the additional tests.

davemgreen

Thanks, LGTM. Are you happy for this to be merged in?

folkertdev · 2025-05-30T17:27:46Z

yes!

github-actions · 2025-05-30T18:36:49Z

@folkertdev Congratulations on having your first Pull Request (PR) merged into the LLVM Project!

Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail here.

If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are working as expected, well done!

llvmbot added the backend:ARM label May 28, 2025

folkertdev changed the title ~~[LLVM]: codegen llvm.roundeven.v* for ARM~~ [ARM]: codegen llvm.roundeven.v* May 28, 2025

folkertdev force-pushed the arm-roundeven branch 2 times, most recently from 5ade51d to 3ac6465 Compare May 28, 2025 23:02

davemgreen reviewed May 29, 2025

View reviewed changes

folkertdev force-pushed the arm-roundeven branch from 3ac6465 to e86efbb Compare May 29, 2025 10:02

davemgreen reviewed May 29, 2025

View reviewed changes

llvm/test/CodeGen/ARM/frintn.ll Outdated Show resolved Hide resolved

llvm/test/CodeGen/ARM/frintn.ll Outdated Show resolved Hide resolved

llvm/test/CodeGen/ARM/frintn.ll Outdated Show resolved Hide resolved

folkertdev force-pushed the arm-roundeven branch 2 times, most recently from a4df8dc to f473a87 Compare May 29, 2025 18:05

davemgreen reviewed May 30, 2025

View reviewed changes

llvm/lib/Target/ARM/ARMInstrVFP.td Outdated Show resolved Hide resolved

[ARM]: codegen llvm.roundeven.v*

ee56f48

folkertdev force-pushed the arm-roundeven branch from f473a87 to ee56f48 Compare May 30, 2025 16:27

davemgreen approved these changes May 30, 2025

View reviewed changes

davemgreen merged commit 46b3892 into llvm:main May 30, 2025
9 of 11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ARM]: codegen `llvm.roundeven.v*` #141786

[ARM]: codegen `llvm.roundeven.v*` #141786

Uh oh!

folkertdev commented May 28, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 28, 2025

Uh oh!

llvmbot commented May 28, 2025

Uh oh!

folkertdev commented May 28, 2025

Uh oh!

davemgreen left a comment

Uh oh!

folkertdev commented May 29, 2025

Uh oh!

davemgreen left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davemgreen commented May 30, 2025

Uh oh!

Uh oh!

folkertdev commented May 30, 2025

Uh oh!

davemgreen left a comment

Uh oh!

folkertdev commented May 30, 2025

Uh oh!

Uh oh!

github-actions bot commented May 30, 2025

Uh oh!

Uh oh!

[ARM]: codegen llvm.roundeven.v* #141786

[ARM]: codegen llvm.roundeven.v* #141786

Uh oh!

Conversation

folkertdev commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 28, 2025

Uh oh!

llvmbot commented May 28, 2025

Uh oh!

folkertdev commented May 28, 2025

Uh oh!

davemgreen left a comment

Choose a reason for hiding this comment

Uh oh!

folkertdev commented May 29, 2025

Uh oh!

davemgreen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davemgreen commented May 30, 2025

Uh oh!

Uh oh!

folkertdev commented May 30, 2025

Uh oh!

davemgreen left a comment

Choose a reason for hiding this comment

Uh oh!

folkertdev commented May 30, 2025

Uh oh!

Uh oh!

github-actions bot commented May 30, 2025

Uh oh!

Uh oh!

[ARM]: codegen `llvm.roundeven.v*` #141786

[ARM]: codegen `llvm.roundeven.v*` #141786

folkertdev commented May 28, 2025 •

edited

Loading