AMDGPU: Report special input intrinsics as free #141948

arsenm · 2025-05-29T13:37:12Z

No description provided.

arsenm · 2025-05-29T13:37:25Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

llvmbot · 2025-05-29T17:07:59Z

@llvm/pr-subscribers-llvm-analysis

@llvm/pr-subscribers-backend-amdgpu

Author: Matt Arsenault (arsenm)

Changes

Full diff: https://github.com/llvm/llvm-project/pull/141948.diff

2 Files Affected:

(modified) llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp (+22-1)
(modified) llvm/test/Analysis/CostModel/AMDGPU/special-argument-intrinsics.ll (+28-28)

diff --git a/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
index 357f8c5cfcd02..f9dbfb18ab7ee 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp
@@ -704,8 +704,29 @@ static bool intrinsicHasPackedVectorBenefit(Intrinsic::ID ID) {
 InstructionCost
 GCNTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
                                   TTI::TargetCostKind CostKind) const {
-  if (ICA.getID() == Intrinsic::fabs)
+  switch (ICA.getID()) {
+  case Intrinsic::fabs:
+    // Free source modifier in the common case.
+    return 0;
+  case Intrinsic::amdgcn_workitem_id_x:
+  case Intrinsic::amdgcn_workitem_id_y:
+  case Intrinsic::amdgcn_workitem_id_z:
+    // TODO: If hasPackedTID, or if the calling context is not an entry point
+    // there may be a bit instruction.
+    return 0;
+  case Intrinsic::amdgcn_workgroup_id_x:
+  case Intrinsic::amdgcn_workgroup_id_y:
+  case Intrinsic::amdgcn_workgroup_id_z:
+  case Intrinsic::amdgcn_lds_kernel_id:
+  case Intrinsic::amdgcn_dispatch_ptr:
+  case Intrinsic::amdgcn_dispatch_id:
+  case Intrinsic::amdgcn_implicitarg_ptr:
+  case Intrinsic::amdgcn_queue_ptr:
+    // Read from an argument register.
     return 0;
+  default:
+    break;
+  }
 
   if (!intrinsicHasPackedVectorBenefit(ICA.getID()))
     return BaseT::getIntrinsicInstrCost(ICA, CostKind);
diff --git a/llvm/test/Analysis/CostModel/AMDGPU/special-argument-intrinsics.ll b/llvm/test/Analysis/CostModel/AMDGPU/special-argument-intrinsics.ll
index ea045e04310be..00dbcff0a021f 100644
--- a/llvm/test/Analysis/CostModel/AMDGPU/special-argument-intrinsics.ll
+++ b/llvm/test/Analysis/CostModel/AMDGPU/special-argument-intrinsics.ll
@@ -7,11 +7,11 @@
 
 define i32 @workitem_id_x() {
 ; ALL-LABEL: 'workitem_id_x'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'workitem_id_x'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.workitem.id.x()
@@ -20,12 +20,12 @@ define i32 @workitem_id_x() {
 
 define amdgpu_kernel void @kernel_workitem_id_x(ptr addrspace(1) %ptr) {
 ; ALL-LABEL: 'kernel_workitem_id_x'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store i32 %result, ptr addrspace(1) %ptr, align 4
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret void
 ;
 ; SIZE-LABEL: 'kernel_workitem_id_x'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.x()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store i32 %result, ptr addrspace(1) %ptr, align 4
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
@@ -36,11 +36,11 @@ define amdgpu_kernel void @kernel_workitem_id_x(ptr addrspace(1) %ptr) {
 
 define i32 @workitem_id_y() {
 ; ALL-LABEL: 'workitem_id_y'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'workitem_id_y'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.workitem.id.y()
@@ -49,12 +49,12 @@ define i32 @workitem_id_y() {
 
 define amdgpu_kernel void @kernel_workitem_id_y(ptr addrspace(1) %ptr) {
 ; ALL-LABEL: 'kernel_workitem_id_y'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store i32 %result, ptr addrspace(1) %ptr, align 4
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret void
 ;
 ; SIZE-LABEL: 'kernel_workitem_id_y'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store i32 %result, ptr addrspace(1) %ptr, align 4
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
@@ -65,11 +65,11 @@ define amdgpu_kernel void @kernel_workitem_id_y(ptr addrspace(1) %ptr) {
 
 define i32 @workitem_id_z() {
 ; ALL-LABEL: 'workitem_id_z'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'workitem_id_z'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.y()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.workitem.id.y()
@@ -78,12 +78,12 @@ define i32 @workitem_id_z() {
 
 define amdgpu_kernel void @kernel_workitem_id_z(ptr addrspace(1) %ptr) {
 ; ALL-LABEL: 'kernel_workitem_id_z'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.z()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.z()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store i32 %result, ptr addrspace(1) %ptr, align 4
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret void
 ;
 ; SIZE-LABEL: 'kernel_workitem_id_z'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.z()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workitem.id.z()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store i32 %result, ptr addrspace(1) %ptr, align 4
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
@@ -94,11 +94,11 @@ define amdgpu_kernel void @kernel_workitem_id_z(ptr addrspace(1) %ptr) {
 
 define i32 @workgroup_id_x() {
 ; ALL-LABEL: 'workgroup_id_x'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.x()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.x()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'workgroup_id_x'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.x()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.x()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.workgroup.id.x()
@@ -107,11 +107,11 @@ define i32 @workgroup_id_x() {
 
 define i32 @workgroup_id_y() {
 ; ALL-LABEL: 'workgroup_id_y'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'workgroup_id_y'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.workgroup.id.y()
@@ -120,11 +120,11 @@ define i32 @workgroup_id_y() {
 
 define i32 @workgroup_id_z() {
 ; ALL-LABEL: 'workgroup_id_z'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'workgroup_id_z'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.workgroup.id.y()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.workgroup.id.y()
@@ -133,11 +133,11 @@ define i32 @workgroup_id_z() {
 
 define i32 @lds_kernel_id() {
 ; ALL-LABEL: 'lds_kernel_id'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.lds.kernel.id()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.lds.kernel.id()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i32 %result
 ;
 ; SIZE-LABEL: 'lds_kernel_id'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i32 @llvm.amdgcn.lds.kernel.id()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i32 @llvm.amdgcn.lds.kernel.id()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i32 %result
 ;
   %result = call i32 @llvm.amdgcn.lds.kernel.id()
@@ -146,11 +146,11 @@ define i32 @lds_kernel_id() {
 
 define ptr addrspace(4) @dispatch_ptr() {
 ; ALL-LABEL: 'dispatch_ptr'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret ptr addrspace(4) %result
 ;
 ; SIZE-LABEL: 'dispatch_ptr'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret ptr addrspace(4) %result
 ;
   %result = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
@@ -159,11 +159,11 @@ define ptr addrspace(4) @dispatch_ptr() {
 
 define i64 @dispatch_id_() {
 ; ALL-LABEL: 'dispatch_id_'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i64 @llvm.amdgcn.dispatch.id()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i64 @llvm.amdgcn.dispatch.id()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret i64 %result
 ;
 ; SIZE-LABEL: 'dispatch_id_'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call i64 @llvm.amdgcn.dispatch.id()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call i64 @llvm.amdgcn.dispatch.id()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret i64 %result
 ;
   %result = call i64 @llvm.amdgcn.dispatch.id()
@@ -172,11 +172,11 @@ define i64 @dispatch_id_() {
 
 define ptr addrspace(4) @implicitarg_ptr() {
 ; ALL-LABEL: 'implicitarg_ptr'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret ptr addrspace(4) %result
 ;
 ; SIZE-LABEL: 'implicitarg_ptr'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret ptr addrspace(4) %result
 ;
   %result = call ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr()
@@ -185,11 +185,11 @@ define ptr addrspace(4) @implicitarg_ptr() {
 
 define ptr addrspace(4) @queue_ptr() {
 ; ALL-LABEL: 'queue_ptr'
-; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.queue.ptr()
+; ALL-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.queue.ptr()
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret ptr addrspace(4) %result
 ;
 ; SIZE-LABEL: 'queue_ptr'
-; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.queue.ptr()
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %result = call ptr addrspace(4) @llvm.amdgcn.queue.ptr()
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret ptr addrspace(4) %result
 ;
   %result = call ptr addrspace(4) @llvm.amdgcn.queue.ptr()

rampitec · 2025-05-29T19:53:34Z

llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp

+  case Intrinsic::amdgcn_workitem_id_x:
+  case Intrinsic::amdgcn_workitem_id_y:
+  case Intrinsic::amdgcn_workitem_id_z:
+    // TODO: If hasPackedTID, or if the calling context is not an entry point


Maybe report 1 for packed tid?

For some reason the calling instruction isn't set in the context where this gets called from, so I can't figure out if it's a kernel or not so I left this for later

Similarly also can't check if the other work items are disabled

Ok for now.

AMDGPU: Report special input intrinsics as free

f69a1eb

arsenm mentioned this pull request May 29, 2025

AMDGPU: Add baseline cost model tests for special argument intrinsics #141947

Open

arsenm added the backend:AMDGPU label May 29, 2025 — with Graphite App

arsenm requested review from jayfoad, jrbyrnes, rampitec, shiltian and Sisyph May 29, 2025 17:07

arsenm marked this pull request as ready for review May 29, 2025 17:07

llvmbot added the llvm:analysis label May 29, 2025

rampitec reviewed May 29, 2025

View reviewed changes

rampitec approved these changes May 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

AMDGPU: Report special input intrinsics as free #141948

AMDGPU: Report special input intrinsics as free #141948

Uh oh!

arsenm commented May 29, 2025

Uh oh!

arsenm commented May 29, 2025 •

edited

Loading

Uh oh!

llvmbot commented May 29, 2025 •

edited

Loading

Uh oh!

rampitec May 29, 2025

Uh oh!

arsenm May 29, 2025

Uh oh!

arsenm May 29, 2025

Uh oh!

rampitec May 29, 2025

Uh oh!

Uh oh!

AMDGPU: Report special input intrinsics as free #141948

Are you sure you want to change the base?

AMDGPU: Report special input intrinsics as free #141948

Uh oh!

Conversation

arsenm commented May 29, 2025

Uh oh!

arsenm commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rampitec May 29, 2025

Choose a reason for hiding this comment

Uh oh!

arsenm May 29, 2025

Choose a reason for hiding this comment

Uh oh!

arsenm May 29, 2025

Choose a reason for hiding this comment

Uh oh!

rampitec May 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arsenm commented May 29, 2025 •

edited

Loading

llvmbot commented May 29, 2025 •

edited

Loading