[Orc] Add JITLink debug support plugin for ELF x86-64

Add a new ObjectLinkingLayer plugin `DebugObjectManagerPlugin` and infrastructure to handle creation of `DebugObject`s as well as their registration in OrcTargetProcess. The current implementation only covers ELF on x86-64, but the infrastructure is not limited to that.

The journey starts with a new `LinkGraph` / `JITLinkContext` pair being created for a `MaterializationResponsibility` in ORC's `ObjectLinkingLayer`. It sends a `notifyMaterializing()` notification, which is forwarded to all registered plugins. The `DebugObjectManagerPlugin` aims to create a  `DebugObject` form the provided target triple and object buffer. (Future implementations might create `DebugObject`s from a `LinkGraph` in other ways.) On success it will track it as the pending `DebugObject` for the `MaterializationResponsibility`.

This patch only implements the `ELFDebugObject` for `x86-64` targets. It follows the RuntimeDyld approach for debug object setup: it captures a copy of the input object, parses all section headers and prepares to patch their load-address fields with their final addresses in target memory. It instructs the plugin to report the section load-addresses once they are available. The plugin overrides `modifyPassConfig()` and installs a JITLink post-allocation pass to capture them.

Once JITLink emitted the finalized executable, the plugin emits and registers the `DebugObject`. For emission it requests a new `JITLinkMemoryManager::Allocation` with a single read-only segment, copies the object with patched section load-addresses over to working memory and triggers finalization to target memory. For registration, it notifies the `DebugObjectRegistrar` provided in the constructor and stores the previously pending`DebugObject` as registered for the corresponding MaterializationResponsibility.

The `DebugObjectRegistrar` registers the `DebugObject` with the target process. `llvm-jitlink` uses the `TPCDebugObjectRegistrar`, which calls `llvm_orc_registerJITLoaderGDBWrapper()` in the target process via `TargetProcessControl` to emit a `jit_code_entry` compatible with the GDB JIT interface [1]. So far the implementation only supports registration and no removal. It appears to me that it wouldn't raise any new design questions, so I left this as an addition for the near future.

[1] https://sourceware.org/gdb/current/onlinedocs/gdb/JIT-Interface.html

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D97335

Author: weliveindetail

llvm/include/llvm/ExecutionEngine/Orc/DebugObjectManagerPlugin.h

@@ -0,0 +1,84 @@
+//===---- DebugObjectManagerPlugin.h - JITLink debug objects ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// ObjectLinkingLayer plugin for emitting debug objects.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_ORC_DEBUGOBJECTMANAGERPLUGIN_H
+#define LLVM_EXECUTIONENGINE_ORC_DEBUGOBJECTMANAGERPLUGIN_H
+
+#include "llvm/ADT/Triple.h"
+#include "llvm/ExecutionEngine/JITLink/JITLink.h"
+#include "llvm/ExecutionEngine/Orc/Core.h"
+#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
+#include "llvm/ExecutionEngine/Orc/TPCDebugObjectRegistrar.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Memory.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+#include <functional>
+#include <map>
+#include <memory>
+#include <mutex>
+
+namespace llvm {
+namespace orc {
+
+class DebugObject;
+
+/// Creates and manages DebugObjects for JITLink artifacts.
+///
+/// DebugObjects are created when linking for a MaterializationResponsibility
+/// starts. They are pending as long as materialization is in progress.
+///
+/// There can only be one pending DebugObject per MaterializationResponsibility.
+/// If materialization fails, pending DebugObjects are discarded.
+///
+/// Once executable code for the MaterializationResponsibility is emitted, the
+/// corresponding DebugObject is finalized to target memory and the provided
+/// DebugObjectRegistrar is notified. Ownership of DebugObjects remains with the
+/// plugin.
+///
+class DebugObjectManagerPlugin : public ObjectLinkingLayer::Plugin {
+public:
+  DebugObjectManagerPlugin(ExecutionSession &ES,
+                           std::unique_ptr<DebugObjectRegistrar> Target);
+  ~DebugObjectManagerPlugin();
+
+  void notifyMaterializing(MaterializationResponsibility &MR,
+                           jitlink::LinkGraph &G, jitlink::JITLinkContext &Ctx,
+                           MemoryBufferRef InputObject) override;
+
+  Error notifyEmitted(MaterializationResponsibility &MR) override;
+  Error notifyFailed(MaterializationResponsibility &MR) override;
+  Error notifyRemovingResources(ResourceKey K) override;
+
+  void notifyTransferringResources(ResourceKey DstKey,
+                                   ResourceKey SrcKey) override;
+
+  void modifyPassConfig(MaterializationResponsibility &MR, const Triple &TT,
+                        jitlink::PassConfiguration &PassConfig) override;
+
+private:
+  ExecutionSession &ES;
+
+  using OwnedDebugObject = std::unique_ptr<DebugObject>;
+  std::map<ResourceKey, OwnedDebugObject> PendingObjs;
+  std::map<ResourceKey, std::vector<OwnedDebugObject>> RegisteredObjs;
+
+  std::mutex PendingObjsLock;
+  std::mutex RegisteredObjsLock;
+
+  std::unique_ptr<DebugObjectRegistrar> Target;
+};
+
+} // namespace orc
+} // namespace llvm
+
+#endif // LLVM_EXECUTIONENGINE_ORC_DEBUGOBJECTMANAGERPLUGIN_H

llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h

@@ -72,6 +72,13 @@ class ObjectLinkingLayer : public RTTIExtends<ObjectLinkingLayer, ObjectLayer>,
                                   const Triple &TT,
                                   jitlink::PassConfiguration &Config) {}
 
+    // Deprecated. Don't use this in new code. There will be a proper mechanism
+    // for capturing object buffers.
+    virtual void notifyMaterializing(MaterializationResponsibility &MR,
+                                     jitlink::LinkGraph &G,
+                                     jitlink::JITLinkContext &Ctx,
+                                     MemoryBufferRef InputObject) {}
+
     virtual void notifyLoaded(MaterializationResponsibility &MR) {}
     virtual Error notifyEmitted(MaterializationResponsibility &MR) {
       return Error::success();

llvm/include/llvm/ExecutionEngine/Orc/TPCDebugObjectRegistrar.h

@@ -0,0 +1,67 @@
+//===- TPCDebugObjectRegistrar.h - TPC-based debug registration -*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// TargetProcessControl based registration of debug objects.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_ORC_TPCDEBUGOBJECTREGISTRAR_H
+#define LLVM_EXECUTIONENGINE_ORC_TPCDEBUGOBJECTREGISTRAR_H
+
+#include "llvm/ExecutionEngine/JITSymbol.h"
+#include "llvm/ExecutionEngine/Orc/TargetProcessControl.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Memory.h"
+
+#include <cstdint>
+#include <memory>
+#include <vector>
+
+namespace llvm {
+namespace orc {
+
+/// Abstract interface for registering debug objects in the target process.
+class DebugObjectRegistrar {
+public:
+  virtual Error registerDebugObject(sys::MemoryBlock) = 0;
+  virtual ~DebugObjectRegistrar() {}
+};
+
+/// Use TargetProcessControl to register debug objects locally or in a remote
+/// target process.
+class TPCDebugObjectRegistrar : public DebugObjectRegistrar {
+public:
+  using SerializeBlockInfoFn =
+      std::vector<uint8_t> (*)(sys::MemoryBlock TargetMemBlock);
+
+  TPCDebugObjectRegistrar(TargetProcessControl &TPC,
+                          JITTargetAddress RegisterFn,
+                          SerializeBlockInfoFn SerializeBlockInfo)
+      : TPC(TPC), RegisterFn(RegisterFn),
+        SerializeBlockInfo(SerializeBlockInfo) {}
+
+  Error registerDebugObject(sys::MemoryBlock TargetMem) override {
+    return TPC.runWrapper(RegisterFn, SerializeBlockInfo(TargetMem))
+        .takeError();
+  }
+
+private:
+  TargetProcessControl &TPC;
+  JITTargetAddress RegisterFn;
+  SerializeBlockInfoFn SerializeBlockInfo;
+};
+
+/// Create a TargetProcessControl-based DebugObjectRegistrar that emits debug
+/// objects to the GDB JIT interface.
+Expected<std::unique_ptr<TPCDebugObjectRegistrar>>
+createJITLoaderGDBRegistrar(TargetProcessControl &TPC);
+
+} // end namespace orc
+} // end namespace llvm
+
+#endif // LLVM_EXECUTIONENGINE_ORC_TDEBUGOBJECTREGISTRAR_H

llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h

@@ -0,0 +1,22 @@
+//===- JITLoaderGDB.h - Register objects via GDB JIT interface -*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Register objects for access by debuggers via the GDB JIT interface.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_JITLOADERGDB_H
+#define LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_JITLOADERGDB_H
+
+#include "llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h"
+#include <cstdint>
+
+extern "C" llvm::orc::tpctypes::CWrapperFunctionResult
+llvm_orc_registerJITLoaderGDBWrapper(uint8_t *Data, uint64_t Size);
+
+#endif // LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_JITLOADERGDB_H

llvm/lib/ExecutionEngine/Orc/CMakeLists.txt

@@ -2,6 +2,7 @@ add_llvm_component_library(LLVMOrcJIT
   CompileOnDemandLayer.cpp
   CompileUtils.cpp
   Core.cpp
+  DebugObjectManagerPlugin.cpp
   DebugUtils.cpp
   ExecutionUtils.cpp
   IndirectionUtils.cpp
@@ -22,6 +23,7 @@ add_llvm_component_library(LLVMOrcJIT
   SpeculateAnalyses.cpp
   TargetProcessControl.cpp
   ThreadSafeModule.cpp
+  TPCDebugObjectRegistrar.cpp
   TPCDynamicLibrarySearchGenerator.cpp
   TPCEHFrameRegistrar.cpp
   TPCIndirectionUtils.cpp