Skip to content

Interprocess debugging doc reviewed; minor fixes for RD doc #1465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 6, 2022
Merged

Interprocess debugging doc reviewed; minor fixes for RD doc #1465

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions docs/RD for UnitTestBot.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ A `Lifetime` instance has these useful methods:
instance is alive (i.e. will not be terminated) during the whole time of _lambda_ execution.
- `createdNested` creates the _child_ `LifetimeDefinition` instance: it can be terminated if the _parent_
instance is terminated as well; or it can be terminated separately, while the parent instance stays alive.
- `usingNested` is the same as the `createNested` method, but behaves like the `Closeable.use` pattern.
- `usingNested` is the same as the `createNested` method but behaves like the `Closeable.use` pattern.

See also:
- `Lifetime.Eternal` is a global `Lifetime` instance that is never terminated.
Expand All @@ -74,8 +74,8 @@ executing all the callbacks because some other thread executes them.

## Rd entities

Rd is a light-weight reactive one-to-one RPC protocol, which is cross-language as well as cross-platform. It can
work on the same or different machines via Internet.
Rd is a lightweight reactive one-to-one RPC protocol, which is cross-language as well as cross-platform. It can
work on the same or different machines via the Internet.

These are some of Rd entities:
- `Protocol` encapsulates the logic of all Rd communications. All the entities should be bound to `Protocol` before
Expand All @@ -89,7 +89,7 @@ These are some of Rd entities:
messages from the other process, but also the ones you `fire`.

- `RdProperty` is a stateful property. You can get the current value and advise the callback — an advised
callback is executed on a current value and on every change.
callback is executed on a current value and every change.
- `RdCall` is the remote procedure call.

There are `RdSet`, `RdMap`, and other entities.
Expand All @@ -112,11 +112,11 @@ Examples:

First, you need to define a `Root` object: only one instance of each `Root` can be assigned to `Protocol`.

There is a `Root` extension — `Ext(YourRoot)` — where you can define your own types and model entities. You can assign
There is a `Root` extension — `Ext(YourRoot)` — where you can define custom types and model entities. You can assign
multiple `Root` extensions to the `Protocol`. To generate the auxiliary structures, define them as direct fields.

DSL:
- `structdef` is a structure with fields that cannot be bound to `Protocol`, but can be serialized. This structure
- `structdef` is a structure with fields that cannot be bound to `Protocol` but can be serialized. This structure
can be `openstruct`, i.e. open for inheritance, and `basestruct`, i.e. abstract. Only `field` can be a member.
- `classdef` is a class that can be bound to a model. It can have `property`, `signal`, `call`, etc.
as members. It is possible to inherit: the class can be `openclass`, `baseclass`.
Expand All @@ -141,7 +141,7 @@ Useful properties in DSL entities:

[Example](https://github.com/korifey/rd_example/blob/main/build.gradle)

`RdGenExtension` configurates `Rdgen`. The properties are:
`RdGenExtension` configures `Rdgen`. The properties are:
- `sources` — the folders with DSL `.kt` files. If there are no `sources`, scan classpath for the inheritors of `Root`
and `Ext`.
- `hashfile` — a folder to store the `.rdgen` hash file for incremental generation.
Expand All @@ -153,8 +153,8 @@ Configure model generation with the `RdGenExtension.generator` method:
- `namespace` — which namespace should be used in the generated source. In Kotlin, it configures the generated package
name.
- `directory` — where to put the generated files.
- `transform` — can be `symmetric`, `asis` and `reversed`. It allows to configure different model interfaces for
various client-server scenarios. _Note:_ in 99% of cases you should use `symmetric`. If you really need another option, consult with someone.
- `transform` — can be `symmetric`, `asis`, and `reversed`. It allows configuring of different model interfaces for
various client-server scenarios. _Note:_ in 99% of cases you should use `symmetric`. If you need another option, consult with someone.
- `language` — can be `kotlin`, `cpp` or `csharp`.

## UnitTestBot project
Expand All @@ -173,7 +173,7 @@ The _IDE process_ starts the _Engine process_. The _IDE process_ keeps the `UtSe
### Engine process

`TestCaseGenerator` and `UtBotSymbolicEngine` run here, in the _Engine process_. The process classpath contains all
the plugin JAR-files (it uses the plugin classpath).
the plugin JAR files (it uses the plugin classpath).

The _Engine process_ _**must**_ run on the JDK that is used in the project under analysis. Otherwise, there will be
numerous problems with code analysis, `soot`, _Reflection_, and the divergence of the generated code Java API will occur.
Expand Down Expand Up @@ -208,28 +208,28 @@ Sometimes the _Instrumented process_ may unexpectedly die due to concrete execut
```
2. There are useful classes in `utbot-rd` to work with Rd and processes:
- `LifetimedProcess` binds a `Lifetime` instance to a process. If the process dies, the `Lifetime` instance
terminates and vice versa. You can terminate the `Lifetime` instance manually — this will destroy the process.
- `ProcessWithRdServer` starts the Rd server and waits for connection.
terminates, and vice versa. You can terminate the `Lifetime` instance manually — this will destroy the process.
- `ProcessWithRdServer` starts the Rd server and waits for the connection.
- `ClientProtocolBuilder` — you can use it in a client process to correctly connect to `ProcessWithRdServer`.
3. How `ProcessWithRdServer` communication works:
- Choose a free port.
- Create a client process, pass the port as an argument.
- Create a client process and pass the port as an argument.
- Both processes create protocols, bind the model and setup callbacks.
- A server process cannot send messages until the _child_ creates a protocol (otherwise, messages are lost), so
the client process have to signal that it is ready.
the client process has to signal that it is ready.
- The client process creates a special file in the `temp` directory, which is observed by a _parent_ process.
- When the parent process spots the file, it deletes this file and sends a special message to the client process
confirming communication success.
- Only when the answer of the client process reaches the server, the processes are ready.
4. How to write custom RPC commands:
- Add new `call` in a model, for example, in `EngineProcessModel`.
- Add a new `call` in a model, for example, in `EngineProcessModel`.
- Re-generate models: there are special Gradle tasks for this in the `utbot-rd/build.gradle` file.
- Add a callback for the new `call` in the corresponding start files, for example, in `EngineProcessMain.kt`.
- **Important**: do not add [`Rdgen`](https://mvnrepository.com/artifact/com.jetbrains.rd/rd-gen) as
an implementation dependency — it breaks some JAR-files as it contains `kotlin-compiler-embeddable`.
an implementation dependency — it breaks some JAR files as it contains `kotlin-compiler-embeddable`.
5. Logging & debugging:
- [Interprocess logging](./contributing/InterProcessLogging.md)
- [Interprocess debugging](./contributing/InterProcessDebugging.md)
6. Custom protocol marshalling types: do not spend time on it until `UtModels` get simpler, e.g. compatible with
6. Custom protocol marshaling types: do not spend time on it until `UtModels` get simpler, e.g. compatible with
`kotlinx.serialization`.

132 changes: 73 additions & 59 deletions docs/contributing/InterProcessDebugging.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,107 +2,121 @@

### Background

We have split the UnitTestBot machinery into three processes. See [doc about processes](../RD%20for%20UnitTestBot.md).
This approach has improved UnitTestBot capabilities, e.g. provided support for various JVMs and scenarios, but also complicated the debugging flow.
We have split the UnitTestBot machinery into three processes. See the [document on UnitTestBot multiprocess
architecture](../RD%20for%20UnitTestBot.md).
This approach has improved UnitTestBot capabilities, e.g., provided support for various JVMs and scenarios but also
complicated the debugging flow.

These are UnitTestBot processes (according to the execution order):

* IDE process
* Engine process
* Instrumented process
* _IDE process_
* _Engine process_
* _Instrumented process_

Usually, main problems happen in the Engine process, but it is not the process we run first.
The most straightforward way to debug the Engine process is the following.
Usually, the main problems happen in the _Engine process_, but it is not the process we run first.
See how to debug UnitTestBot processes effectively.

### Enable Debugging
### Enable debugging

IDE debugging is pretty straightforward - start `runIde` task in `utbot-intellij` project from IDEA with debug.
Debugging the _IDE process_ is pretty straightforward: start the debugger session (**Shift+F9**) for the `runIde`
Gradle task in `utbot-intellij` project from your IntelliJ IDEA.

To debug the _Engine process_ and the _Instrumented process_, you need to enable the debugging options:
1. Open [`UtSettings.kt`](../../utbot-framework-api/src/main/kotlin/org/utbot/framework/UtSettings.kt).
2. There are two similar options: `runEngineProcessWithDebug` and `runInstrumentedProcessWithDebug` — enable the
relevant one(s). There are two ways to do this:
* You can create the `~/.utbot/settings.properties` file and write the following:

For engine and instrumented processes you need to enable some options:
1. Open [`UtSettings.kt`](../../utbot-framework-api/src/main/kotlin/org/utbot/framework/UtSettings.kt)
2. There are 2 similar options: `runEngineProcessWithDebug` and `runInstrumentedProcessWithDebug`.
3. Enable for processes you need to debug. It can be done in 2 ways:
* Can create `~/.utbot/settings.properties` file and write following:
```
runEngineProcessWithDebug=true
runInstrumentedProcessWithDebug=true
```
After you will need to restart IDEA you want to debug.
* ***Discouraged***: change in source file, but this will involve moderate project recompilation.
4. Additionally, you can set additional options for JDWP agent if debug is enabled:
* `engineProcessDebugPort` and `instrumentedProcessDebugPort` - port for debugging.
Default values - 5005 for Engine and 5006 for Instrumented processes.
* `suspendEngineProcessExecutionInDebugMode` and `suspendInstrumentedProcessExecutionInDebugMode` - whether JDWP agent should
suspend process until debugger is connected.

More formally, if debug is enabled following switch is added to engine process JVM at start by default:

Then restart the IntelliJ IDEA instance you want to debug.

* **Discouraged**: you can change the options in the source file, but this will involve moderate project
recompilation.
4. You can set additional options for the Java Debug Wire Protocol (JDWP) agent if debugging is enabled:
* `engineProcessDebugPort` and `instrumentedProcessDebugPort` are the ports for debugging.

Default values:
- 5005 for the _Engine process_
- 5006 for the _Instrumented process_

* `suspendEngineProcessExecutionInDebugMode` and `suspendInstrumentedProcessExecutionInDebugMode` define whether
the JDWP agent should suspend the process until the debugger is connected.

More formally, if debugging is enabled, the following switch is added to the _Engine process_ JVM at the start by
default:
```
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,quiet=y,address=5005"
"-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,quiet=y,address=5005"
```
These options regulate values for parts `suspend` and `address`, for example with following in `~/.utbot/settings.properties`:

These options set `suspend` and `address` values. For example, with the following options in `~/.utbot/settings.properties`:
```
runEngineProcessWithDebug=true
engineProcessDebugPort=12345
suspendEngineProcessExecutionInDebugMode=false
```
result switch will be:
the resulting switch will be:
```
-agentlib:jdwp=transport=dt_socket,server=n,suspend=y,quiet=y,address=12345"
"-agentlib:jdwp=transport=dt_socket,server=n,suspend=n,quiet=y,address=12345"
```
See `org.utbot.intellij.plugin.process.EngineProcess.Companion.getDebugArgument`
5. For information about logs - see [this](InterProcessLogging.md).
See `org.utbot.intellij.plugin.process.EngineProcess.Companion.debugArgument` for switch implementation.
5. For information about logs, refer to the [Interprocess logging](InterProcessLogging.md) guide.

### Run configurations for debugging the Engine process

There are 3 basic run configurations:
1. `Run IDE` - run plugin in IDEA
2. `Utility configuration/Listen for Instrumented Process` - listen on 5006 port if instrumented process is available for debug
3. `Utility configuration/Listen for Engine Process` - listen on 5005 port if engine process is available for debug
There are three basic run configurations:
1. `Run IDE` configuration allows running the plugin in IntelliJ IDEA.
2. `Utility Configurations/Listen for Instrumented Process` configuration allows listening to port 5006 to check if
the _Instrumented process_ is available for debugging.
3. `Utility Configurations/Listen for Engine Process` configuration allows listening to port 5005 to check if the _Engine process_ is available for debugging.

On top of them, there are 3 compound run configurations for debugging:
1. `Debug Engine Process` and `Debug Instrumented Process` - combo for debug IDE and selected process
3. `Debug All` - debug all 3 processes.
On top of them, there are three compound run configurations for debugging:
1. `Debug Engine Process` and `Debug Instrumented Process` — a combination for debugging the _IDE process_ and
the selected process.
3. `Debug All` — a combination for debugging all three processes.

For debug configurations to work you need to provide required properties in `~/.utbot/settings.properties`.
If you either change port and/or suspend mode - do review utility configuration to change default values as well.
To make debug configurations work properly, you need to set the required properties in `~/.utbot/settings.properties`. If you change the _port number_ and/or the _suspend mode_, do change these default values in the corresponding Utility Configuration.

### How to debug

Let's see through example of how to debug IDE to engine process communication.
Let's walk through an example illustrating how to debug the "_IDE process_ → _Engine process_" communication.

1. In your current IntelliJ IDEA with source, use breakpoints to define where the program needs to be stopped. For example, set the breakpoints at `EngineProcess.generate`
and somewhere in `watchdog.wrapActiveCall(generate)`.
2. Select `Debug Engine Process` configuration, add required parameters to `~/.utbot/settings.properties` and start debug.
3. Generate tests with UnitTestBot in the debug IDE instance.
4. The debug IDE instance will stop generation (if you have not changed the debug parameters). If you take no action, test generation will be cancelled by timeout.
5. When the Engine process started (build processes have finished, and the progress bar says: _"Generate tests: read
classes"_), there will be
6. Wait for the program to be suspended upon reaching the first breakpoint in Engine proces.
7. If symbolic execution is not turned on - часть магии может нахуй не случиться
1. In your current IntelliJ IDEA with source code, use breakpoints to define where the program needs to be stopped. For example, set the breakpoints at `EngineProcess.generate` and somewhere in `watchdog.wrapActiveCall(generate)`.
2. Select the `Debug Engine Process` configuration, add the required parameters to `~/.utbot/settings.properties` and
start the debugger session.
3. Generate tests with UnitTestBot in the debug IDE instance. Make sure symbolic execution is turned on, otherwise some processes do not even start.
4. The debug IDE instance will stop generation (if you have not changed the debug parameters). If you take no action, test generation will be canceled by timeout.
5. When the _Engine process_ has started (build processes have finished, and the progress bar says: _"Generate
tests: read classes"_), there will be another debug window — "Listen for Engine Process", — which automatically
connects and starts debugging.
6. Wait for the program to be suspended upon reaching the first breakpoint in the _Engine process_.

### Interprocess call mapping

Now you are standing on a breakpoint in the IDE process, for example, the process stopped on:
Now you are standing on a breakpoint in the _IDE process_, for example, the process stopped on:

`EngineProcess.generate()`
EngineProcess.generate()

If you would go along execution, it reaches the next line (you are still in the IDE process):
If you go along the execution, it reaches the next line (you are still in the _IDE process_):

`engineModel.generate.startBlocking(params)`
engineModel.generate.startBlocking(params)

It seems that the test generation itself should occur in the Engine process and there should be an entry point in the Engine process.
How can we find it?
It seems that test generation itself should occur in the _Engine process_ and there should be an entry point in the _Engine process_.
How can we find it?

Standing on the breakpoint `engineModel.generate.startBlocking(params)`, you may right-click in IDE on `EngineProcessModel.generate` and **Go to Declaration or
Usage**. This would navigate to the definition of `RdCall` (which is responsible for cross-process communication) in file `EngineProcesModel.Generated.kt`.
Standing on the breakpoint at `engineModel.generate.startBlocking(params)`, right-click on
`EngineProcessModel.generate` and **Go to** > **Declaration or Usages**. This navigates to the `RdCall` definition (which is
responsible for cross-process communication) in the `EngineProcesModel.Generated.kt` file.

Now **Find Usages** for `EngineProcessModel.generate` and see the point where `RdCall` is passed to the next method:

watchdog.wrapActiveCall(generate)

This is the point where `RdCall` is called in the Engine process.
This is the point where `RdCall` is called in the _Engine process_.

Actually you could have skipped the previous step and used **Find Usages** right away, but it is useful to know where `RdCall` is defined.
You could have skipped the previous step and used **Find Usages** right away, but it is useful to know
where `RdCall` is defined.

If you are interested in the trailing lambda of `watchdog.wrapActiveCall(generate)`, set the breakpoint here.