From 6faf0ccac30381d3a895d505e036a8406640ba0d Mon Sep 17 00:00:00 2001 From: Olga Naumenko <64418523+olganaumenko@users.noreply.github.com> Date: Mon, 5 Dec 2022 13:47:39 +0300 Subject: [PATCH 1/2] RD_for_UnitTestBot.md reviewed --- docs/RD for UnitTestBot.md | 334 ++++++++++++++++++++----------------- 1 file changed, 184 insertions(+), 150 deletions(-) diff --git a/docs/RD for UnitTestBot.md b/docs/RD for UnitTestBot.md index 48064d5424..4df061fa27 100644 --- a/docs/RD for UnitTestBot.md +++ b/docs/RD for UnitTestBot.md @@ -1,192 +1,226 @@ -# Multi-process architecture - -## Table of content -- [Overview](#overview) -- [Lifetimes](#lifetimes) - - [Lifetime](#lifetime) - - [LifetimeDefinition](#lifetimedefinition) -- [Rd](#rd) -- [Rdgen](#rdgen) - - [Model DSL](#model-dsl) - - [Gradle](#gradle) -- [UtBot project](#utbot-project) - - [IDEA process](#idea-process) - - [Engine process](#engine-process) - - [Instrumented process](#instrumented-process) - - [Commons](#useful) +# Multiprocess architecture ## Overview -UtBot consists of 3 different processes: -1. `IDEA process` - the one where plugin part executes. Also can be called `plugin process`, `IDE process`. -2. `Engine process` - process where unit test generation engine executes. -3. `InstrumentedProces` - process where concrete execution takes place. -These processes are built on top of [JetBrains.RD](https://github.com/JetBrains/rd). It is crucial to understand -this library, so it's better describing it first(as there are no documentation about it in repo;)). +UnitTestBot consists of three processes: +1. `IDE process` — the process where the plugin part executes. We also call it the _plugin process_ or the + _IntelliJ IDEA process_. +2. `Engine process` — the process where the test generation engine executes. +3. `Instrumented process` — the process where concrete execution takes place. -RD is mostly about 3 components: +These processes are built on top of the [Reactive distributed communication framework (Rd)](https://github.com/JetBrains/rd) developed by JetBrains. Understanding Rd is crucial for understanding UnitTestBot, so we briefly describe this library here. -1. Lifetimes +To gain an insight into Rd, one should grasp these Rd concepts: +1. Lifetime 2. Rd entities -3. Rdgen +3. `Rdgen` + +## Lifetime concept + +Imagine an object holding resources that should be released upon the object's death. In Java, +`Closeable` and `AutoCloseable` interfaces are introduced to help release resources that the object is holding. +Support for `try`-with-resources in Java and `Closeable.use` in Kotlin are also implemented to assure that the +resources are closed after the execution of the given block. + +Though, releasing resources upon the object's death is still problematic: +1. An object's lifetime can be more complicated than the scope of `Closeable.use`. +2. If `Closeable` is a function parameter, should we close it? +3. Multithreading and concurrency may lead to more complex situations. +4. Considering all these issues, how should we correctly close the objects that depend on some other object's + lifetime? How can we perform this task in a fast and easy way? + +So, Rd introduces the concept of `Lifetime`. + +### `Lifetime` class + +`Lifetime` is an abstract class, with `Lifetime` and `LifetimeDefinition` as its inheritors. The `Lifetime` objects are +instances of `LifetimeDefinition` and thus they can be terminated. You can register callbacks in the `Lifetime` object and then terminate it — all the registered callbacks will be executed upon the termination. + +Though all `Lifetime` objects are instances of `LifetimeDefinition`, there are conventions for using them: +1. Do not cast `Lifetime` to `LifetimeDefinion` unless you are the one who created `LifetimeDefinition`. +2. If you introduce `LifetimeDefinition` somewhere, you should attach it to another `Lifetime` or provide + the code that terminates it. + +A `Lifetime` object has these useful methods: +- `onTermination` executes _lambda_/_closeable_ when the `Lifetime` object is terminated. If an object has been + already terminated, it executes _lambda_/_closeable_ instantly. Termination proceeds on a thread that has invoked + `LifetimeDefinition.terminate`. Callbacks are executed in the **reversed order**, which is _LIFO_: the last added + callback is executed first. +- `onTerminationIfAlive` is the same as `onTermination`, but the callback is executed only if the `Lifetime` + object is `Alive`. +- `executeIfAlive` executes _lambda_ if the `Lifetime` object is `Alive`. This method guarantees that the `Lifetime` + object is not terminated until _lambda_ execution is complete. +- `createdNested` creates the _child_ `LifetimeDefinition` object that is terminated along with the _parent_ one. +- `usingNested` is the same as the `createNested` method, but behaves like the `Closeable.use` pattern. +- `Eternal` prevents a `Lifetime` object from being terminated. +- `Terminated` makes a `Lifetime` object terminated. +- `status` — see more details in the +[LifetimeStatus.kt](https://github.com/JetBrains/rd/blob/9b806ccc770515f6288c778581c54607420c16a7/rd-kt/rd-core/src/main/kotlin/com/jetbrains/rd/util/lifetime/LifetimeStatus.kt) class from the Rd repository. There are three + convenient methods: `IsAlive`, `IsNotAlive`, `IsTerminated`. + +### `LifetimeDefinition` class + +`LifetimeDefinition` instances have the `terminate` method that terminates a `Lifetime` object and invokes all +the registered callbacks. If multiple concurrent terminations occur, the method may sometimes return before +executing all the callbacks because some other thread executes them. + +## Rd entities + +Rd is a light-weight reactive one-to-one RPC protocol, which is cross-language as well as cross-platform. It can +work on the same or different machines via Internet. + +These are Rd entities: +- `Protocol` encapsulates the logic of all Rd communications. All the entities should be bound to `Protocol` before + being used. `Protocol` contains `IScheduler`, which executes a _runnable_ object on a different thread. +- `RdSignal` is an entity allowing one to **fire and forget**. You can add a callback for every received message + via the `advise(lifetime, callback)` method. There are two interfaces: `ISink` that only allows advising for + messages and `ISignal` that can also `fire` events. There is also a `Signal` class with the same behavior + but without remote communication. + +**Important:** if you `advise` and `fire` from the same process, your callback receives _not only_ +messages from the other process, but also the ones you `fire`. + +- `RdProperty` is a stateful property. You can get the current value and advise the callback — an advised + callback is executed on a current value and on every change. +- `RdCall`is the remote procedure call. + +There are `RdSet`, `RdMap`, and other entities. + +An `async` property allows you to `fire` entities from any thread. Otherwise, you would need to do it from +the `Protocol.scheduler` thread: all Rd entities should be bound to the `Protocol` from the `scheduler` thread, or you +would get an exception. + +## `Rdgen` + +`Rdgen` generates custom classes and requests that can be bound to protocol and advised. There is a special model DSL +for it. -Let's dive in each of them: - -## Lifetimes - -Imagine an object having some resources that should be freed -when object dies. For this purpose Java introduced interfaces -```Closeable\AutoCloseable```. Also, 2 helper functions were introduced: try-with-resources and Kotlin's ```Closeable.use```. There are several problems: -1. Object's lifetime can be more complicated that ```Closeable.use``` scope. -2. If function parameter is ```Closeable``` - should you close it? -3. Multiple closes. -4. Concurrent closes. -5. If you have several objects that depends on another object's lifetime - how to correctly close all of them with respect to issues 1-4? How to make it simple, convenient and fast? - -And so Lifetime was introduced. -### Lifetime: -```Lifetime``` is a class, where you can register callbacks and which can be terminated once, thus executing all registered callbacks. - -```Lifetime``` is an abstract class, it's inheritor - ```LifetimeDefinition```. The only difference - ```LifetimeDefinition``` can be terminated. Though all ```Lifetime``` are instances of ```LifetimeDefinition```, there are some conventions: -1. Do not cast ```Lifetime``` to ```LifetimeDefinion``` unless you are the one who created ```LifetimeDefinition```. -2. If you introduce somewhere ```LifetimeDefinition``` - either attach it to another ```Lifetime``` or provide code that terminates it. - -Useful ```Lifetime``` methods: - -- ```onTermination``` - Executes lambda/closeable when lifetime terminates. If already terminated - executes instantly. Termination will proceed on thread that called ```LifetimeDefinition.terminate()```. Callbacks will be executed in ***reversed order***, that is LIFO - last added callback will be executed first. -- ```onTerminationIfAlive``` - same as ```OnTermination```, but callback will not be executed if lifetime is not alive. -- ```executeIfAlive``` - executes lambda if lifetime is alive. This method guarantees that lifetime will not be terminated until lambda completes. -- ```createdNested``` - creates child LifetimeDefinition that will be terminated if parent does. -- ```usingNested``` - same as ```createNested```, but like ```Closeable.use``` pattern. -- ```Eternal``` - lifetime that never terminates. -- ```Terminated``` - lifetime that already terminated. -- ```status``` - see [LifetimeStatus.kt](https://github.com/JetBrains/rd/blob/9b806ccc770515f6288c778581c54607420c16a7/rd-kt/rd-core/src/main/kotlin/com/jetbrains/rd/util/lifetime/LifetimeStatus.kt) in RD repo. There are 3 convenient method: ```IsAlive, IsNotAlive, IsTerminated```. - -### LifetimeDefinition: -- ```terminate``` - terminates ```Lifetime``` and calls all callbacks. Sometimes if multiple concurrent terminations occurred - method will return before executing all callbacks because some other thread is doing this. +### Model DSL -## Rd -Rd is a cross-language, cross-platform, light-weight, reactive, one-to-one rpc protocol. Can work either on the same or different machines via internet. +Examples: +1. [Korifey](https://github.com/korifey/rd_example/blob/main/src/main/kotlin/org/korifey/rd_example/model/Root.kt) — + a simple one. +2. [Rider Unity plugin](https://github.com/JetBrains/resharper-unity/tree/net223/rider/protocol/src/main/kotlin/model) — a complicated one. -Useful entities: -- ```Protocol``` - encapsulation of all logic regarding rd communication. All entities should be bound to protocol before using. Contains ```IScheduler``` which executes runnables on different thread. -- ```RdSignal``` - entity for ___fire and forget___. You can add callback for every received message via ```advise(lifetime, callback)``` method. There are 2 interfaces - ```ISink``` which allows only to advise for messages, and ```ISignal``` which can also ```fire``` events. Also, there is just ```Signal``` class with same behaviour, but without remote communication. +First, you need to define a `Root` object: only one instance of each `Root` can be assigned to `Protocol`. - ___Important___: if you advise and fire from the same process - your callback will receive ___not only___ messages from another process, but also the ones you fire. -- ```RdProperty``` - stateful property. You can get current value, you can advise - advised callback will be executed on current value and on every change. -- ```RdCall``` - remote procedure call. +There is a `Root` extension — `Ext(YourRoot)` — where you can define your own types and model entities. You can assign +multiple `Root` extensions to the `Protocol`. To generate the auxiliary structures, define them as direct fields. -Also there are ```RdSet```, ```RdMap``` and many other. +DSL: +- `structdef` is a structure with fields that cannot be bound to `Protocol`, but can be serialized. This structure + can be `openstruct`, i.e. open for inheritance, and `basestruct`, i.e. abstract. Only `field` can be a member. +- `classdef` is a class that can be bound to a model. It can have `property`, `signal`, `call`, etc. + as members. It is possible to inherit: the class can be `openclass`, `baseclass`. +- `interfacedef` is provided to define interfaces. Use `method` to create a signature. -There is ```async``` property that allows you to ```fire``` entities from any thread. Otherwise you would need to do it from ```Protocol.scheduler``` thread. All rd entities should be at first +You can use `extends` and `implements` to implement inheritance. -## Rdgen -Generates custom classes and requests which can be bound to protocol and advised. There is special model DSL for it. -### Model DSL -Eexample: -1. [Korifey](https://github.com/korifey/rd_example/blob/main/src/main/kotlin/org/korifey/rd_example/model/Root.kt) - quite simple -2. [Rider Unity plugin](https://github.com/JetBrains/resharper-unity/tree/net223/rider/protocol/src/main/kotlin/model) - complicated one +_Note:_ `Rdgen` can generate models for C# and C++. Their structs and classes have different behavior. -First you need to define ```Root``` object - only one instance of each root can be assigned to protocol. +Rd entities — only in bindable models (`Ext`, `classdef`): +- `property` +- `signal` +- `source` +- `sink` +- `array` and `immutablelist` -Then there is root extension ```Ext(YourRoot)``` where you can define your own types and model entities. You can assign multiple extensions of root for the protocol. Auxillary structures can be defined as direct fields - they will be also generated. +Useful properties in DSL entities: +- `async` — the same as `async` in Rd entities +- `docs` — provides KDoc/Javadoc documentation comments for the generated entity -DSL: -- ```structdef``` - structure with fields that cannot be bound to protocol, but can be serialized. Can be open for inheritace - ```openstruct```, can be abstract - ```basestruct```. Can have only ```field``` as member. -- ```classdef``` - class that can be bould to model. Can also have ```property```, ```signal```, ```call``` e.t.c. as members. It is possible to do inheritance - ```openclass```, ```baseclass```. -- ```interfacedef``` - to define interfaces. Use ```method``` to create signature. - - You can use ```extends``` and ```implements``` to work with inheritance. - - ___N.B.___ - rdgen can generate models also for C# and C++. Their structs and classes have a little different behaviour. -- Rd entities - only in bindable models(```Ext```, ```classdef```): - - ```property``` - - ```signal``` - - ```source``` - - ```sink``` - - ```array``` and ```immutablelist``` -- Useful properties in dsl entities: - - async - as ```async``` in RD entities - - docs - provides kdoc/javadoc for generated entity ### Gradle + [Example](https://github.com/korifey/rd_example/blob/main/build.gradle) -```RdGenExtension``` configurates Rdgen, useful properties: -- ```sources``` - folders with dsl .kt files. If not present, scan classpath for inheritors of ```Root``` and ```Ext```. -- ```hashfile``` - folder to store hash file ```.rdgen``` for incremental generation. -- ```packages``` - java package names to search toplevels, delimited by ','. Example: ```com.jetbrains.rd.model.nova,com,org```. -- Configuring model generation with method ```RdGenExtension.generator```: - - root - for which root you are declaring this generator - - namespace - which namespace should be used in generated source. In kotlin it configures generated packaged name. - - directory - where to put generated files. - - transform - can be ```symmetric```, ```asis``` and ```reversed```. This allows to configure model interface differently for client-server scenarios. +`RdGenExtension` configurates `Rdgen`. The properties are: +- `sources` — the folders with DSL `.kt` files. If there are no `sources`, scan classpath for the inheritors of `Root` + and `Ext`. +- `hashfile` — a folder to store the `.rdgen` hash file for incremental generation. +- `packages` — `.java` package names to search in toplevels, delimited by `,`. Example: `com.jetbrains.rd.model.nova,com,org`. + +Configure model generation with the `RdGenExtension.generator` method: +- `root` — for which root this generator is declared. +- `namespace` — which namespace should be used in the generated source. In Kotlin, it configures the generated package + name. +- `directory` — where to put the generated files. +- `transform` — can be `symmetric`, `asis` and `reversed`. It allows to configure different model interfaces for + various client-server scenarios. _Note:_ in 99% of cases you should use `symmetric`. If you really need another option, consult with someone. +- `language` — can be `kotlin`, `cpp` or `csharp`. - P.S. This is legacy from distant past, in 99% of time you should use ```symmetric```. In 1% chance - consult with somebody if you really need another option. - - language - can be ```kotlin```, ```cpp``` and ```csharp```. +## UnitTestBot project -## UtBot project +The `utbot-rd` Gradle project contains model sources in `rdgenModels`. You can find them at +[`utbot-rd/src/main/rdgen/org/utbot/rd/models`](../utbot-rd/src/main/rdgen/org/utbot/rd/models). -There is another gradle project ```utbot-rd``` which contains model sources in ```rdgenModels```. -Look at [```utbot-rd/src/main/rdgen/org/utbot/rd/models```](../utbot-rd/src/main/rdgen/org/utbot/rd/models) +### IDE process -### IDEA process -Uses bundled JetBrains JDK. Code in `utbot-intellij` ___must___ be compatible will all JDKs and plugin SDKs, which are used by our officially supported IntellijIDEA versions. -See [`utbot-intellij/build.gradle.kts`](../utbot-intellij/build.gradle.kts), parts `sinceBuild` and `untilBuild`. +An _IDE process_ uses bundled JetBrains JDK. Code in `utbot-intellij` _**must**_ be compatible will all JDKs and plugin +SDKs, used by our officially supported Intellij IDEA versions. +See `sinceBuild` and `untilBuild` in [`utbot-intellij/build.gradle.kts`](../utbot-intellij/build.gradle.kts). -Starts `Engine process`. Maintains `UtSettings` instance in memory and updates it from IDEA. -Other processes ask this process for settings via RD RPC. +The _IDE process_ starts the _Engine process_. The _IDE process_ keeps the `UtSettings` instance in memory and gets updates for it from Intellij IDEA. The other processes "ask" the _IDE process_ about settings via Rd RPC. ### Engine process -`TestCaseGenerator` and `UtBotSymbolicEngine` run here. Process classpath contains all plugin jars(more precisely - it uses plugin classpath). +`TestCaseGenerator` and `UtBotSymbolicEngine` run here, in the _Engine process_. The process classpath contains all +the plugin JAR-files (it uses the plugin classpath). -___Must___ run on JDK, which uses project we analyze. Otherwise there will be numerous problems with code analysis, soot, reflection and -devirgention of generated code Java API. +The _Engine process_ _**must**_ run on the JDK that is used in the project under analysis. Otherwise, there will be +numerous problems with code analysis, `soot`, Reflection, and the divergence of the generated code Java API will occur. -Currently, it is prohibited to run more than 1 generation process simultaneously(something with native libs). -However, logging for processes relies on that fact, so they can exclusively write to log file. +Currently, it is prohibited to run more than **one** generation process simultaneously (the limitation is related to +the characteristics of the native libraries). The process logging mechanism relies on +that fact, so UnitTestBot processes can exclusively write to a log file. -IDEA starting point - class [`EngineProcess`](../utbot-intellij/src/main/kotlin/org/utbot/intellij/plugin/process/EngineProcess.kt). -Process start file - [`EngineProcessMain`](../utbot-framework/src/main/kotlin/org/utbot/framework/process/EngineProcessMain.kt). -Starts `Instrumented process`. +The starting point in the _IDE process_ is the +[`EngineProcess`](../utbot-intellij/src/main/kotlin/org/utbot/intellij/plugin/process/EngineProcess.kt) class. +The _Engine process_ start file is +[`EngineProcessMain`](../utbot-framework/src/main/kotlin/org/utbot/framework/process/EngineProcessMain.kt). +The _Engine process_ starts the _Instrumented process_. ### Instrumented process -Start points at `Engine process`: classes [`InstrumentedProcess`](../utbot-instrumentation/src/main/kotlin/org/utbot/instrumentation/rd/InstrumentedProcess.kt) and [`ConcreteExecutor`](../utbot-instrumentation/src/main/kotlin/org/utbot/instrumentation/ConcreteExecutor.kt). -First one is state encapsulation, second is used to implement request logic for concrete execution. - -Runs on the same JDK as `Engine process` to erase deviation from `Engine process`. -Sometimes might unexpectedly die due concrete execution. +The starting points in the _Engine process_ are the +[`InstrumentedProcess`](../utbot-instrumentation/src/main/kotlin/org/utbot/instrumentation/rd/InstrumentedProcess.kt) +and the [`ConcreteExecutor`](../utbot-instrumentation/src/main/kotlin/org/utbot/instrumentation/ConcreteExecutor.kt) +classes. The first one encapsulates the state, while the second one implements the request logic for concrete execution. +The _Instrumented process_ runs on the same JDK as the _Engine process_ to prevent deviation from the _Engine process_. +Sometimes the _Instrumented process_ may unexpectedly die due to concrete execution. -### Useful +### Useful info -1. if you need to use rd somewhere - add following dependencies: +1. If you need to use Rd, add the following dependencies: ``` implementation group: 'com.jetbrains.rd', name: 'rd-framework', version: rdVersion implementation group: 'com.jetbrains.rd', name: 'rd-core', version: rdVersion ``` -2. There are some useful classes in `utbot-rd` to work with processes & rd: - - ```LifetimedProcess``` - binds ```Lifetime``` to process. If process dies - lifetime terminates and vice versa. You can terminate lifetime manually - this will destroy process. - - ```ProcessWithRdServer``` - also starts Rd server and waits for connection. - - `ClientProtocolBuilder` - use in client process to correctly connect to `ProcessWithRdServer`. -3. How ```ProcessWithRdServer``` communication works: - - Choose free port - - Create client process, pass port as argument - - Both processes create protocols, bind model and setup callbacks - - Server process cannot send messages before child creates protocol, otherwise messages will be lost. So client process needs to signal that he is ready. - - Client process creates special file in temp dir, that is observed by parent process. - - When parent process spots file - he deletes it, and then sends special message for client process confirming communication succeed. - - Only after client process answer reaches server - then processes are ready. -4. How to write custom RPC commands - - Add new ```call``` in some model, for example in ```EngineProcessModel```. - - Regenerate models: there are special gradle tasks for it in `utbot-rd/build.gradle` file. - - Add callback for new ```call``` in corresponding start files, for example in `EngineProcessMain.kt`. - - ___Important___ - do not add [`Rdgen`](https://mvnrepository.com/artifact/com.jetbrains.rd/rd-gen) as implementation dependency, it breaks some `.jar`s as it contains `kotlin-compiler-embeddable`. -5. Logs & Debug - - Logs - [inter process logging](./contributing/InterProcessLogging.md) - - Debug - [inter process debugging](./contributing/InterProcessDebugging.md) -6. Custom protocol marshalling types - Do not spend time on it until UtModels would get simpler, for example Kotlinx.serialization compatible. +2. There are useful classes in `utbot-rd` to work with Rd and processes: + - `LifetimedProcess` binds a `Lifetime` instance to a process. If the process dies, the `Lifetime` instance + terminates and vice versa. You can terminate the `Lifetime` instance manually — this will destroy the process. + - `ProcessWithRdServer` starts the Rd server and waits for connection. + - `ClientProtocolBuilder` — you can use it in a client process to correctly connect to `ProcessWithRdServer`. +3. How `ProcessWithRdServer` communication works: + - Choose a free port. + - Create a client process, pass the port as an argument. + - Both processes create protocols. Bind the model and setup callbacks. + - A server process cannot send messages until the _child_ creates a protocol (otherwise, messages are lost), so + the client process have to signal that it is ready. + - The client process creates a special file in the `temp` directory, which is observed by a _parent_ process. + - When the parent process spots the file, it deletes this file and sends a special message to the client process + confirming communication success. + - Only when the answer of the client process reaches the server, the processes are ready. +4. How to write custom RPC commands: + - Add new `call` in a model, for example, in `EngineProcessModel`. + - Re-generate models: there are special Gradle tasks for this in the `utbot-rd/build.gradle` file. + - Add a callback for the new `call` in the corresponding start files, for example, in `EngineProcessMain.kt`. + - **Important**: do not add [`Rdgen`](https://mvnrepository.com/artifact/com.jetbrains.rd/rd-gen) as + an implementation dependency — it breaks some JAR-files as it contains `kotlin-compiler-embeddable`. +5. Logging & debugging: + - [Interprocess logging](./contributing/InterProcessLogging.md) + - [Interprocess debugging](./contributing/InterProcessDebugging.md) +6. Custom protocol marshalling types: do not spend time on it until `UtModels` get simpler, e.g. compatible with + `kotlinx.serialization`. From fa209591a368fe94debbe858b05e2e075ff985b8 Mon Sep 17 00:00:00 2001 From: Olga Naumenko <64418523+olganaumenko@users.noreply.github.com> Date: Mon, 5 Dec 2022 20:01:30 +0300 Subject: [PATCH 2/2] Wordings fixed, requested changes implemented --- docs/RD for UnitTestBot.md | 51 ++++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 21 deletions(-) diff --git a/docs/RD for UnitTestBot.md b/docs/RD for UnitTestBot.md index 4df061fa27..c70de8525f 100644 --- a/docs/RD for UnitTestBot.md +++ b/docs/RD for UnitTestBot.md @@ -8,7 +8,8 @@ UnitTestBot consists of three processes: 2. `Engine process` — the process where the test generation engine executes. 3. `Instrumented process` — the process where concrete execution takes place. -These processes are built on top of the [Reactive distributed communication framework (Rd)](https://github.com/JetBrains/rd) developed by JetBrains. Understanding Rd is crucial for understanding UnitTestBot, so we briefly describe this library here. +These processes are built on top of the [Reactive distributed communication framework (Rd)](https://github.com/JetBrains/rd) developed by JetBrains. Rd plays a crucial role in UnitTestBot machinery, so we briefly +describe this library here. To gain an insight into Rd, one should grasp these Rd concepts: 1. Lifetime @@ -31,36 +32,43 @@ Though, releasing resources upon the object's death is still problematic: So, Rd introduces the concept of `Lifetime`. -### `Lifetime` class +### `Lifetime` -`Lifetime` is an abstract class, with `Lifetime` and `LifetimeDefinition` as its inheritors. The `Lifetime` objects are -instances of `LifetimeDefinition` and thus they can be terminated. You can register callbacks in the `Lifetime` object and then terminate it — all the registered callbacks will be executed upon the termination. +_Note:_ the described relationships and behavior refer to the JVM-related part of Rd. + +`Lifetime` is an abstract class, with `LifetimeDefinition` as its inheritor. `LifetimeDefinition` +has only one difference from its parent: `LifetimeDefinition` can be terminated. +Each `Lifetime` variable is an instance of `LifetimeDefinition` (we later call it "`Lifetime` instance"). You +can register callbacks in this `Lifetime` instance — all of them will be executed upon the termination. Though all `Lifetime` objects are instances of `LifetimeDefinition`, there are conventions for using them: 1. Do not cast `Lifetime` to `LifetimeDefinion` unless you are the one who created `LifetimeDefinition`. 2. If you introduce `LifetimeDefinition` somewhere, you should attach it to another `Lifetime` or provide the code that terminates it. -A `Lifetime` object has these useful methods: -- `onTermination` executes _lambda_/_closeable_ when the `Lifetime` object is terminated. If an object has been +A `Lifetime` instance has these useful methods: +- `onTermination` executes _lambda_/_closeable_ when the `Lifetime` instance is terminated. If an instance has been already terminated, it executes _lambda_/_closeable_ instantly. Termination proceeds on a thread that has invoked `LifetimeDefinition.terminate`. Callbacks are executed in the **reversed order**, which is _LIFO_: the last added callback is executed first. - `onTerminationIfAlive` is the same as `onTermination`, but the callback is executed only if the `Lifetime` - object is `Alive`. -- `executeIfAlive` executes _lambda_ if the `Lifetime` object is `Alive`. This method guarantees that the `Lifetime` - object is not terminated until _lambda_ execution is complete. -- `createdNested` creates the _child_ `LifetimeDefinition` object that is terminated along with the _parent_ one. + instance is `Alive`. +- `executeIfAlive` executes _lambda_ if the `Lifetime` instance is `Alive`. This method guarantees that the `Lifetime` + instance is alive (i.e. will not be terminated) during the whole time of _lambda_ execution. +- `createdNested` creates the _child_ `LifetimeDefinition` instance: it can be terminated if the _parent_ + instance is terminated as well; or it can be terminated separately, while the parent instance stays alive. - `usingNested` is the same as the `createNested` method, but behaves like the `Closeable.use` pattern. -- `Eternal` prevents a `Lifetime` object from being terminated. -- `Terminated` makes a `Lifetime` object terminated. -- `status` — see more details in the + +See also: +- `Lifetime.Eternal` is a global `Lifetime` instance that is never terminated. +- `Lifetime.Terminated` is a global `Lifetime` instance that has been already terminated. +- `status` — find more details in the [LifetimeStatus.kt](https://github.com/JetBrains/rd/blob/9b806ccc770515f6288c778581c54607420c16a7/rd-kt/rd-core/src/main/kotlin/com/jetbrains/rd/util/lifetime/LifetimeStatus.kt) class from the Rd repository. There are three convenient methods: `IsAlive`, `IsNotAlive`, `IsTerminated`. -### `LifetimeDefinition` class +### `LifetimeDefinition` -`LifetimeDefinition` instances have the `terminate` method that terminates a `Lifetime` object and invokes all +`LifetimeDefinition` instances have the `terminate` method that terminates a `Lifetime` instance and invokes all the registered callbacks. If multiple concurrent terminations occur, the method may sometimes return before executing all the callbacks because some other thread executes them. @@ -69,9 +77,9 @@ executing all the callbacks because some other thread executes them. Rd is a light-weight reactive one-to-one RPC protocol, which is cross-language as well as cross-platform. It can work on the same or different machines via Internet. -These are Rd entities: +These are some of Rd entities: - `Protocol` encapsulates the logic of all Rd communications. All the entities should be bound to `Protocol` before - being used. `Protocol` contains `IScheduler`, which executes a _runnable_ object on a different thread. + being used. `Protocol` contains `IScheduler`, which executes a _runnable_ instance on a different thread. - `RdSignal` is an entity allowing one to **fire and forget**. You can add a callback for every received message via the `advise(lifetime, callback)` method. There are two interfaces: `ISink` that only allows advising for messages and `ISignal` that can also `fire` events. There is also a `Signal` class with the same behavior @@ -82,7 +90,7 @@ messages from the other process, but also the ones you `fire`. - `RdProperty` is a stateful property. You can get the current value and advise the callback — an advised callback is executed on a current value and on every change. -- `RdCall`is the remote procedure call. +- `RdCall` is the remote procedure call. There are `RdSet`, `RdMap`, and other entities. @@ -137,7 +145,8 @@ Useful properties in DSL entities: - `sources` — the folders with DSL `.kt` files. If there are no `sources`, scan classpath for the inheritors of `Root` and `Ext`. - `hashfile` — a folder to store the `.rdgen` hash file for incremental generation. -- `packages` — `.java` package names to search in toplevels, delimited by `,`. Example: `com.jetbrains.rd.model.nova,com,org`. +- `packages` — Java package names to search in toplevels, delimited by `,`. Example: `com.jetbrains.rd.model.nova,com, + org`. Configure model generation with the `RdGenExtension.generator` method: - `root` — for which root this generator is declared. @@ -167,7 +176,7 @@ The _IDE process_ starts the _Engine process_. The _IDE process_ keeps the `UtSe the plugin JAR-files (it uses the plugin classpath). The _Engine process_ _**must**_ run on the JDK that is used in the project under analysis. Otherwise, there will be -numerous problems with code analysis, `soot`, Reflection, and the divergence of the generated code Java API will occur. +numerous problems with code analysis, `soot`, _Reflection_, and the divergence of the generated code Java API will occur. Currently, it is prohibited to run more than **one** generation process simultaneously (the limitation is related to the characteristics of the native libraries). The process logging mechanism relies on @@ -205,7 +214,7 @@ Sometimes the _Instrumented process_ may unexpectedly die due to concrete execut 3. How `ProcessWithRdServer` communication works: - Choose a free port. - Create a client process, pass the port as an argument. - - Both processes create protocols. Bind the model and setup callbacks. + - Both processes create protocols, bind the model and setup callbacks. - A server process cannot send messages until the _child_ creates a protocol (otherwise, messages are lost), so the client process have to signal that it is ready. - The client process creates a special file in the `temp` directory, which is observed by a _parent_ process.