|
1 |
| -# Result & Error Handling API of the Instrumented Process |
| 1 | +# Instrumented process API: handling errors and results |
2 | 2 |
|
3 |
| -## Terminology |
| 3 | +In UnitTestBot Java, there are three processes: |
| 4 | +* IDE process |
| 5 | +* Engine process |
| 6 | +* Instrumented process |
4 | 7 |
|
5 |
| -- The _instrumented process_ is an external process used for the isolated invocation. |
6 |
| -- The `ConcreteExecutor` is a class which provides smooth and concise interaction with the _instrumented process_. It works in the _main process_. |
7 |
| -- A client is an object which directly uses the `ConcreteExecutor`, so it works in the _main process_ as well. |
8 |
| -- An _Instrumentation_ is an object which has to be passed to the `ConcreteExecutor`. It defines the logic of invocation and bytecode instrumentation in the _instrumented process_. |
| 8 | +The IDE process launches the plugin so a user can request test generation. |
| 9 | +Upon the user request, the Engine process is initiated — it is responsible for the input values generation. |
9 | 10 |
|
10 |
| -## Common |
| 11 | +Here, in the Engine process, there is a `ConcreteExecutor` class, |
| 12 | +conveying the generated input values to the `InstrumentedProcess` class. |
| 13 | +The `InstrumentedProcess` class creates the third physical process — |
| 14 | +the Instrumented process that runs the user functions concretely with the provided input values |
| 15 | +and returns the execution result. |
11 | 16 |
|
12 |
| -Basically, if any exception happens inside the _instrumented process_, it is rethrown to the client process via RD. |
13 |
| -- Errors which do not cause the termination of the _instrumented process_ are wrapped in `InstrumentedProcessError`. Process won't be restarted, so client's requests will be handled by the same process. We believe, that the state of the _instrumented process_ is consistent, but in some tricky situations it **may be not**. Such situations should be reported as bugs. |
14 |
| -- Some of the errors lead to the instant death of the _instrumented process_. Such errors are wrapped in `InstrumentedProcessDeathException`. Before processing the next request, the _instrumented process_ will be restarted automatically, but it can take some time. |
| 17 | +A _client_ is an object that uses the `ConcreteExecutor` directly — it works in the Engine process as well. |
15 | 18 |
|
16 |
| -The extra logic of error and result handling depends on the provided instrumentation. |
| 19 | +`ConcreteExecutor` expects an `Instrumentation` object, which is responsible for, say, mocking static methods. In UnitTestBot Java, we use `UtExecutionInstrumentation` that implements the `Instrumentation` interface. |
17 | 20 |
|
18 |
| -## UtExecutionInstrumentation |
| 21 | +Basically, if an exception occurs in the Instrumented process, |
| 22 | +it is rethrown to the client object in the Engine process via Rd. |
19 | 23 |
|
20 |
| -The next sections are related only to the `UtExecutionInstrumentation` passed to the _instrumented process_. |
| 24 | +## Concrete execution outcomes |
21 | 25 |
|
22 |
| -The calling of `ConcreteExecutor::executeAsync` instantiated by the `UtExecutionInstrumentation` can lead to the three possible situations: |
23 |
| -- `InstrumentedProcessDeathException` occurs. Usually, this situation means there is an internal issue in the _instrumented process_, but, nevertheless, this exception should be handled by the client. |
24 |
| -- `InstrumentedProcessError` occurs. It also means an internal issue and should be handled by the client. Sometimes it happens because the client provided the wrong configuration or parameters, but the _instrumented process_ **can't determine exactly** what's wrong with the client's data. The cause contains the description of the phase which threw the exception. |
25 |
| -- No exception occurs, so the `UtConcreteExecutionResult` is returned. It means that everything went well during the invocation or something broke down because of the wrong input, and the _instrumented process_ **knows exactly** what's wrong with the client's input. The _instrumented process_ guarantees that the state **is consistent**. The exact reason of failure is a `UtConcreteExecutionResult::result` field. It includes: |
26 |
| - - `UtSandboxFailure` --- violation of permissions. |
27 |
| - - `UtTimeoutException` --- the test execution time exceeds the provided time limit (`UtConcreteExecutionData::timeout`). |
28 |
| - - `UtExecutionSuccess` --- the test executed successfully. |
29 |
| - - `UtExplicitlyThrownException` --- the target method threw exception explicitly (via `throw` instruction). |
30 |
| - - `UtImplicitlyThrownException` --- the target method threw exception implicitly (`NPE`, `OOB`, etc. or it was thrown inside the system library) |
31 |
| - - etc. |
| 26 | +`ConcreteExecutor` is parameterized with `UtExecutionInstrumentation`. When the `ConcreteExecutor::executeAsync` method is called, it leads to one of the three possible outcomes: |
32 | 27 |
|
33 |
| -### How the error handling works |
| 28 | +* `InstrumentedProcessDeathException` |
34 | 29 |
|
35 |
| -The pipeline of the `UtExecutionInstrumentation::invoke` consists of 6 phases: |
36 |
| -- `ValueConstructionPhase` --- constructs values from the models. |
37 |
| -- `PreparationPhase` --- prepares statics, etc. |
38 |
| -- `InvocationPhase` --- invokes the target method. |
39 |
| -- `StatisticsCollectionPhase` --- collects the coverage and execution-related data. |
40 |
| -- `ModelConstructionPhase` --- constructs the result models from the heap objects (`Any?`). |
41 |
| -- `PostprocessingPhase` --- restores statics, clear mocks, etc. |
| 30 | +Some errors lead to the instant termination of the Instrumented process. |
| 31 | + Such errors are wrapped in `InstrumentedProcessDeathException`. |
| 32 | + Prior to processing the next request, the Instrumented process is restarted automatically, though it can take time. |
| 33 | +`InstrumentedProcessDeathException` means that there is an Instrumented process internal issue. |
| 34 | +Nonetheless, this exception is handled in the Engine process. |
| 35 | + |
| 36 | +* `InstrumentedProcessError` |
| 37 | + |
| 38 | +Errors that do not cause the Instrumented process termination are wrapped in `InstrumentedProcessError`. |
| 39 | + The process is not restarted, so client's requests will be handled by the same process. |
| 40 | + We believe that the Instrumented process state is consistent but in some tricky situations it _may be not_. |
| 41 | + These situations should be reported as bugs. |
| 42 | +`InstrumentedProcessError` also means |
| 43 | +that there is an Instrumented process internal issue that should be handled by the client object |
| 44 | +(in the Engine process). |
| 45 | +The issue may occur because the client provides the wrong configuration or parameters, |
| 46 | +but the Instrumented process cannot exactly determine what's wrong with the client's data: |
| 47 | +one can find a description of the phase the exception has been thrown from. |
| 48 | + |
| 49 | +* `UtConcreteExecutionResult` |
| 50 | + |
| 51 | +If the Instrumented process performs well, |
| 52 | +or something is broken but the Instrumented process knows exactly what is wrong with the input, `UtConcreteExecutionResult` is returned. |
| 53 | +The Instrumented process guarantees that the state is _consistent_. |
| 54 | +A `UtConcreteExecutionResult::result` field helps to find the exact reason for a failure: |
| 55 | +* `UtSandboxFailure` — permission violation; |
| 56 | +* `UtTimeoutException` — test execution time exceeds the provided time limit (`UtConcreteExecutionData::timeout`); |
| 57 | +* `UtExecutionSuccess` — successful test execution; |
| 58 | +* `UtExplicitlyThrownException` — explicitly thrown exception for a target method (via `throw` instruction); |
| 59 | +* `UtImplicitlyThrownException` — implicitly thrown exception for a target method (`NPE`, `OOB`, etc., or an exception thrown inside the system library). |
| 60 | + |
| 61 | +## Error handling implementation |
| 62 | + |
| 63 | +The pipeline of `UtExecutionInstrumentation::invoke` includes 6 phases: |
| 64 | +1. `ValueConstructionPhase` — constructs values from the models; |
| 65 | +2. `PreparationPhase` — prepares statics, etc.; |
| 66 | +3. `InvocationPhase` — invokes the target method; |
| 67 | +4. `StatisticsCollectionPhase` — collects coverage and execution-related data; |
| 68 | +5. `ModelConstructionPhase` — constructs the result models from the heap objects (`Any?`); |
| 69 | +6. `PostprocessingPhase` — restores statics, clears mocks, etc. |
42 | 70 |
|
43 | 71 | Each phase can throw two kinds of exceptions:
|
44 |
| -- `ExecutionPhaseStop` --- indicates that the phase want to stop the invocation of the pipeline completely, because it's already has a result. The returned result is the `ExecutionPhaseStop::result` field. |
45 |
| -- `ExecutionPhaseError` --- indicates that an unexpected error happened inside the phase execution, so it's rethrown to the main process. |
| 72 | +- `ExecutionPhaseStop` — indicates that the phase tries to stop the invocation pipeline completely because it already has a result. The returned result is the `ExecutionPhaseStop::result` field. |
| 73 | +- `ExecutionPhaseError` — indicates that an unexpected error has occurred during the phase execution, and this error is rethrown to the Engine process. |
| 74 | + |
| 75 | +`PhasesController::computeConcreteExecutionResult` then matches on the exception type: |
| 76 | +* it rethrows the exception if the type is `ExecutionPhaseError`, |
| 77 | +* it returns the result if type is `ExecutionPhaseStop`. |
46 | 78 |
|
47 |
| -The `PhasesController::computeConcreteExecutionResult` then matches on the exception type and rethrows the exception if it's an `ExecutionPhaseError`, and returns the result if it's an `ExecutionPhaseStop`. |
| 79 | +## Timeout |
48 | 80 |
|
49 |
| -### Timeout |
| 81 | +Concrete execution is limited in time: the `UtExecutionInstrumentation::invoke` method is subject to timeout as well. |
50 | 82 |
|
51 |
| -There is a time limit on the concrete execution, so the `UtExecutionInstrumentation::invoke` method must respect it. We wrap phases which can take a long time with the `executePhaseInTimeout` block, which internally keeps and updates the elapsed time. If any wrapped phase exceeds the timeout, we return the `TimeoutException` as the result of that phase. |
| 83 | +For `UtExecutionInstrumentation` in the Instrumented process, we wrap the phases that can take a long time with the `executePhaseInTimeout` block. |
| 84 | +This block tracks the elapsed time. |
| 85 | +If a phase wrapped with this block exceeds the timeout, it returns `TimeoutException`. |
52 | 86 |
|
53 |
| -The clients cannot depend that cancellation request immediately breaks the invocation inside the _instrumented process_. The invocation is guaranteed to finish in the time of passed timeout. It may or **may not** finish earlier. Already started query in instrumented process is **uncancellable** - this is by design. |
| 87 | +One cannot be sure that the cancellation request immediately breaks the invocation pipeline inside the Instrumented process. |
| 88 | +Invocation is guaranteed to finish within timeout. |
| 89 | +It may or _may not_ finish earlier. |
| 90 | +The request that has been sent to the Instrumented process is _uncancellable_ by design. |
54 | 91 |
|
55 |
| -Even if the `TimeoutException` occurs, the _instrumented process_ is ready to process new requests. |
| 92 | +Even if the `TimeoutException` occurs, the Instrumented process is ready to process the new requests. |
0 commit comments