Skip to content

Implement a UtSettings option to enable or disable sandboxing #785

Closed
@dtim

Description

@dtim

Description

Sandbox changes the behavior of the concrete executor, and it affects the analysis of code that contains reflection, Unsafe class methods etc.

There are currently two options to manage sandbox permissions.

  • Enable a specific set of operations or all operations in the sandbox configuration file.
  • Wrap the body of the method under test with AccessController.doPrivileged method call to explicitly enable privileged operations when the MUT is concretely executed.

It seems that these options are sufficient: end users would either agree that potentially dangerous code is not executed (especially implicitly during the test suite generation), or enable a set of "acceptable" operations or their specific invocations.

At the same time, writing unit tests for UnitTestBot itself requires more fine-grained configuration of the sandbox. Unit tests for utbot-framework should ensure that the symbolic engine and the code generator work with any combination of user settings. Consider a test suite that checks that calls to Unsafe methods are correctly analyzed. We need to check three configurations:

  • Sandboxing is turned on, no unsafe operations are permitted, concrete executor signals an expected error.
  • Sandboxing is turned on, user code is wrapped with AccessController.doPrivileged.
  • Sandboxing is turned off, all code is enabled (or all necessary operations are explicitly enabled in the configuration file).

It seems inconvenient to create and drop configuration files when running specific tests. Using doPrivileged is OK but it checks a specific configuration, because adding code to MUT changes the behavior of the symbolic engine. As a result, we need a way to control the behavior of the concrete executor on a per-test basis.

Proposal:
* Add a new command line option to the concrete executor: --disable-sandbox (all operations are enabled if it is defined).
* Add a new UtSettings option: UtSettings.disableSandbox (false by default). If it is true, then --disable-sandbox argument
will be added to the concrete executor run command.
* Add a new utility function inline fun <reified T> withoutSandbox(block: () -> T): T to run specific tests with sandboxing turned off.

Sandbox will be enabled by default, and it can be disabled only when necessary. Users can use this option to disable sandboxing as an alternative method to granting all permissions in the sandbox configuration file.

Expected behavior

It will be possible to disable sandbox for specific tests and to keep it enabled for all other tests. Users will have one more (simpler) configuration option but no additional power

Environment

This feature request is targeted to framework unit tests, although setting the option in the .utbot/settings.properties will affect the plugin and CLI as well. It does not depend on any other environment options.

Metadata

Metadata

Assignees

Labels

ctg-enhancementNew feature, improvement or change request

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions