Skip to content

Identical cases with and without mocking are not minimized #1988

Open
@alisevych

Description

@alisevych

Description

Symbolic engine generates test case with mocking,
Fuzzing is generating the same test case without mocking.
They are not minimized and both are present in resulting test class.

To Reproduce

  1. Run the 'utbot' project in IntelliJ Idea 2022.2.4 Ultimate
  2. Install plugin built from unit-test-bot/rc3102023 branch
  3. Generate tests with default settings for mock.MockFinalClassExample
  4. Open the generated test

Expected behavior

If Fuzzing is generating the same test as Symbolic engine, it should be excluded from the test class.

Actual behavior

The execution path of the tested class and input parameters are the same.
The difference is in class instantiation for the inner field.

So this is an open question - do we need to exclude this case from the test class?

Visual proofs (screenshots, logs, images)

    ///region SYMBOLIC EXECUTION: SUCCESSFUL EXECUTIONS for method useFinalClass()

    public void testUseFinalClass_XEquals1() {
        MockFinalClassExample mockFinalClassExample = new MockFinalClassExample();
        FinalClass finalClassMock = mock(FinalClass.class);
        (when(finalClassMock.provideInt())).thenReturn(1);
        mockFinalClassExample.intProvider = finalClassMock;

        int actual = mockFinalClassExample.useFinalClass();

        assertEquals(1, actual);
    }

    ///region FUZZER: SUCCESSFUL EXECUTIONS for method useFinalClass()

    public void testUseFinalClassReturnsOne() {
        MockFinalClassExample mockFinalClassExample = new MockFinalClassExample();
        FinalClass finalClass = new FinalClass();
        mockFinalClassExample.intProvider = finalClass;

        int actual = mockFinalClassExample.useFinalClass();

        assertEquals(1, actual);
    }

Environment

Windows 10 Pro
IntelliJ IDEA 2022.2.4 Ultimate

Additional context

As discussed with @Markoutte , instrumented process enhancements are required.
We need mechanisms to treat these cases as identical.
Similar as we need enhancements to leave both cases with true/false input/output - nevertheless they are covering the same lines.

Metadata

Metadata

Assignees

Labels

comp-fuzzingIssue is related to the fuzzingcomp-instrumented-processIssue is related to Instrumented processcomp-minimizerIssue is related to Minimization phasectg-bugIssue is a bug

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions