Skip to content

Missing errors in SARIF report and on Problems view #2023

Open
@alisevych

Description

@alisevych

Description

Some test methods from Error suites are missing on Problems view and in SARIF report.
Probably the issue scope is bigger.

To Reproduce

  1. Run the 'utbot' project in IntelliJ Idea 2022.3 Ultimate
  2. Install plugin built from unit-test-bot/rc2023.3 branch
  3. Use plugin to Generate and Run tests for types/Generics with default settings
  4. Wait Problems view is opened and tests are run

Expected behavior

All test methods from error suites should be shown on Problems view.

Actual behavior

9 tests are failing, 7 are displayed on Problems view.
2 tests are missing in SARIF report either.

Visual proofs (screenshots, logs, images)

Comparing Error suites and Problems view, these 2 are missing:

    @Test
    @DisplayName("mapAsNonStaticField: object.nonStaticMap.put(\"key\", \"value\") : True -> ThrowNullPointerException")
    public void testMapAsNonStaticField_MapPut() {
        Generics generics = new Generics();
        CollectionAsField object = new CollectionAsField();
        object.nonStaticMap = null;
        
        /* This test fails because method [org.utbot.examples.types.Generics.mapAsNonStaticField] produces [java.lang.NullPointerException]
            org.utbot.examples.types.Generics.mapAsNonStaticField(Generics.java:27) */
        generics.mapAsNonStaticField(object);
    }

    @Test
    @DisplayName("methodWithArrayTypeBoundary:  -> throw NullPointerException")
    public void testMethodWithArrayTypeBoundaryThrowsNPE() {
        Generics generics = new Generics();
        
        /* This test fails because method [org.utbot.examples.types.Generics.methodWithArrayTypeBoundary] produces [java.lang.NullPointerException]
            org.utbot.examples.types.ArrayTypeParameters.methodWithArrayTypeBoundary(Generics.java:51)
            org.utbot.examples.types.Generics.methodWithArrayTypeBoundary(Generics.java:45) */
        generics.methodWithArrayTypeBoundary();
    }

image

image

Environment

Windows 10 Pro
IntelliJ IDEA 2022.3.3 Ultimate
Gradle, JDK 17

Additional context

Checked in a new Gradle project with JDK 8 - 1 test was missing

Metadata

Metadata

Assignees

Labels

comp-sarifIssue is related to SARIF report or Problems tabctg-bugIssue is a bug

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions