Skip to content

Python: generate tests for timeouted functions #1939

Closed
@tyuldashev

Description

@tyuldashev

Description
Currently when function under test reaches Hanging test timeout limit then Python UTBot just skips such case. Instead we suggest to generate test with expectation that it fails within specified timeout or at least create disabled test with comment about potential timeout.

Expected behavior

In case function timeouts during concrete execution Python UTBot may generate test which would handle that situation by adding some timeout constraint the same as UTBot Java does that with JUnit5 @Timeout annotation.

Potential alternatives

Also we could generate test which is disabled by default and add comment that test is disabled due timeout.

Context
Sample from Java with usage of JUnit5, similar expected to be implemented in Python.

User code:

    int timeout(int a, int b) throws InterruptedException {
        Thread.sleep(2000);
        return a + b;
    }

Generated test, with @Timeout annotation which would prevent test from hanging more than 1sec.

    @Test
    @DisplayName("timeout: return a + b -> TimeoutExceeded")
    @org.junit.jupiter.api.Timeout(value = 1000L, unit = TimeUnit.MILLISECONDS)
    public void testTimeout_ThreadSleep() throws InterruptedException {
        Sets sets = new Sets();
        
        /* This execution may take longer than the 1000 ms timeout
         and therefore fail due to exceeding the timeout. */
        assertTimeoutPreemptively(Duration.ofMillis(1000L), () -> sets.timeout(-255, -255));
    }

Metadata

Metadata

Assignees

Labels

comp-codegenIssue is related to code generatorctg-enhancementNew feature, improvement or change requestlang-pythonIssue is related to python support

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions