Skip to content

Fuzzer can't generate values (even without any modifications) when there are some modifications it fails to use #2428

Open
@IlyaMuravjov

Description

@IlyaMuravjov

Description (abstract example)

When using Seed.Recursive with modification X that has a parameter of type T, while there are no seeds for type T, fuzzer doesn't generate any values for that Seed.Recursive (except for the ones produced by its Routine.Empty, which is typically just null)

Real world (concrete) example

No integration tests using topicRepository.save(Topic) modification are generated for methods of TopicService class in Medical-Web-App project (or any other class that depends on TopicRepository). That happens because:

  1. There's Topic.setCreationTime(LocalDateTime) modification.
  2. Java fuzzer can't generate any values for LocalDateTime (a separate issue Java fuzzer can't create any values for types with no public constructor (e.g. LocalDateTime) #2437).
  3. Fuzzing platform fails to generate non-null values for Topic type due to point 2 and 3 (concern of this issue).
  4. Modification topicRepository.save(Topic) is only used with null values of topic, which causes modification itself to fail before we even get to method under test.

To Reproduce

Run the following unit test.

fun `fuzzer can generate non empty values even when it can't modify it due to lack of seeds`() {
   class Type(val hasSeeds: Boolean)
   class Value(val isEmpty: Boolean)

   runBlocking {
       withTimeout(1000) {
           runFuzzing(
               { _, type ->
                   if (type.hasSeeds) sequenceOf(Seed.Recursive(
                       construct = Routine.Create(emptyList()) { Value(isEmpty = false) },
                       modify = sequenceOf(Routine.Call(listOf(Type(hasSeeds = false))) { _, _ ->
                           fail("Value is generated for type with no seeds")
                       }),
                       empty = Routine.Empty { Value(isEmpty = true) }
                   ))
                   else emptySequence()
               },
               Description(listOf(Type(hasSeeds = true)))
           ) { _, (value) ->
               if (value.isEmpty) BaseFeedback(result = Unit, Control.CONTINUE)
               else BaseFeedback(result = Unit, Control.STOP)
           }
       }
   }
}

Expected behavior

Test passes.

Actual behavior

Test fails.

Visual proofs (screenshots, logs, images)

Timed out waiting for 1000 ms
kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 1000 ms
    (Coroutine boundary)
    at org.utbot.fuzzing.FuzzerSmokeTest$fuzzer can generate non empty values even when it can't modify it due to lack of seeds$1.invokeSuspend(FuzzerSmokeTest.kt:445)
Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 1000 ms
    at app//kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:184)
    at app//kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:154)
    at app//kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:508)
    at app//kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
    at app//kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:108)
    at java.base@19.0.2/java.lang.Thread.run(Thread.java:1589)

Metadata

Metadata

Assignees

No one assigned

    Labels

    comp-fuzzingIssue is related to the fuzzingctg-bugIssue is a bug

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions