Skip to content

Socket connection in python fuzzing #1844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Mar 3, 2023
Merged

Conversation

tamarinvs19
Copy link
Collaborator

@tamarinvs19 tamarinvs19 commented Feb 22, 2023

Description

Added socket connection between python and kotlin. Now it works 10 times faster then previous verision.

Fixes #1856

How to test

Manual tests

Generate tests for python function, for example, with many branches.

Self-check list

Check off the item if the statement is true. Hint: [x] is a marked item.

Please do not delete the list or its items.

  • I've set the proper labels for my PR (at least, for category and component).
  • PR title and description are clear and intelligible.
  • I've added enough comments to my code, particularly in hard-to-understand areas.
  • The functionality I've repaired, changed or added is covered with automated tests.
  • Manual tests have been provided optionally.
  • The documentation for the functionality I've been working on is up-to-date.

@tamarinvs19 tamarinvs19 added ctg-refactoring Issue related to refactoring process comp-fuzzing Issue is related to the fuzzing lang-python Issue is related to python support labels Feb 22, 2023
@tamarinvs19 tamarinvs19 self-assigned this Feb 22, 2023
@tamarinvs19 tamarinvs19 marked this pull request as draft February 22, 2023 15:57
Copy link
Collaborator

@Markoutte Markoutte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment an exception is thrown, or no tests generated for this example:

def test3(x: str) -> bool:
    if x[0] == 'b':
        if x[1] == 'a':
            if x[2] == 'd':
                if x[3] == '!':
                    raise RuntimeError("Very bad")
    return True

Vyacheslav Tamarin and others added 5 commits February 27, 2023 10:51
# Conflicts:
#	utbot-python/src/main/kotlin/org/utbot/python/PythonEngine.kt
#	utbot-python/src/main/kotlin/org/utbot/python/PythonTestCaseGenerator.kt
#	utbot-python/src/main/resources/requirements.txt
@tamarinvs19 tamarinvs19 marked this pull request as ready for review February 27, 2023 12:02
@tyuldashev tyuldashev added this to the 2023.03 Release milestone Mar 1, 2023
…entation in state dict, move up ADDITIONAL_LIMIT, update utbot_executor
data class PythonFeedback(
override val control: Control = Control.CONTINUE,
val result: Trie.Node<Instruction> = Trie.emptyNode(),
val executionFeedback: FuzzingExecutionFeedback?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feedback is used as a part of equals/hashCode, therefore 2 same paths with different result will be recognized as different feedbacks. Is it intented?

fun getOffsetLine(sourceFileContent: String, offset: Int): Int {
return sourceFileContent.take(offset).count { it == '\n' } + 1
}

fun <T> weightedRandom(elems: List<T>, weights: List<Double>): T {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can find similar util method in RandomExtension.kt (in module utbot-fuzzing) with name Random.chooseOne. Also, please not, that random in your case doesn't use seed, therefore, every run will be different, that usually is undesirable.

}.first
val value = Random.nextDouble()
return elems[borders.indexOfFirst { it >= value }]
val index = Random.chooseOne(weights.map { it / sum }.toDoubleArray())
Copy link
Collaborator

@Markoutte Markoutte Mar 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The random is still no-seed random. Also, you can pass to the chooseOne just weights.toDoubleArray. The method chooseOne does sum itself, so, there's no need to do it before.

@@ -52,6 +53,18 @@ class PythonEngine(
private val pythonTypeStorage: PythonTypeStorage,
) {

private val cache = mutableMapOf<Pair<PythonMethodDescription, List<PythonTreeWrapper>>, PythonExecutionResult>()

private fun addExecutionToCache(key: Pair<PythonMethodDescription, List<PythonTreeWrapper>>, result: PythonExecutionResult) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend extracting the cache into a separate class to facilitate further work with it.

@tochilinak tochilinak enabled auto-merge (squash) March 3, 2023 11:22
@tochilinak tochilinak merged commit 9d358d7 into main Mar 3, 2023
@tochilinak tochilinak deleted the tamarinvs19/utbot-python-sockets branch March 3, 2023 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-fuzzing Issue is related to the fuzzing ctg-refactoring Issue related to refactoring process lang-python Issue is related to python support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add socket connection between UnitTestBot and PythonExecutor
4 participants