Skip to content

Implement state initialization by fuzzing #312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

SBOne-Kenobi
Copy link
Collaborator

@SBOne-Kenobi SBOne-Kenobi commented Jun 27, 2022

Description

Closes #217

Idea was described in issue. Briefly we use fuzzing as seed for SMT Solver to avoid bad initialization.

Implementation details

  • Created UtModelVisitor to traverse UtModels with collecting constraints needed to initialize SMT Solver.
  • Provided a method of UtSolver that apply initial constraints once that allows use this constraints as seed for SMT Solver.
  • Parameters are collected while traversing JimpleStmts (IdentetyStmt) thus we apply fuzzing as seed when we meet last IdentetyStmt.

Now this feature is turning off. To turn on look at UtSettings::useFuzzingInitialization.

Type of Change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Autotesting

All CI tests passed with this feature turning on.

Manual Scenario

I tested it using IDEA plugin on guava project. Generate tests several times with turning on this feature and without it then compare results (coverage).

Checklist (remove irrelevant options):

  • The change followed the style guidelines of the UTBot project
  • Self-review of the code is passed
  • The change contains enough commentaries, particularly in hard-to-understand areas
  • New documentation is provided or existed one is altered
  • No new warnings (except new TODOs)
  • Tests that prove my change is effective
  • All tests pass locally with my changes

@SBOne-Kenobi SBOne-Kenobi requested a review from CaelmBleidd June 27, 2022 09:34
@SBOne-Kenobi SBOne-Kenobi self-assigned this Jun 27, 2022
@SBOne-Kenobi SBOne-Kenobi marked this pull request as draft June 27, 2022 09:58
Copy link
Member

@CaelmBleidd CaelmBleidd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting questions

I think it is a good idea to write a doc about what is going on or at least a brief description in the PR. It is not obvious what is happening and I didn't completely get it

@SBOne-Kenobi SBOne-Kenobi force-pushed the sbone/fuzzing-in-symbolic-engine branch 5 times, most recently from 2e0e8d4 to a1db2de Compare June 29, 2022 12:25
@SBOne-Kenobi SBOne-Kenobi requested a review from CaelmBleidd June 29, 2022 12:33
@SBOne-Kenobi SBOne-Kenobi marked this pull request as ready for review June 29, 2022 12:39
@SBOne-Kenobi SBOne-Kenobi force-pushed the sbone/fuzzing-in-symbolic-engine branch from a1db2de to 90e85f4 Compare July 21, 2022 08:40
@SBOne-Kenobi SBOne-Kenobi requested a review from Markoutte July 21, 2022 12:31
Copy link
Collaborator

@Markoutte Markoutte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but I think we need to address the problem when fuzzer can generate very many inputs

@SBOne-Kenobi SBOne-Kenobi requested a review from Markoutte July 21, 2022 14:26

// create new state initialized by fuzzing if it is the last identity stmt for parameter
val initializedStateByFuzzing = if (
UtSettings.useFuzzingInitialization &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you extract these variables into separate variables with meaningful names?

constraints += mkNot(mkEq(value.addr, nullObjectAddr))
val arrayLength = traverser.memory.findArrayLength(value.addr)
constraints += mkEq(arrayLength.align(), model.length.primitiveToSymbolic())
val type = model.classId.toType() as ArrayType
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same about finding a soot class and then .arrayType

}

override fun visitAssemble(model: UtAssembleModel): List<UtBoolExpression> {
// TODO: not supported
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will it be supported?

val constraints = mutableListOf<UtBoolExpression>()
val type = model.classId.toType() as RefType
val typeStorage = traverser.typeResolver.constructTypeStorage(type, useConcreteType = true)
constraints += traverser.typeRegistry.typeConstraint(value.addr, typeStorage).isConstraint()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we need a correctness constraint?

* Call check of SMT Solver with [initialConstraints]. Used to initialize SMT Solver by fuzzing.
*/
fun checkWithInitialConstraints(initialConstraints: List<UtBoolExpression>): UtSolverStatus {
check(respectSoft = false).let { statusHolder ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we ignore soft constraints by default here? Wouldn't it be better to check with them?

}
}
val translatedConstraints = initialConstraints.translate()
val statusHolder = when (val status = check(translatedConstraints, mutableMapOf())) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

emptyMap?

Comment on lines +150 to +151
val ordinal = traverser.memory.findOrdinal(model.classId.toSoot().type, value.addr)
listOf(mkEq(ordinal.expr, model.value.ordinal.primitiveToLiteral()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this one constraint is enough? I guess we need to use the name field too to avoid two constructed enums with different ordinal but the same name.

Copy link
Collaborator

@Markoutte Markoutte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fuzzing part looks good to me

@Markoutte Markoutte self-requested a review July 22, 2022 07:35
@SBOne-Kenobi SBOne-Kenobi marked this pull request as draft September 6, 2022 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fuzzing in symbolic analyzing
4 participants