Skip to content

Better naming for functions and comments could be deduced from fuzzer #288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 30, 2022

Conversation

Markoutte
Copy link
Collaborator

Description

Generates more reasonable names of tests which were generated by fuzzer.

Fixes #180

Type of Change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Manual Scenario

Create method like this:

public boolean isStringShorterThan(String string, int size) {
    return string.length() < size;
}

Generate test with fuzzing. There's 2 test should be generated (can be a little different):

@Test
@DisplayName("isStringShorterThan: string has special characters, size = 0 -> return false")
public void testIsStringShorterThanReturnsFalseWithBlankStringAndCornerCase() {
    Test180 test180 = new Test180();

    boolean actual = test180.isStringShorterThan("\n\t\r", 0);

    assertFalse(actual);
}

@Test
@DisplayName("isStringShorterThan: string = blank string, size = Int.MAX_VALUE -> return true")
public void testIsStringShorterThanReturnsTrueWithBlankStringAndCornerCase() {
    Test180 test180 = new Test180();

    boolean actual = test180.isStringShorterThan("   ", Integer.MAX_VALUE);

    assertTrue(actual);
}

Checklist (remove irrelevant options):

  • The change followed the style guidelines of the UTBot project
  • Self-review of the code is passed
  • The change contains enough commentaries, particularly in hard-to-understand areas
  • New documentation is provided or existed one is altered
  • No new warnings
  • Tests that prove my change is effective
  • All tests pass locally with my changes

@Markoutte
Copy link
Collaborator Author

Markoutte commented Jun 23, 2022

@amandelpie I implemented idea that fuzzer can add information about each value it creates. At the moment this information is pretty simple, like:

  1. value is set: %var% = 3
  2. value constraint: %var% <= 10
  3. value characteristic: %var% has special characters

Here, %var% is a placeholder for a parameter name that will be replaced later in test name generation. I can add some additional information in FuzzedValue if needed for more complicated test name creation.

if (GENERATE_DISPLAY_NAMES) {
if (GENERATE_DISPLAY_NAMES
// do not rewrite display name if already set
&& traceTags.execution.displayName.isNullOrBlank()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably we need to move this behaviour to settings? Like GENERATE_DISPLAY_NAMES
Because it's very direct logic in the internals of summarization module

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is good idea to keep global settings at all. Also, I don't like to add new settings until it really necessary. I think it is better to make this settings be customizable through the Summarization instance. But at the moment I find Summarization code is a bit messy.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting TODO

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to use alternative format with todos like TODO: link to the issue or discussion on the Github. If you disagree with the settings model lets discuss it, don't hide the thoughts

package org.utbot.fuzzer.names

/**
* Information that can be used to generate tests.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To generate tests or test names?

@@ -25,7 +25,9 @@ class ArrayModelProvider(
length = arraySize,
arrayClassId.elementClassId!!.defaultValueModel(),
mutableMapOf()
)
).fuzzed {
this.summary = "%var% = ${arrayClassId.elementClassId!!.simpleName}[$arraySize]"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

%var% - is very popular place in you code and look like is one of the basic concepts. But it requires a time to understand what is the final goal and area of usage.

Could you please provide Readme or extended comments related to the concept of %var% usage and replacing. Now it requires a time to restore the concept from the code.

The minor architecture note in .md format also will be useful.

Some comments like above methods createTestName/displayName was very useful for me.

Just imagine that somebody need to extend your functionality or fix a bug in it, please help him

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is described as a kdoc in FuzzedValue class for summary field. The test is follow:

 /**
     * Summary is a piece of useful information that clarify why this value has a concrete value.
     *
     * It supports a special character `%var%` that is used as a placeholder for parameter name.
     *
     * For example:
     * 1. `%var% = 2` for a value that have value 2
     * 2. `%var% >= 4` for a value that shouldn't be less than 4
     * 3. `foo(%var%) returns true` for values that should be passed as a function parameter
     * 4. `%var% has special characters` to describe content
     */

Copy link
Collaborator

@amandelpie amandelpie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some comments and suggestions.
I'd like a few basic rules which were proposed in this PR and moreover, I suppose NameSuggesters could be independently used for all other summaries like a simple level instead of just ignoring summaries when it fails.

@Markoutte Markoutte force-pushed the pelevin/better-test-names branch from 2dd8c98 to 5fd0342 Compare June 27, 2022 10:59
@Markoutte Markoutte requested a review from amandelpie June 28, 2022 08:44
@Markoutte Markoutte force-pushed the pelevin/better-test-names branch from 5fd0342 to ca65095 Compare June 29, 2022 11:05
Copy link
Collaborator

@amandelpie amandelpie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for me

@Markoutte Markoutte force-pushed the pelevin/better-test-names branch from ca65095 to cef1577 Compare June 30, 2022 11:21
@Markoutte Markoutte merged commit 8f1d85c into main Jun 30, 2022
@Markoutte Markoutte deleted the pelevin/better-test-names branch June 30, 2022 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Better naming for functions and comments could be deduced from fuzzer routines
2 participants