Skip to content

Scripts for jlearch #279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 30, 2022
Merged

Scripts for jlearch #279

merged 9 commits into from
Jun 30, 2022

Conversation

Atos1337
Copy link
Member

@Atos1337 Atos1337 commented Jun 22, 2022

Description

Add scripts for automation training and evaluating of jlearch.

Fixes #229

Type of Change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Manual Scenario

Look at docs/jlearch/setup.md

@Atos1337 Atos1337 marked this pull request as draft June 22, 2022 10:31
@Atos1337 Atos1337 requested a review from amandelpie June 22, 2022 13:22
@Atos1337 Atos1337 marked this pull request as ready for review June 22, 2022 13:22
@Atos1337 Atos1337 marked this pull request as draft June 22, 2022 13:43
@Atos1337 Atos1337 force-pushed the Atos1337/jlearch-scripts branch from 68a6dca to 1b9caf0 Compare June 23, 2022 07:38
@amandelpie
Copy link
Collaborator

amandelpie commented Jun 26, 2022

Hi @Atos1337 I reproduced the pipeline, its finished with the report creation, but in console it finishes with the following text (I've trained on the smallest piece of data and model is not accurate) with build errors

Please let me know if it's normal behaviour (probably we need to document it)

rg.antlr.v4.codegen.model.RuleFunctionTest > testGetNameByFuzzer1 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6890
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer2]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer3]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer4]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName10]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName11]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName12]: FAILURE

org.antlr.v4.codegen.model.RuleFunctionTest > testGetName12 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6810
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName13]: SUCCESS
Test summary: FAILURE (1079 tests, 30 successes, 1049 failures, 0 skipped)

1079 tests completed, 1049 failed

> Task :utbot-junit-contest:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':utbot-junit-contest:test'.
> There were failing tests. See the report at: file:///home/zaleslaw/IdeaProjects/UTBotJava/utbot-junit-contest/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 12s
36 actionable tasks: 4 executed, 32 up-to-date

@Atos1337 Atos1337 marked this pull request as ready for review June 27, 2022 04:09
@Atos1337
Copy link
Member Author

Atos1337 commented Jun 27, 2022

Hi @Atos1337 I reproduced the pipeline, its finished with the report creation, but in console it finishes with the following text (I've trained on the smallest piece of data and model is not accurate) with build errors

Please let me know if it's normal behaviour (probably we need to document it)

rg.antlr.v4.codegen.model.RuleFunctionTest > testGetNameByFuzzer1 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6890
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer2]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer3]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetNameByFuzzer4]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName10]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName11]: SUCCESS
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName12]: FAILURE

org.antlr.v4.codegen.model.RuleFunctionTest > testGetName12 FAILED
    java.lang.NullPointerException at RuleFunctionTest.java:6810
[org.antlr.v4.codegen.model.RuleFunctionTest] [testGetName13]: SUCCESS
Test summary: FAILURE (1079 tests, 30 successes, 1049 failures, 0 skipped)

1079 tests completed, 1049 failed

> Task :utbot-junit-contest:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':utbot-junit-contest:test'.
> There were failing tests. See the report at: file:///home/zaleslaw/IdeaProjects/UTBotJava/utbot-junit-contest/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 12s
36 actionable tasks: 4 executed, 32 up-to-date

@amandelpie it's OK, because UTBot can generate tests that should be failed. So in this case, there are no build errors, there are just failed generated tests.

Copy link
Collaborator

@amandelpie amandelpie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, @Atos1337 if it's could be reproduced on the random Ubuntu machine and could be tuned by myself in some settings, I approve this PR

Please add some comments based on my suggestions

@Atos1337 Atos1337 force-pushed the Atos1337/jlearch-scripts branch from 953b5e5 to 885cfc1 Compare June 29, 2022 07:58
@Atos1337 Atos1337 force-pushed the Atos1337/jlearch-scripts branch from 885cfc1 to ad88c03 Compare June 29, 2022 13:25
@Atos1337 Atos1337 merged commit abbf596 into main Jun 30, 2022
@Atos1337 Atos1337 deleted the Atos1337/jlearch-scripts branch June 30, 2022 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Scripts for JLearch
2 participants