Manual testing of UtUtils class generation - build#2022.11.***

[alisevych]

Initial set-up

Check that the IntelliJ Idea UTBot plugin can be successfully installed

• Take build with changes for UtUtils generation
• Download plugin
• Check downloaded zip-file size < 100 MB
• Open IntelliJ IDE
• Remove previously installed UTBot plugin
• Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
• Open the project in the IDE
• Install the downloaded plugin

Kotlin support - investigate and add checks

• If we generate Kotlin tests for Java code - Kotlin UtUtils class is generated?
• If UtUtils class needs update - mockito support f.i. - and there are 2 UtUtils classes already - only one class is recreated?
• Are their versions / mockito on/off treated separately?

Answer: kotlin and java UtUtils class is independent.

Go through manual scenarios

	Pre-condition	Condition	Class under test	Output
	JAVA -> JAVA without kotlin.UtUtils
1	No UtUtils class	Generate a test class, that does not need utility methods	IntExamples.java	No UtUtils class is produced.
2	No UtUtils class	Generate a test class, that (a) needs a utility method, (b) does not use mocking	QueueUsages.java	Regular UtUtils class (without mocking support) is produced.
3	Regular UtUtils class (without mocking support) exists	Generate a test class, that (a) needs a utility method, (b) uses mocking	CommonMocksExample.java	UtUtils class with mocking support is produced.
4	UtUtils class with mocking support exists	Generate a test class, that (a) needs a utility method, (b) does not use mocking	QueueUsages.java	UtUtils class with mocking support from the previous generation stays the same.
5	UtUtils class with mocking support exists	Generate another test class, that (a) needs a utility method, (b) does not use mocking + create and choose another Test sources root: utbot-sample/src/test/java2	QueueUsages.java	No additional UtUtils class is generated in utbot-sample/src/test/java2. UtUtils class with mocking support from the previous generation stays the same.
6	No UtUtils class (delete previously generated test classes and UtUtils classes)	Generate a test class, that (a) does not need utility methods (b) uses mocking	MockRandomExamples.java and choose the method randomAsParameter() for test generation	No UtUtils class is produced.
	JAVA -> KOTLIN
7	No UtUtils class	Generate a Kotlin test class for Java code, that does not need utility methods	IntExamples.java	No Java or Kotlin UtUtils class is produced.
8	No UtUtils class	Generate a Kotlin test class for Java code, that (a) needs a utility method, (b) does not use mocking	ArrayOfObjects.java choose arrayWithSucc method	Regular Kotlin UtUtils class (without mocking support) is produced.
9	Regular UtUtils class (without mocking support) exists	Generate a Kotlin test class for Java code, that (a) needs a utility method, (b) uses mocking	CommonMocksExample.java	Kotlin UtUtils class with mocking support is produced.
	KOTLIN -> KOTLIN without java.UtUtils
10	No java.UtUtils or kotlin.UtUtils classes (delete previously generated test classes and UtUtils classes), convert BitOperators.java to BitOperators.kt	Generate a Kotlin test class for Kotlin code, that (a) needs a utility method, (b) does not use mocking	BitOperators.kt	No kotlin.UtUtils class is produced.
11	No UtUtils class, convert MockWithFieldExample.java to MockWithFieldExample.kt	Generate a Kotlin test class for Kotlin code, that (a) needs a utility method, (b) does not use mocking	MockWithFieldExample.kt	Regular kotlin.UtUtils Kotlin class (without mocking support) is produced. No java.UtUtils created.
12	Regular kotlin.UtUtils class (without mocking support) exists, convert MockRandomExamples.java to MockRandomExamples.kt	Generate a Kotlin test class for Kotlin code, that (a) needs a utility method, (b) uses mocking	MockRandomExamples.kt	kotlin.UtUtils Kotlin class (without mocking support) is produced. No java.UtUtils created.
	JAVA -> JAVA with kotlin.UtUtils
13	kotlin.UtUtils class with mocking support exists	Generate a Java test class for Java code, that does not need utility methods	IntExamples.java	No java.UtUtils class is produced. kotlin.UtUtils left as is.
14	kotlin.UtUtils class with mocking support exists	Generate a Java test class for Java code, that (a) needs a utility method, (b) does not use mocking	QueueUsages.java	Regular java.UtUtils class is produced without mocking. kotlin.UtUtils left as is.
15	Regular java.UtUtils class without mocking support exists and kotlin.UtUtils class with mocking support exists	Generate a Java test class for Java code, that (a) needs a utility method, (b) uses mocking	CommonMocksExample.java	java.UtUtils class is produced with mocking support. kotlin.UtUtils left as is.
	KOTLIN -> KOTLIN with java.UtUtils
16	No kotlin.UtUtils exists (remove previously created), java.UtUtils with mocking support exist	Generate a Kotlin test class for Kotlin code, that (a) needs a utility method, (b) uses mocking	MockRandomExamples.kt	kotlin.UtUtils Kotlin class (without mocking support) is produced. java.UtUtils left as is.

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16

IDEA interaction checks

• Gradle / Maven / IntelliJ
• in new test sources root
• in test sources root in other module

[tyuldashev]

• size is 120Mb and more than required 100Mb
• Reduce size of plugin zip #1333

[tyuldashev]

Issues found:

• UtUtils with mocks rewritten by UtUtils without mocks #1378
• UtUtils has region without name #1380
• Bad formatting of UtUtils Javadoc after it has been updated #1385

[tyuldashev]

Three UtUtils related issues are fixed and verified on IDEA 2022.2.3 and UTBot 2022.11.551

[alisevych]

Found by @tyuldashev

UtUtils was updated and version will be different since:

• Update version of UtUtils class #1993

Need to add test on this to checklist. Like:

• install release version with previous version of UtUtils
• create a new project
• generate tests
• check UtUtils is created
• uninstall release version
• install new plugin version with new UtUtils
• regenerate tests
• check UtUtils class was recreated