|
1 | 1 | # UTBot Fuzzer
|
2 | 2 |
|
3 |
| -Fuzzer generates method input values to improve method coverage or find unexpected errors. In UTBot next strategies can be used to find those values: |
| 3 | +Fuzzer generates method input values to improve method coverage or find unexpected errors. In UTBot next strategies can be used to find values like: |
4 | 4 |
|
5 | 5 | * **Default values** for objects, e.g. 0, 0.0, empty string or null values.
|
6 |
| -* **Bound values** of primitives types, e.g. Integer.MAX_VALUE, Double.POSITIVE_INFINITY, etc. |
| 6 | +* **Corner case values** of primitives types, e.g. Integer.MAX_VALUE, Double.POSITIVE_INFINITY, etc. |
7 | 7 | * **Method constants and their simple mutations** of primitive types.
|
8 |
| -* **Simple objects** created via its constructors. |
| 8 | +* **Objects** created via its constructors or field mutators. |
| 9 | + |
| 10 | +After values are found fuzzer creates all its possible combinations and runs methods with these combinations. |
| 11 | + |
| 12 | +For example, if a method has two parameters of types `boolean` and `int` the follow values can be found: |
| 13 | +``` |
| 14 | +boolean = [false, true] |
| 15 | +int = [0, MAX_VALUE, MIN_VALUE] |
| 16 | +``` |
| 17 | + |
| 18 | +Now, fuzzer creates `2 * 3 = 6` combinations of them: |
| 19 | + |
| 20 | +``` |
| 21 | +[false, 0], [false, MAX_VALUE], [false, MIN_VALUE], [true, 0], [true, MAX_VALUE], [true, MIN_VALUE] |
| 22 | +``` |
| 23 | + |
| 24 | +To find more branches of execution as fast as possible fuzzer also shuffles |
| 25 | +combinations and supplies them for the running. |
9 | 26 |
|
10 | 27 | ## Design
|
11 | 28 |
|
12 |
| -Fuzzer requires model providers which are a simple functions to create a set of `UtModel` for a given `ClassId`. Fuzzer iterates through these providers and creates models, which are used for generating all possible combinations. Each combination contains values for the method. For example, if a method has `String, double, int` as parameters then fuzzer can create combination `"sometext", Double.POSITIVE_INFINITY, 0`. |
| 29 | +Fuzzer requires model providers that create a set of `UtModel` for a given `ClassId`. |
| 30 | +Fuzzer iterates through these providers and creates models, which are used for generating combinations later. |
| 31 | +Each combination contains concrete values that can be accepted by the method. |
| 32 | +For example, if a method has signature with `String, double, int` as parameters then fuzzer can create combination `"sometext", Double.POSITIVE_INFINITY, 0`. |
13 | 33 |
|
14 | 34 | Fuzzer's entry point is:
|
15 | 35 | ```kotlin
|
16 | 36 | // org.utbot.fuzzer.FuzzerKt
|
17 |
| -fun fuzz(method: FuzzedMethodDescription, vararg models: ModelProvider): Sequence<List<UtModel>> |
| 37 | +fun fuzz(method: FuzzedMethodDescription, vararg models: ModelProvider): Sequence<List<FuzzedValue>> |
18 | 38 | ```
|
19 | 39 |
|
| 40 | +`FuzzedMethodDescription` stores comprehensive information about a method: |
| 41 | +* signature (parameters and return types) |
| 42 | +* name/package/class (optional) |
| 43 | +* constants found in the method body (should be replaced with CGF when possible) |
| 44 | + |
| 45 | +`ModelProvider` provides models for a give parameters set as described below. |
| 46 | + |
| 47 | +Fuzz method returns a sequence of acceptable values for the method in random order. The sequence is lazy. |
| 48 | + |
20 | 49 | Model provider should implement
|
21 | 50 |
|
22 | 51 | ```kotlin
|
23 |
| -fun generate(description: FuzzedMethodDescription, consumer: BiConsumer<Int, UtModel>) |
| 52 | +fun generate(description: FuzzedMethodDescription): Sequence<FuzzedParameter> |
24 | 53 | ```
|
25 |
| -where consumer accepts 2 values: index of a parameter for the method and model for this parameter. For every parameter should exist at least one `UtModel`. `ModelProvider.withFallback` can be used to process those classes which cannot be processed by provider. Several providers can be combined into one by using `ModelProvider.with(anotherModel: ModelProvider)`. |
| 54 | +For every parameter should exist at least one `UtModel`. `ModelProvider.withFallback` can be used to process those classes which cannot be processed by provider. |
| 55 | +Several providers can be combined into one by using `ModelProvider.with(anotherModel: ModelProvider)`. |
26 | 56 |
|
27 | 57 | Common way to generate all combinations is:
|
28 | 58 |
|
29 | 59 | ```kotlin
|
30 | 60 | ObjectModelProvider()
|
31 | 61 | .with(PrimitiveModelProvider)
|
32 | 62 | // ...
|
33 |
| - .with(NullModelProvider) |
| 63 | + .with(ObjectModelProvider) |
34 | 64 | .withFallback { classId ->
|
35 | 65 | createDefaultModelByClass(classID)
|
36 | 66 | }
|
|
39 | 69 |
|
40 | 70 | ```kotlin
|
41 | 71 | // org.utbot.fuzzer.FuzzerKt
|
42 |
| -fun defaultModelProviders(idGenerator: ToIntFunction<ClassId>) |
| 72 | +fun defaultModelProviders(idGenerator: IntSupplier) |
43 | 73 | ```
|
44 | 74 |
|
45 |
| -## List of builtin Providers |
| 75 | +## List of builtin providers |
| 76 | + |
| 77 | +### PrimitiveDefaultsModelProvider |
| 78 | + |
| 79 | +Creates default values for every primitive types: |
| 80 | + |
| 81 | +``` |
| 82 | +boolean: false |
| 83 | +byte: 0 |
| 84 | +short: 0 |
| 85 | +int: 0 |
| 86 | +long: 0 |
| 87 | +float: 0.0 |
| 88 | +double: 0.0 |
| 89 | +char: \u0000 |
| 90 | +string: "" |
| 91 | +``` |
46 | 92 |
|
47 | 93 | ### PrimitiveModelProvider
|
48 | 94 |
|
49 | 95 | Creates default values and some corner case values such as Integer.MAX_VALUE, 0.0, Double.NaN, empty string, etc.
|
50 | 96 |
|
| 97 | +``` |
| 98 | +boolean: false, true |
| 99 | +byte: 0, 1, -1, Byte.MIN_VALUE, Byte.MAX_VALUE |
| 100 | +short: 0, 1, -1, Short.MIN_VALUE, Short.MAX_VALUE |
| 101 | +int: 0, 1, -1, Integer.MIN_VALUE, Integer.MAX_VALUE |
| 102 | +long: 0, 1, -1, Long.MIN_VALUE, Long.MAX_VALUE |
| 103 | +float: 0.0, 1.1, -1.1, Float.MIN_VALUE, Float.MAX_VALUE, Float.NEGATIVE_INFINITY, Float.POSITIVE_INFINITY, Float.NaN |
| 104 | +double: 0.0, 1.1, -1.1, Double.MIN_VALUE, Double.MAX_VALUE, Double.NEGATIVE_INFINITY, Double.POSITIVE_INFINITY, Double.NaN |
| 105 | +char: Char.MIN_VALUE, Char.MAX_VALUE |
| 106 | +string: "", " ", "string", "\n\t\r" |
| 107 | +``` |
| 108 | + |
| 109 | +### PrimitiveWrapperModelProvider |
| 110 | + |
| 111 | +Creates primitive models for boxed types: `Boolean`, `Byte`, `Short`, `Integer`, `Long`, `Double`, `Float`, `Character`, `String` |
| 112 | + |
51 | 113 | ### ConstantsModelProvider
|
52 | 114 |
|
53 | 115 | Uses information about concrete values from `FuzzedMethodDescription#concreteValues` to generate simple values.
|
54 |
| -At the moment, only primitive values are supported. |
| 116 | +Only primitive values are supported. |
| 117 | + |
| 118 | +### NullModelProvider |
| 119 | + |
| 120 | +Creates `UtNullModel` for every reference class. |
| 121 | + |
| 122 | +### EnumModelProvider |
| 123 | + |
| 124 | +Creates models for any enum type. |
| 125 | + |
| 126 | +### CollectionModelProvider |
| 127 | + |
| 128 | +Creates concrete collections for collection interfaces: `List`, `Set`, `Map`, `Collection`, `Iterable`, `Iterator` |
| 129 | + |
| 130 | +### ArrayModelProvider |
| 131 | + |
| 132 | +Creates an empty and non-empty for any type. |
55 | 133 |
|
56 | 134 | ### ObjectModelProvider
|
57 | 135 |
|
58 |
| -Creates models of class that has public constructor with primitives as parameters only. |
| 136 | +ObjectModelProvider is the most sophisticated provider. It creates model of class that has public constructors |
| 137 | +and public mutators (fields or setters/getters). If class has constructor that accepts another object within an argument that value |
| 138 | +is created recursively. Depth of recursion is limited to 1. Thus, for inner object fuzzing doesn't try to use every |
| 139 | +constructor but find the one with the least number of parameters and, if it is possible, only |
| 140 | +constructor with primitives values. If there is available only constructor with another object as a parameter then |
| 141 | +`null` is passed to it. |
| 142 | + |
| 143 | +Let's look at this example: |
| 144 | + |
| 145 | +```java |
| 146 | +class A { |
| 147 | + private int a; |
| 148 | + private Object object; |
| 149 | + |
| 150 | + public A(int a, A o) { |
| 151 | + this.a = a; |
| 152 | + this.o = o; |
| 153 | + } |
| 154 | +} |
| 155 | +``` |
59 | 156 |
|
60 |
| -### NullModelProvider |
| 157 | +For it fuzzing create these models: |
| 158 | +``` |
| 159 | +new Object(0, new A(0, null)); |
| 160 | +new Object(Integer.MIN_VALUE, new A(0, null)); |
| 161 | +new Object(Integer.MAX_VALUE, new A(0, null)); |
| 162 | +new Object(0, new A(Integer.MIN_VALUE, null)); |
| 163 | +new Object(Integer.MIN_VALUE, new A(Integer.MIN_VALUE, null)); |
| 164 | +new Object(Integer.MAX_VALUE, new A(Integer.MIN_VALUE, null)); |
| 165 | +new Object(0, new A(Integer.MAX_VALUE, null)); |
| 166 | +new Object(Integer.MIN_VALUE, new A(Integer.MAX_VALUE, null)); |
| 167 | +new Object(Integer.MAX_VALUE, new A(Integer.MAX_VALUE, null)); |
| 168 | +``` |
| 169 | + |
| 170 | +For classes that have empty public constructor and field mutators all those mutators will be fuzzed as well. |
| 171 | +Field mutators are listed below: |
| 172 | +* public or package-private (and accessible) non-final non-static fields |
| 173 | +* pairs of setter/getter that satisfy the common agreement: |
| 174 | + * setter/getter is public or package-private (and accessible) |
| 175 | + * have field name as a postfix, e.g.: `int myField -> * setMyField(int v)/int getMyField()`, where * means any returned type |
| 176 | + |
| 177 | +For example, fields _a_, _b_ and _d_ will be fuzzed, but _c_ and _e_ will not: |
| 178 | + |
| 179 | +```java |
| 180 | +class A { |
| 181 | + int a; |
| 182 | + public char b; |
| 183 | + public final int c = 0; |
| 184 | + private String d; |
| 185 | + private boolean e; |
| 186 | + |
| 187 | + public A setD(String s) { |
| 188 | + this.d = s; |
| 189 | + return this; |
| 190 | + } |
| 191 | + |
| 192 | + public String getD() { |
| 193 | + return d; |
| 194 | + } |
| 195 | + |
| 196 | + public boolean getE() { |
| 197 | + return e; |
| 198 | + } |
| 199 | +} |
| 200 | +``` |
| 201 | + |
| 202 | +### Other providers |
61 | 203 |
|
62 |
| -Creates `UtNullModel` for every reference class. |
| 204 | +There are several other providers that can find some values, using addition information, |
| 205 | +like `CharToStringModelProvider` that takes all chars found in `charAt(i) == c` statement |
| 206 | +and merge them into several strings. |
0 commit comments