You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/TESTING.md
+34-81Lines changed: 34 additions & 81 deletions
Original file line number
Diff line number
Diff line change
@@ -28,13 +28,13 @@ Note: This configuration file must be in [JSON format](http://www.w3schools.com/
28
28
Note: Unfortunately JSON format is not allowing you to have comments inside JSON code.
29
29
30
30
Let’s see some example and let's try to configure small "test farm" with three devices connected to your host computer. In this example no peripherals (like SD card or EEPROM) are connected to our Mbed boards. We will use three platforms in this example:
After connecting boards to our host machine (PC) we can check which serial ports and disks they occupy. For our example let's assume that:
35
-
*```LPC1768``` serial port is on ```COM4``` and disk drive is ```J:```.
36
-
*```KL25Z``` serial port is on ```COM39``` and disk drive is ```E:```.
37
-
*```NUCLEO_F103RB``` serial port is on ```COM11``` and disk drive is ```I:```.
35
+
*```LPC1768``` serial port is on ```COM4``` and disk drive is ```J:```.
36
+
*```KL25Z``` serial port is on ```COM39``` and disk drive is ```E:```.
37
+
*```NUCLEO_F103RB``` serial port is on ```COM11``` and disk drive is ```I:```.
38
38
If you are working under Linux your port and disk could look like /dev/ttyACM5 and /media/usb5.
39
39
40
40
This information is needed to create ```muts_all.json``` configuration file. You can create it in ```mbed/tools/``` directory:
@@ -50,13 +50,13 @@ Its name will be passed to ```singletest.py``` script after ```-M``` (MUTs speci
50
50
"disk":"J:\\",
51
51
"peripherals": []
52
52
},
53
-
53
+
54
54
"2" : {"mcu": "KL25Z",
55
55
"port":"COM39",
56
56
"disk":"E:\\",
57
57
"peripherals": []
58
58
},
59
-
59
+
60
60
"3" : {"mcu": "NUCLEO_F103RB",
61
61
"port":"COM11",
62
62
"disk":"I:\\",
@@ -65,7 +65,7 @@ Its name will be passed to ```singletest.py``` script after ```-M``` (MUTs speci
65
65
}
66
66
```
67
67
68
-
Note: We will leave field ```peripherals``` empty for the sake of this example. We will explain it later. All you need to do now is to properly fill fields ```mcu```, ```port``` and ```disk```.
68
+
Note: We will leave field ```peripherals``` empty for the sake of this example. We will explain it later. All you need to do now is to properly fill fields ```mcu```, ```port``` and ```disk```.
69
69
70
70
Note: Please make sure files muts_all.json and test_spec.json are in tools/ directory. We will assume in this example they are.
71
71
Where to find ```mcu``` names? You can use option ```-S``` of ```build.py``` script (in ```mbed/tools/``` directory) to check all supported off-line MCUs names.
@@ -123,10 +123,10 @@ Please note that for MUT no. 2 few extra parameters were defined: ```copy_method
123
123
124
124
*```copy_method``` - STRING - tells test suite which binary copy method should be used.
125
125
You may notice that ```singletest.py``` command line help contains description about:
126
-
* Option ```-c``` (in MUTs file called ```copy_method```) with available copy methods supported by test suite plugin system.
126
+
* Option ```-c``` (in MUTs file called ```copy_method```) with available copy methods supported by test suite plugin system.
127
127
* Option ```-r``` (in MUTs file called reset_type) with available reset methods supported by test suite plugin system.
128
128
*```reset_type``` - STRING - some boards may require special reset handling, for example vendor specific command must be executed to reset device.
129
-
*```reset_tout``` - INTEGER - extra timeout just after device is reseted. May be used to wait for few seconds so device may finish booting, flashing data internally etc.
129
+
*```reset_tout``` - INTEGER - extra timeout just after device is reseted. May be used to wait for few seconds so device may finish booting, flashing data internally etc.
130
130
131
131
Part of help listing for singletest.py:
132
132
```
@@ -208,11 +208,11 @@ Having multiple configuration files allows you to manage your test scenarios in
208
208
209
209
* Set up all platforms and toolchains used during testing.
210
210
* Define (using script's ```-n``` switch) which tests you want to run during testing.
211
-
* Just run regression (all tests). Regression is default setting for test script.
211
+
* Just run regression (all tests). Regression is default setting for test script.
212
212
213
213
You can also force ```singletest.py``` script to:
214
214
* Run only peripherals' tests (switch ```-P```) or
215
-
* Just skip peripherals' tests (switch ```-C```).
215
+
* Just skip peripherals' tests (switch ```-C```).
216
216
* Build mbed SDK, libraries and corresponding tests with multiple cores, just use ```-j X``` option where ```X``` is number of cores you want to use for compilation.
* Execute each test case multiple times with ```--global-loops X``` option, where ```X``` number of repeats. Additionally use option ```-W``` to continue repeating test cases execution only if they continue to fail.
* Option ```--loops``` can be used to overwrite global loop count and redefine loop count for particular tests. Define test loops as ```TEST_ID=X``` where ```X``` is integer and separate loops count definitions by comma if necessary. E.g. ```TEST_1=5,TEST_2=20,TEST_3=2```.
228
+
```
229
+
* Option ```--loops``` can be used to overwrite global loop count and redefine loop count for particular tests. Define test loops as ```TEST_ID=X``` where ```X``` is integer and separate loops count definitions by comma if necessary. E.g. ```TEST_1=5,TEST_2=20,TEST_3=2```.
This will execute test ```RTOS_1``` ten (10) times and test ```RTOS_2``` five (5) times.
234
234
* Force non default copy method. Note that mbed platforms can be flashed with just binary drag&drop. We simply copy file onto mbed's disk and interface chip flashes target MCU with given binary. Force non standard (Python specific) copy method by using option ```-c COPY_METHOD``` where ```COPY_METHOD``` can be shell, command line copy command like: ```cp```, ```copy````, ```xcopy``` etc. Make sure those commands are available from command line!
* Run only selected tests. You can select which tests should be executed when you run test suite. Use ```-n``` switch to define tests by their ids you want to execute. Use comma to separate test ids:
@@ -319,20 +319,20 @@ This example will show you how to configure single device, run general tests or
319
319
2. We will also connect EEPROM ```24LC256``` to SDA, SCL pins of our Nucleo board and define 24LC256 peripheral to make sure our test suite will run all available tests for ```24LC256```.
320
320
321
321
Let's configure our one MUT and set uARM as the only compiler we will use to compiler Mbed SDK and tests.
322
-
We also need to create two configuration files ```muts_all.json``` and ```test_spec.json``` to pass our small testbed configuration to test script.
322
+
We also need to create two configuration files ```muts_all.json``` and ```test_spec.json``` to pass our small testbed configuration to test script.
323
323
324
324
muts_all.json:
325
325
```json
326
326
{
327
-
"1" : {
327
+
"1" : {
328
328
"mcu": "NUCLEO_F334R8",
329
329
"port":"COM46",
330
330
"disk":"E:\\",
331
331
"peripherals": ["24LC256"]
332
332
}
333
333
}
334
334
```
335
-
Note: By defining ```"peripherals": ["24LC256"]``` we are passing to test suite information that this particular board has EEPROM 24LC256 connected to our board.
335
+
Note: By defining ```"peripherals": ["24LC256"]``` we are passing to test suite information that this particular board has EEPROM 24LC256 connected to our board.
336
336
337
337
test_spec.json:
338
338
```json
@@ -342,7 +342,7 @@ test_spec.json:
342
342
}
343
343
}
344
344
```
345
-
Note:
345
+
Note:
346
346
* Please make sure device is connected before we will start running tests.
347
347
* Please make sure files ```muts_all.json``` and ```test_spec.json``` are in ```mbed/tools/``` directory.
Mbed SDK test suite supports writing tests using CppUTest. All you need to do it to provide CppUTest sources and includes with Mbed SDK port. This is already done for you so all you need to do it to get proper sources in your project directory.
423
423
CppUTest’s core design principles are:
424
-
* Simple in design and simple in use.
425
-
* Portable to old and new platforms.
424
+
* Simple in design and simple in use.
425
+
* Portable to old and new platforms.
426
426
* Build with Test-driven Development in mind.
427
427
428
428
## From where you can get more help about CppUTest library and unit testing
429
429
• You can read [CppUTest manual](http://cpputest.github.io/manual.html)
*[CppUTest on GitHub](https://github.com/cpputest/cpputest)
432
-
* Finally, if you think unit testing is new concept for you, you can have a grasp of it on Wikipedia pages about [unit testing](http://en.wikipedia.org/wiki/Unit_testing) and continue from there.
432
+
* Finally, if you think unit testing is new concept for you, you can have a grasp of it on Wikipedia pages about [unit testing](http://en.wikipedia.org/wiki/Unit_testing) and continue from there.
433
433
434
434
## How to add CppUTest to your current Mbed SDK installation
After above three steps you should have proper directory structure. All you need to do now is to configure your ```mbed_settings.py``` in ```mbed``` directory. Please refer to mbed SDK build script documentation for details.
496
496
497
-
## CppUTest with mbed port
497
+
## CppUTest with mbed port
498
498
To make sure you actualy have CppUTest library with mbed SDK port you can go to CppUTest ```armcc``` platform directory:
499
499
```
500
500
$ cd c:/Projects/Project/cpputest/src/Platforms/armcc/
@@ -505,7 +505,7 @@ You should find part of code responsible for porting console on default serial p
505
505
```c++
506
506
#include"Serial.h"
507
507
usingnamespacembed;
508
-
508
+
509
509
intPlatformSpecificPutchar(int c)
510
510
{
511
511
/* Please modify this block for test results to be reported as
@@ -514,7 +514,7 @@ int PlatformSpecificPutchar(int c)
514
514
#define NEED_TEST_REPORT_AS_CONSOLE_OUTPUT 1
515
515
#if NEED_TEST_REPORT_AS_CONSOLE_OUTPUT
516
516
extern Serial console;
517
-
517
+
518
518
#define NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE 1
519
519
#if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE
520
520
/* CppUTest emits \n line terminators in its reports; some terminals
@@ -523,7 +523,7 @@ int PlatformSpecificPutchar(int c)
@@ -596,7 +596,7 @@ All tests defined in test suite are described in ```mbed/tools/tests.py``` file.
596
596
Note: In dependency section we've added library ```CPPUTEST_LIBRARY``` which is pointing build script to CppUTest library with mbed port. This is a must for unit tests to be compiled with CppUTest library.
597
597
598
598
### Tests are now divided into two types:
599
-
#### 'Hello world' tests
599
+
#### 'Hello world' tests
600
600
First type of test cases we call 'hello world' tests. They do not dependent on CppUTest library and are monolithic programs usually composed of one main function. You can find this tests in below example directories:
601
601
602
602
*```mbed/libraries/tests/mbed/```
@@ -606,7 +606,7 @@ First type of test cases we call 'hello world' tests. They do not dependent on C
606
606
607
607
Usually ‘hello world’ test cases are using ```test_env.cpp``` and ```test_env.h``` files which implement simple test framework used to communicate with host test and help test framework instrument your tests.
608
608
609
-
Below you can see listing of ```test_env.h``` file which contains simple macro definitions used to communicate (via serial port printouts) between test case (on hardware) and host test script (on host computer).
609
+
Below you can see listing of ```test_env.h``` file which contains simple macro definitions used to communicate (via serial port printouts) between test case (on hardware) and host test script (on host computer).
610
610
Each use case should print on console basic information like:
611
611
* Default test case timeout.
612
612
* Which host test should be used to supervise test case execution.
0 commit comments