SCORE tutorials
Tutorial: Testing a simple program.
This tutorial introduces the main steps needed to test a simple C program with SCORE.
Setup SCORE
To setup SCORE, unzip the SCORE tarball file, add SCORE-0.1.1/bin to $PATH and build CIL. To build CIL, You may need the OCaml compiler version 3.08 or higher that can be downloaded from http://caml.inria.fr/
$ tar jxf SCORE-0.1.1.tar.bz2 $ cd SCORE-0.1.1 $ export PATH=`pwd`/bin:$PATH $ cd cil && ./configure && make && cd ..
If you already build CIL, you can skip this step.
The target program code
To automatically generate test cases for a target program by SCORE, symbolic inputs should be specified. To specify a variable as a symbolic input, we use the SCORE_int() function(line 7), which takes two arguments: a name of the variable to specify as a symbolic input, and a user-given name of the symbolic input, which can be any string. In this example, three integer variables a, b, and c are specified as symbolic inputs.
1 /* examples/simple.c */
2 #include <score.h>
3 #include <stdio.h>
4 int main(){
5 // a, b and c are symbolic inputs
6 int a, b, c;
7 SCORE_int(a, "1st input a"); SCORE_int(b, "2nd input b"); SCORE_int(c, "3rd input c");
8 if (a == 1){
9 if (b == 2){
10 if (c == 3*a + b){
11 // Suppose that this is an error location
12 printf("ERROR\n");
13 return 1;
14 }}}
15 return 0;
16 }
Compiling the target code
To perform symbolic execution during concrete execution of the target program, a target program need to be instrumented to insert probe functions into the target program. For a simple single-file C program, you can use scorec in bin/ to instrument and compile the target program.
$ cd examples/ $ scorec simple.c gcc -D_GNUCC -E -I../bin/../include -m32 -DCIL=1 simple.c -o /tmp/cil-lq41J1mr.i /home/yhkim/research/SCORE/tool/cil/obj/x86_LINUX/cilly.asm.exe --out /tmp/cil-GR6Msppr.cil.c --doScoreInstrument --envmachine /tmp/cil-lq41J1mr.i gcc -D_GNUCC -E -I../bin/../include -m32 /tmp/cil-GR6Msppr.cil.c -o /tmp/cil-XD36KAAj.cil.i gcc -D_GNUCC -c -I../bin/../include -m32 -m32 -o /tmp/cil-aOXYiQnl.o /tmp/cil-XD36KAAj.cil.i simple.c:15: warning: ‘__score_skip__’ attribute directive ignored ... ignorable warning messages ... simple.c:24: warning: ‘__score_skip__’ attribute directive ignored ../bin/../include/score.h:128: warning: ‘__score_skip__’ attribute directive ignored simple.c: In function ‘main’: simple.c:26: warning: cast from pointer to integer of different size gcc -D_GNUCC -o simple -I../bin/../include -m32 -m32 /tmp/cil-aOXYiQnl.o ../bin/../lib/libscore.a -lm -L../bin/../lib -lpthread -lstdc++ -lz3-gmp -m32 Read 6 branches. Read 13 nodes. Wrote 4 branch edges.
The bin/scorec script generates two intermediate files. One important generated file is the instrumented target program executable, whose name is same to the source file name excluding .c extension. Another important file is branches, which is used by the SCORE client to get the branch structure of the target program.
Running SCORE
To run SCORE on the target program, a user should run the SCORE server program first. Note that the port on which the SCORE server listen should not be filtered by a firewall.
$ score_server -p 12345 score_server: listen on port 12345. score_server: wait 1 clients.
In this tutorial, the SCORE client will generate maximum 10 test cases (-i 10) for the target program (-t ./simple) and store the generated test cases into the output directory (-o ./score-output). -a and -p options specify the ip address and the port number of the SCORE server to connect, repsectively.
$ score_client -i 10 -t ./simple -a 127.0.0.1 -p 12345 -o ./score-output Iteration 0 (0s): covered 0 branches [0 reached funs (total 0 branches)]. score_client: data comm. socket is bound to 0.0.0.0:44692 Iteration 1 (0s): covered 1 branches [1 reached funs (total 6 branches)]. Iteration 2 (0s): covered 3 branches [1 reached funs (total 6 branches)]. Iteration 3 (0s): covered 5 branches [1 reached funs (total 6 branches)]. ERROR Iteration 4 (0s): covered 6 branches [1 reached funs (total 6 branches)]. Total waiting time: 0.100389 s # of generated TCs: 4
Each iteration corresponds to a signle run of the target program under test. At each iteration, score_client prints the number of branches that have been covered through the iterations so far. In addition, score_client reports the number of reached functions so far and the number of all branches in those reached functions. Total waiting time is a total amount of time during which score_client waits to receive test case pairs from another client when it has no test case pairs to work on. In this tutorial, SCORE generates total 4 test cases to explore all possible executions of the target program.
SCORE-generated test cases
The test cases generated by SCORE are stored in the directory specified by the -o option of score_client. These test cases can be read with the print_testcase utility.
$ print_testcase score-output/testcase_4.test # of symbolic inputs: 3 sym input 0 name: 1st input a size: 4 data: \x00\x00\x00\x01 sym input 1 name: 2nd input b size: 4 data: \x00\x00\x00\x02 sym input 2 name: 3rd input c size: 4 data: \x00\x00\x00\x05
For the given test case generated by SCORE, The print_testcase utility reports the number of symbolic inputs, and the name, size, and data of each symbolic input in the test case. The data is represented in hexadecimal representation. For example, the 4th generated test case consists of the three symbolic inputs a, b, and c and the generated inputs for the symbolic inputs are 1, 2 and 5, respectively.
