GSOC2015 - Guidelines on writing test scripts

[Rohan Bavishi]

Hi everyone,

I have been reading the code of the existing test-suite for quite a
few days, and I feel I am getting the hang of the philosophy behind
the structure used for building these scripts. Therefore, I feel I
should try and get a first hand experience in writing these, by coming
up with some basic test schemes for the simple system calls not yet
covered.

According to my understanding, the method currently used for the basic
system calls is -
1. A C program performs a certain system call, covering various
possible instances if required.

2. A shell script is used to check -
    a. Whether the system-call is available on the machine
    b. Whether strace runs successfully while calling the C program
    c. Whether the strace output matches the expected one, which is
sometimes checked by regexes within the script or by using an awk
program.

My question -

Suppose I plan to write a testbench for syscalls like chdir, mkdir,
fcntl (though probably not useful as they haven't been covered in the
suite). I'm not really clear as to how exhaustive of the instances do
I need to be while making that particular system call.
For example, if I try testing chdir, do I try to enforce all the
possible error codes possible (ENOENT, ENOTDIR etc.), and match output
using an awk script to check line by line, or is a single variant
enough.

Also, I see that a lot of system calls are not covered, including
basic ones. I'm not sure which are the ones at a higher priority of
being covered in the suite. Does the project envision a coverage of
all the system calls, or only the ones that are "useful", in the sense
that the rest are pretty reliably traced by strace.

I want to contribute some small piece of code to get myself acquainted
with the practices followed. Since it is difficult to explicitly find
bugs, I was thinking maybe I could start with writing some easy tests
for the basic calls as mentioned above with a focus on learning the
methodology of writing suites. But again, I don't really know whether
these basic-system-call checks will be any useful.

I am pretty new to the concept of contributing to open-source, hence
it is difficult for me to figure these things out on my own. I thank
you for your patience.

Regards,

Rohan Bavishi

Personal Links:

Github: https://github.com/rbavishi
LinkedIn: http://in.linkedin.com/in/rbavishi

Sophomore,
Department of Computer Science & Engg.
Indian Institute of Technology, Kanpur