GSOC2015 - Suggestions/Criticism - Some Test Scripts for Basic System Calls

[Rohan Bavishi]

Hi everyone,

As I had mentioned in the thread before, I had planned to write some
small test-suite code in order to get familiar with the code base. I
have come up with some small preliminary test-benches covering
system-calls 'chdir' and 'fcntl'. I'm posting the Github link, so as
to not clutter this thread with long, boring code.

Github: https://github.com/rbavishi/GSOC2015-strace

Any critique/suggestion regarding coding style, errors, code quality
is much appreciated.

Also, I did some crude hacks to include these tests in the Makefile
for testing, and both of them passed on my system.

Thank you so much for your help.


>> Suppose I plan to write a testbench for syscalls like chdir, mkdir,
>> fcntl (though probably not useful as they haven't been covered in the
>> suite). I'm not really clear as to how exhaustive of the instances do
>> I need to be while making that particular system call.
>> For example, if I try testing chdir, do I try to enforce all the
>> possible error codes possible (ENOENT, ENOTDIR etc.), and match output
>> using an awk script to check line by line, or is a single variant
>> enough.
>
>I think we need a good code coverage.  In your example, chdir parser is
>essentially a single printpath call, so the main focus of chdir test is
>going to be a printpath test (unless there is another test for printpath).
>I see no need to test all possible error codes of every syscall, usually
>a successful call and an unsuccessful call is enough.
>
>> Also, I see that a lot of system calls are not covered, including
>> basic ones. I'm not sure which are the ones at a higher priority of
>> being covered in the suite. Does the project envision a coverage of
>> all the system calls, or only the ones that are "useful", in the sense
>> that the rest are pretty reliably traced by strace.
>
>Automatic testing in strace is a relatively new project, so most of tests
>we have now are either regression tests or tests for recently implemented
>syscall decoders.  The ultimate goal is a coverage of all system calls.
>
>> I want to contribute some small piece of code to get myself acquainted
>> with the practices followed. Since it is difficult to explicitly find
>> bugs, I was thinking maybe I could start with writing some easy tests
>> for the basic calls as mentioned above with a focus on learning the
>> methodology of writing suites. But again, I don't really know whether
>> these basic-system-call checks will be any useful.
>
>Yes, starting with some simple tests might be a good idea.
>
>
>--
>ldv

-- 
Regards,

Rohan Bavishi

Sophomore
Department of Computer Science & Engg.
Indian Institute of Technology Kanpur