GSOC2015 - Guidelines on writing test scripts

[Dmitry V. Levin]

On Tue, Mar 10, 2015 at 05:22:39PM +0530, Rohan Bavishi wrote:
[...]
> Suppose I plan to write a testbench for syscalls like chdir, mkdir,
> fcntl (though probably not useful as they haven't been covered in the
> suite). I'm not really clear as to how exhaustive of the instances do
> I need to be while making that particular system call.
> For example, if I try testing chdir, do I try to enforce all the
> possible error codes possible (ENOENT, ENOTDIR etc.), and match output
> using an awk script to check line by line, or is a single variant
> enough.

I think we need a good code coverage.  In your example, chdir parser is
essentially a single printpath call, so the main focus of chdir test is
going to be a printpath test (unless there is another test for printpath).
I see no need to test all possible error codes of every syscall, usually
a successful call and an unsuccessful call is enough.

> Also, I see that a lot of system calls are not covered, including
> basic ones. I'm not sure which are the ones at a higher priority of
> being covered in the suite. Does the project envision a coverage of
> all the system calls, or only the ones that are "useful", in the sense
> that the rest are pretty reliably traced by strace.

Automatic testing in strace is a relatively new project, so most of tests
we have now are either regression tests or tests for recently implemented
syscall decoders.  The ultimate goal is a coverage of all system calls.

> I want to contribute some small piece of code to get myself acquainted
> with the practices followed. Since it is difficult to explicitly find
> bugs, I was thinking maybe I could start with writing some easy tests
> for the basic calls as mentioned above with a focus on learning the
> methodology of writing suites. But again, I don't really know whether
> these basic-system-call checks will be any useful.

Yes, starting with some simple tests might be a good idea.


-- 
ldv
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.strace.io/pipermail/strace-devel/attachments/20150311/523059a0/attachment.bin>