Work Outline for Comprehensive Test Suite

[Eugene Syromiatnikov]

On Fri, Mar 27, 2020 at 09:54:14PM +0530, Bran S wrote:
> Below is the work outline for the gsoc proposal, please suggest
> improvements wherever required.
> 

> 1) Currently the strace-DDD.test fails, and as mentioned on the
> mailing list there is an intermittent problem with strace-DD.test[1].
> The solution to the strace-DDD.test is already proposed on the mailing
> list[2]. It would be implemented and submitted for review.
> 
> 2) After that the problem with strace-DD.test will be reproduced and a
> solution will be proposed and discussed for further action.

These are already solved by Dmitry[1][2] as it has been blocking the
package update on several architectures in Debian for quite some time.

> 3) Involve in extending decoder capability which could be helpful in
> understanding to write tests better. For ex: extending HDIO_* decoder.
> As currently only HDIO_GETGEO is available.
> 
> 4) Write decoder tests wherever required. The coverage of files like
> fs_x_ioctl, statx, sysctl, ubi, kvm, term is close 0%. These would be
> the first targets.

Note that fs_x_ioctl.c, sysctl.c, ubi.c, and term.c contain pretty old code,
a significant update of the decoders is required there as well.  statx.c
has only one line that is not covered (on systems where the corresponding
statx.gen tests is run).  What is the expected timeline here?

> 5) Further discuss about critical files which need test coverage. For
> ex: prctl, process, ioctl, etc.

What other examples you can provide here?

> 6) Understand previous code[3][4][5] and move on to write behavioural
> corner case tests to check for type sizes, signedness, handling of
> pointers to invalid memory, etc.

I don't think that these specific tests are crucially enlightening,
they are mere examples (among many others) of some corner-case behaviour
of strace itself.  The checks for type sizes/signedness and invalid
pointers are pretty regular and comprise the bulk of decoder tests,
on the other side.

> [1] https://lists.strace.io/pipermail/strace-devel/2020-March/009477.html
> [2] https://lists.strace.io/pipermail/strace-devel/2020-March/009536.html
> [3] https://gitlab.com/strace/strace/commit/99d8edef0078
> [4] https://gitlab.com/strace/strace/commit/a02a583b3fca
> [5] https://gitlab.com/strace/strace/commit/4278e6613f48

[1] https://gitlab.com/strace/strace/commit/00a29af7081e
[2] https://gitlab.com/strace/strace/commit/4ad7fce79c16