Work Outline for Comprehensive Test Suite

[Bran S]

On Tue, 31 Mar 2020 at 08:16, Eugene Syromiatnikov <esyr at redhat.com> wrote:
>
> On Fri, Mar 27, 2020 at 09:54:14PM +0530, Bran S wrote:
> > Below is the work outline for the gsoc proposal, please suggest
> > improvements wherever required.
> >
>
> > 1) Currently the strace-DDD.test fails, and as mentioned on the
> > mailing list there is an intermittent problem with strace-DD.test[1].
> > The solution to the strace-DDD.test is already proposed on the mailing
> > list[2]. It would be implemented and submitted for review.
> >
> > 2) After that the problem with strace-DD.test will be reproduced and a
> > solution will be proposed and discussed for further action.
>
> These are already solved by Dmitry[1][2] as it has been blocking the
> package update on several architectures in Debian for quite some time.

Yes I received the commit notification yesterday. So these 2 are out.

> > 3) Involve in extending decoder capability which could be helpful in
> > understanding to write tests better. For ex: extending HDIO_* decoder.
> > As currently only HDIO_GETGEO is available.
> >
> > 4) Write decoder tests wherever required. The coverage of files like
> > fs_x_ioctl, statx, sysctl, ubi, kvm, term is close 0%. These would be
> > the first targets.
>
> Note that fs_x_ioctl.c, sysctl.c, ubi.c, and term.c contain pretty old code,
> a significant update of the decoders is required there as well.  statx.c
> has only one line that is not covered (on systems where the corresponding
> statx.gen tests is run).  What is the expected timeline here?

Yes, timeline is something I am not sure about. I am thinking 5-6
weeks to update decoders for fs_x_ioctl.c, sysctl.c, ubi.c, and term.c
Is 5-6 weeks too much or am I being over optimistic ?

> > 5) Further discuss about critical files which need test coverage. For
> > ex: prctl, process, ioctl, etc.
>
> What other examples you can provide here?

Other examples would be files with less than 60% coverage. Like unwind.c, dm.c

I think the project now boils down to two simple points:
1) Write Decoders as mentioned above. (5-6 weeks)
2) Then write tests for those decoders and for the ones with less than
60% coverage. (5-6 weeks)

> > 6) Understand previous code[3][4][5] and move on to write behavioural
> > corner case tests to check for type sizes, signedness, handling of
> > pointers to invalid memory, etc.
>
> I don't think that these specific tests are crucially enlightening,
> they are mere examples (among many others) of some corner-case behaviour
> of strace itself.  The checks for type sizes/signedness and invalid
> pointers are pretty regular and comprise the bulk of decoder tests,
> on the other side.

Ok so since these checks are part of tests, I shouldn't mention this separately