[PATCH v2 2/2] ioctl console test v2

[Bran S]

On Wed, 18 Mar 2020 at 03:22, Eugene Syromiatnikov <esyr at redhat.com> wrote:
>
> On Tue, Mar 17, 2020 at 10:11:38PM +0530, Bran S wrote:
> > On Mon, 9 Mar 2020 at 19:19, Eugene Syromiatnikov <esyr at redhat.com> wrote:
> > >
> > > On Mon, Mar 09, 2020 at 07:06:49PM +0530, Bran S wrote:
> > > > I am interested in the Comprehensive test suite project.
> > >
> > > I'd suggest building strace with LCOV with different kernel headers
> > > on different arches (./configure --enable-code-coverage && make check &&
> > > make code-coverage-capture) and look at parts of the code (and branches)
> > > that are not covered by tests.  Other possibility is to check out
> > > codecov[1], but it's vaguely representative (it's built against rather
> > > old kernel and only on x86_64).
> > >
> > > [1] https://codecov.io/gh/strace/strace
> >
> > On running `make code-coverage-capture` I am getting below error.
> > ----------
> > lcov  --directory . --capture --output-file
> > "strace-5.5.0.67.269b-coverage.info.tmp" --test-name
> > "strace_5_5_0_67_269b" --no-checksum --compat-libtool  --rc
> > lcov_branch_coverage=1 --gcov-tool "gcov"
> > Capturing coverage data from .
> > Found gcov version: 8.2.1
> > Using intermediate gcov format
> > Scanning . for .gcda files ...
> > Found 313 data files in .
> > Processing libmpers_m32_a-fetch_struct_stat64.gcda
> > geninfo: ERROR: GCOV failed for
> > /home/mybox/strace/libmpers_m32_a-fetch_struct_stat64.gcda!
> > make: *** [Makefile:9957: code-coverage-capture] Error 29
> > ----------
> >
> > Searching about I found a suggestion to make sure that the gcc and
> > gcov versions are same.
> > My gcc and gcov version are same i.e. gcc (GCC) 8.2.1 20180801 and
> > gcov (GCC) 8.2.1 20180801 respectively.
> > Any ideas ?
>
> Interesting.  What is the environment you're using?

Fedora 4.18.16-300.fc29.x86_64

>
> > In the project "Comprehensive test suite" one would need to go through
> > the files at codecov and find out untested portion of the code and
> > then write tests to test those.
>
> More or less, yes.  The spirit is to check various traces and use cases
> (like, how strace behaves, what decoders print in various cases),
> but the coverage is a good baseline for that; at least, lack of coverage
> signals the lack of testing (the opposite, meanwhile isn't true:
> the presence of line/branch coverage doesn't mean by itself that the code
> is thoroughly tested, so the test cases that cover not previously covered
> code usage patterns are very much welcome as well).
>
> > For example: bpf.c is 61% covered
>
> It seems that the system you've used for the testing skips
> bpf-obj_get_info_by_fd* tests.

I haven't run coverage test, 61.26% is mentioned on codecov[1].

>
> > , within bpf.c lines 126 and 127 are
> > untested. So the task at hand would be to write tests so that those
> > lines are executed.
>
> Yes, the bare minimum is like that.  Note, however, that not all code
> is easily reachable (for example, conditions for some out of bounds/memory
> handling routines are rather difficult to set up).
>
> > Is my understanding correct ? Please correct me where I am wrong and
> > please point out what I have missed in understanding.
>
> Yes, you're correct.  Meanwhile. I would suggest to also look into some
> lower-hanging fruits, like untested ioctl decoders (ubi.c, fs_x_ioctl.c,
> kvm.c, and some others are good candidates), unwinder-related stuff
> (unwind*.c and mmap_cache.c), some syscall decoders, even (prctl.c and,
> funnily, process.c, for example).
>

Yes looking into prctl.c.
Thanks.

[1] https://codecov.io/gh/strace/strace/src/master/bpf.c