[RFC][PATCH] Increase time-out values in test cases

[Dmitry V. Levin]

Hi,

On Fri, Aug 11, 2023 at 10:46:25AM +0200, Helge Deller wrote:
> This is a RFC, if strace should really be so strict on the various
> time-out values in some tests, and if those values should be increased.
> 
> The reasoning is, that if strace is built on package-builder machines
> (e.g. in my case the debian machines building the debian strace
> package), the build is happening on a machine which may concurrently
> build other packages (in another chroot) in parallel.
> If this happens, it's not guaranteed that a "sleep 1" returns in 1.0x or
> 1.1x seconds. An example can be seen here:
> https://buildd.debian.org/status/fetch.php?pkg=strace&arch=hppa&ver=6.4-0.1&stamp=1691669659&raw=0
> where the strace-r test returned in 1.469987 seconds (which should be
> basically OK too), but due to the expected return values the test fails
> and the package isn't successfully built.
> So, the time until a test is finished may be undeterministic in such
> scenarios.
> 
> I'm not asking to apply the patch below as-is, it's just
> meant to show the issue and help in this RFC.
> Maybe there is another way to work around that issue?
> 
> Helge
> 
> Signed-off-by: Helge Deller <deller at gmx.de>
> 
> ---
> 
> diff --git a/tests/count.test b/tests/count.test
> index c129a9366..fe0420130 100755
> --- a/tests/count.test
> +++ b/tests/count.test
> @@ -30,8 +30,8 @@ grep_log()
>  }
> 
>  GENERIC=' *[^ ]+ +0\.0[^n]*nanosleep *'
> -WALLCLOCK=' *[^ ]+ +(1\.[01]|0\.99)[^n]*nanosleep *'
> -WALLCLOCK1='100\.00 +(1\.[01]|0\.99)[^n]*nanosleep'
> +WALLCLOCK=' *[^ ]+ +(1\.[01234]|0\.99)[^n]*nanosleep *'
> +WALLCLOCK1='100\.00 +(1\.[01234]|0\.99)[^n]*nanosleep'
>  HALFCLOCK=' *[^ ]+ +0\.[567][^n]*nanosleep *'

I agree this might be an issue for heavily overloaded hardware, but the
purpose of this and other tests is to check whether the values measured by
strace are relevant.  Maybe there is a way to redesign these tests not to
rely on a fixed sleep time of "sleep 1".  For example, the traced programs
could have printed how much actual sleep time did they have.  However,
this wouldn't be as simple change as this tweaking of regexps.


-- 
ldv