[RFC][PATCH] Increase time-out values in test cases

[Helge Deller]

On 8/29/23 12:42, Dmitry V. Levin wrote:
> Hi,
>
> On Fri, Aug 11, 2023 at 10:46:25AM +0200, Helge Deller wrote:
>> This is a RFC, if strace should really be so strict on the various
>> time-out values in some tests, and if those values should be increased.
>>
>> The reasoning is, that if strace is built on package-builder machines
>> (e.g. in my case the debian machines building the debian strace
>> package), the build is happening on a machine which may concurrently
>> build other packages (in another chroot) in parallel.
>> If this happens, it's not guaranteed that a "sleep 1" returns in 1.0x or
>> 1.1x seconds. An example can be seen here:
>> https://buildd.debian.org/status/fetch.php?pkg=strace&arch=hppa&ver=6.4-0.1&stamp=1691669659&raw=0
>> where the strace-r test returned in 1.469987 seconds (which should be
>> basically OK too), but due to the expected return values the test fails
>> and the package isn't successfully built.
>> So, the time until a test is finished may be undeterministic in such
>> scenarios.
>>
>> I'm not asking to apply the patch below as-is, it's just
>> meant to show the issue and help in this RFC.
>> Maybe there is another way to work around that issue?
>>
>> Helge
>>
>> Signed-off-by: Helge Deller <deller at gmx.de>
>>
>> ---
>>
>> diff --git a/tests/count.test b/tests/count.test
>> index c129a9366..fe0420130 100755
>> --- a/tests/count.test
>> +++ b/tests/count.test
>> @@ -30,8 +30,8 @@ grep_log()
>>   }
>>
>>   GENERIC=' *[^ ]+ +0\.0[^n]*nanosleep *'
>> -WALLCLOCK=' *[^ ]+ +(1\.[01]|0\.99)[^n]*nanosleep *'
>> -WALLCLOCK1='100\.00 +(1\.[01]|0\.99)[^n]*nanosleep'
>> +WALLCLOCK=' *[^ ]+ +(1\.[01234]|0\.99)[^n]*nanosleep *'
>> +WALLCLOCK1='100\.00 +(1\.[01234]|0\.99)[^n]*nanosleep'
>>   HALFCLOCK=' *[^ ]+ +0\.[567][^n]*nanosleep *'
>
> I agree this might be an issue for heavily overloaded hardware, but the
> purpose of this and other tests is to check whether the values measured by
> strace are relevant.  Maybe there is a way to redesign these tests not to
> rely on a fixed sleep time of "sleep 1".  For example, the traced programs
> could have printed how much actual sleep time did they have.  However,
> this wouldn't be as simple change as this tweaking of regexps.

That's true.
Nevertheless, the patch I proposed will already help in the meantime.
It's not 100% guaranteed that the tests will succeed then for us, but
it's more likely.
Any chance to apply the patch as-is in the meantime? That would help already.

Helge