[RFC] make strace handle SIGTRAP properly

[Denys Vlasenko]

Hi,

(please don't remove me from CC: when replying)

During recent lkml discussions about fixing some long-standing problems
with ptrace, I had to look in strace source and experiment with it a
bit. (CC-ing some participants).

One irritating thing I noticed is that we *still* don't handle
user-generated SIGTRAPs. There are users who do want that to work:

https://bugzilla.redhat.com/show_bug.cgi?id=162774

Currently, strace "handles" SIGTRAP by code like this:

#if defined (I386)
        if (upeek(tcp, 4*EAX, &eax) < 0)
                return -1;
        if (eax != -ENOSYS && !(tcp->flags & TCB_INSYSCALL)) {
                if (debug)
                        fprintf(stderr, "stray syscall exit: eax = %ld\n", eax);
                return 0;
        }
#elif ...

Think what will happen if SIGTRAP arrives while process is in userspace
and eax happened to contain value equal to -ENOSYS...

In order to make it work properly, that is, to detect SIGTRAPs just like
we do with any other signals, we need to use ptrace(PTRACE_SETOPTIONS,
PTRACE_O_TRACESYSGOOD). Otherwise, syscall traps are indistinguishable
from real SIGTRAPs. Well, there is some extremely fragile and/or
arch-specific way way to distinguish them by looking at GETSIGINFO
results or registers (see above), but why should we bother? Linux ptrace
has PTRACE_O_TRACESYSGOOD for many years now, we _can_ use it.

Another twist in this story is that pesky extra SIGTRAP generated after
execve. If we start handling real SIGTRAPs, we should not be fooled by
it.

And again, even though there may be a difficult way to detect it, I
think we should use the easy one: PTRACE_O_TRACEEXEC option. It disables
post-execve SIGTRAP (and enables intra-execve PTRACE_EVENT_EXEC, which
is easily distinguishable form other stops, so we can ignore it easily).

Roland's worry was that in some old kernels ptrace(PTRACE_SETOPTIONS)
was working, but the options themselves were buggy. I trust him that
there were such kernels, but it's history now, right?
So, I propose to attempt ptrace(PTRACE_SETOPTIONS) only on relatively
recent kernels. (say, starting from 2.6.30? (arbitrarily chosen))

Any objections to this plan?

-- 
vda