strace fails to trace call stack of clone from pthread_create

[Masatake YAMATO]

On Fri, 2 Nov 2018 09:46:06 -0400, Zhouyang Jia <jiazhouyang09 at gmail.com> wrote:
> FYI, I also found clone(2) from fork(3) works fine, while clone(2)
> from pthread_create(3) has the bug.
> So, I guess the problem might be caused by thread mechanism.

Interesting. I'd like to see your source code of program for testing
clone(2).

       int clone(int (*fn)(void *), void *child_stack,
                 int flags, void *arg, ...
                 /* pid_t *ptid, void *newtls, pid_t *ctid */ );

I wonder what you passed to `child_stack'.

Masatake YAMATO

> Best,
> Zhouyang
> 
> On Fri, Nov 2, 2018 at 2:49 AM Masatake YAMATO <yamato at redhat.com> wrote:
> 
>> > Hi everyone,
>> >
>> > I find strace fails to trace call stack of clone syscall (called by
>> > pthread_create) in both main thread and child thread.
>> >
>> > How to reproduce:
>> >
>> > Get a simple pthread_create example:
>> > http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html
>> > Compile: gcc -pthread test_pthread.c -o test_pthread
>> > Trace: strace -k -ff -o test_pthread ./test_pthread
>> >
>> > In main thread:
>> >     clone(child_stack=0x7f56c52b1ff0,
>> >
>> flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
>> > parent_tidptr=0x7f56c52b29d0, tls=0x7f56c52b2700,
>> > child_tidptr=0x7f56c52b29d0) = 69700
>> >      > /lib/x86_64-linux-gnu/libc-2.23.so(clone+0x31) [0x1073e1]
>> >
>> > In child thread:
>> >     set_robust_list(0x7f56c52b29e0, 24) = 0
>> >      > /lib/x86_64-linux-gnu/libpthread-2.23.so(start_thread+0x64)
>> [0x7654]
>> >      > /lib/x86_64-linux-gnu/libc-2.23.so(clone+0x6d) [0x10741d]
>> >
>> > We can see the call stacks of both main thread and child thread start
>> from
>> > libc. Is there any way to get the call stack from the main function? I'm
>> > not sure whether this is a bug or not.
>>
>> It looks a bug.
>>
>> About the most of all system calls, strace captures its stack trace after
>> kernel executes the system call. However, about some system calls,
>> caputirng
>> after executing is not useful. I put makers "SE" on such a system call in
>> syscallent.h. SE is defined as STACKTRACE_CAPTURE_ON_ENTER in code.
>> I put SE markers on execve and _exit for exapmles.
>>
>> I should put the maker on clone, fork, and vfrok, too.
>> Before writing this mail, I expected the bug you reported will be fixed by
>> putting
>> the marker on the system calls like:
>>
>> diff --git a/linux/x86_64/syscallent.h b/linux/x86_64/syscallent.h
>> index 63ec52e8..2be891bd 100644
>> --- a/linux/x86_64/syscallent.h
>> +++ b/linux/x86_64/syscallent.h
>> @@ -54,9 +54,9 @@
>>  [ 53] = { 4,   TN,             SEN(socketpair),
>> "socketpair"            },
>>  [ 54] = { 5,   TN,             SEN(setsockopt),
>> "setsockopt"            },
>>  [ 55] = { 5,   TN,             SEN(getsockopt),
>> "getsockopt"            },
>> -[ 56] = { 5,   TP,             SEN(clone),                     "clone"
>>              },
>> -[ 57] = { 0,   TP,             SEN(fork),                      "fork"
>>               },
>> -[ 58] = { 0,   TP,             SEN(vfork),                     "vfork"
>>              },
>> +[ 56] = { 5,   TP|SE,          SEN(clone),                     "clone"
>>              },
>> +[ 57] = { 0,   TP|SE,          SEN(fork),                      "fork"
>>               },
>> +[ 58] = { 0,   TP|SE,          SEN(vfork),                     "vfork"
>>              },
>>  [ 59] = { 3,   TF|TP|SE|SI,    SEN(execve),                    "execve"
>>               },
>>  [ 60] = { 1,   TP|SE,          SEN(exit),                      "exit"
>>               },
>>  [ 61] = { 4,   TP,             SEN(wait4),                     "wait4"
>>              },
>>
>>
>> However, though I put the maker on them, I cannot get expected result
>> either using libunwind or libdw.
>>
>> I read the implemention of the wrapper for clone systecall.
>> It is not written in C but in x86-64 Assembly language.
>> So I guess the frame structure(?) that the unwinder expects are
>> not constructed when executing clone syscall. So unwinder
>> cannot capture a stack trace in the context.
>>
>> I have to take much more time for understanding the issue.
>>
>> Masatake YAMATO
>>
>> > Thanks,
>> > Zhouyang
>>