[PATCH] rt_sigaction: mask down to 32 bits to hide unusable sign-extension artifact

[Dmitry V. Levin]

On Mon, Jun 02, 2025 at 04:48:06PM +0200, Ahelenia Ziemiańska wrote:
> new_sigaction's sa_flags is (sometimes) an ulong which is (sometimes)
> 64 bits; only 32 bits can ever be used, and currently libcs sign-extend
> SA_RESETHAND to set the top half. Mask it down since it's unused and unusable.
> 
> Before:
>   rt_sigaction(SIGBUS, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER|SA_RESETHAND|0xffffffff00000000, sa_restorer=0x7fb45a2dc050}, NULL, 8) = 0
> 
> After:
>   rt_sigaction(SIGBUS, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER|SA_RESETHAND, sa_restorer=0x7f95242c6050}, NULL, 8) = 0
> 
> Link: https://www.openwall.com/lists/musl/2025/06/01/1
> Link: https://sourceware.org/pipermail/libc-alpha/2025-June/167148.html

In strace, we usually mask out unused bits when the kernel in known to
consistently ignore those bits across all supported kernel versions.

In this case, the kernel behaviour changed by commit v5.11-rc1~136^2~4 [1].
Before that commit, unused bits weren't fully cleared, neither when
storing the userspace-provided sa_flags nor when returning them later
via oldact argument.

Taking this into consideration, I don't think it would be correct for
strace to ignore those bits for kernel versions prior to v5.11.

[1] https://git.kernel.org/torvalds/c/23acdc76f1798b090bb9dcc90671cd29d929834e


-- 
ldv