Bug 218607

libpsx is a wrapper for thread creation and management that uses -lpthread.

On the face of it your program makes no reference to `pthread_create` in the output of `objdump -x a.out`. The `libpsx` code expects the linking to wrap these function calls with some accounting code that keeps track of POSIX threads.

```
$ pkg-config --libs libpsx
-lpsx -lpthread -Wl,-wrap,pthread_create
```

I'm not familiar with how c++ `<thread>` is implemented. Does it even use `pthread_create` on Linux?

Running `ltrace a.out` on your program does not suggest that the `$(pkg-config --libs libpsx)` linking options are attaching themselves to anything. You should see something like: `__wrap_pthread_create` in the output at thread creation time.

> Running `ltrace a.out` on your program does not suggest that the
> `$(pkg-config --libs libpsx)` linking options are attaching themselves to
> anything.

That's the function it's calling here according to ltrace: std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)())

Should we also have a libpsx++ then? I suppose it'd wrap the above function and use the same sighandling mechanism from libpsx?

> I also tried to convert the C++ program to a C program using pthread_create()
> manually, and it didn't work (one of the threads will just die indicating
> that the sighandler for libpsx isn't installed?). I really don't understand
> how one is supposed to use libpsx.

I finally found out what was wrong with my C version of the same program. You can ignore this part of my initial report.

It seems the only remaining problem now is how one is supposed to use libpsx in C++ programs. Should we have a libpsx++ library that C++ programs link against?

Possibly, but there is a lot of name mangling going on, so I'm not sure.

Ideally, if it can be made to work, we would provide both wrapper calls in the same library and not need a separate library build.

> a lot of name mangling going on

libc++ (LLVM) might be using a different function to wrap as well, so maybe there's yet another function to wrap later. Unfortunately I don't have a handy libc++-based env to test right now. However having just libstdc++ should be good enough for a start anyway as that's the most common ABI one founds in Linux systems by a large margin.

This isn't very promising:

https://stackoverflow.com/a/8405029/14760867

I wonder if a potential workaround more natural to C++ might be to use inheritance to provide, say, a `psx::thread` object, and substitute wrapper methods for creation? It is not clear to me how useful this would be since the wrapping would only apply to code that used `psx::thread`, but it might be workable for some projects.

> This isn't very promising [...]

Why isn't <https://stackoverflow.com/a/8405029/14760867> promising? Doesn't wrapping the mangled name look good enough? I think that should be the way to go.

The real problem should actually come later, adding a C++ compilation unit so you can access std::thread::native_handle() to do the bookkeeping for the new thread.

If you do decide to add a C++ compilation unit to libpsx, you complicate the build for libpsx by adding a new dependency (a C++ compiler).

If you decide to add the C++ compilation unit in a different library, then you complicate orchestration between the new library and libpsx (libcap using libpsx should work on C++ projects that link against the new library as well... and C libraries in the C++ project calling pthread_create() directly should also work here).

> I wonder if a potential workaround more natural to C++ might be to use
> inheritance to provide, say, a `psx::thread` object, and substitute wrapper
> methods for creation? It is not clear to me how useful this would be since
> the wrapping would only apply to code that used `psx::thread`, but it might
> be workable for some projects.

This wouldn't work for my C++ project. I don't control 3rd party libraries that might create threads. And I do need to use std::thread so functions such as std::notify_all_at_thread_exit() work correctly. std::*_at_thread_exit() functions trigger some hardcoded libstdc++ functions that are guaranteed to execute only after thread-local variables are destroyed.

How many mangled names would we have to wrap though? Is the mangling even consistent for single compiler, libc++ variant?

> How many mangled names would we have to wrap though? Is the mangling even
> consistent for single compiler, libc++ variant?

C++ mangling is very stable. The developers for libstdc++ and libc++ also take ABI stability very seriously.

It'd require mangling for two functions in total:

* libstdc++ thread constructor
* libc++ thread constructor

However I suggest to start with just libstdc++ to gather experience. It can expand to support libc++ in later releases.

So I took a look at this (try.cc):

////
#include <iostream>
#include <thread>
#include <sys/psx_syscall.h>

#include <unistd.h>
#include <sys/prctl.h>
#include <sys/syscall.h>

extern "C" void __wrap__ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE() {
    std::cout << "oh no\n";
    exit(1);
}

int main()
{
  std::thread t{[]() {
      int x = prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0, 0);
      sleep(2);
      int y = prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0, 0);
      std::cout << "thread was: " << x << " is: " << y << "\n";
  }};
  int x = prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0, 0);
  sleep(1);
  psx_syscall6(__NR_prctl, PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0, 0);
  int y = prctl(PR_GET_NO_NEW_PRIVS, 0, 0, 0, 0, 0);
  std::cout << "main was: " << x << " is: " << y << "\n";
  t.join();
}
////

Compiled with:

$ g++ try.cc $(pkg-config --libs libpsx) -Wl,-wrap,_ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE
$ ./a.out
oh no

Which does imply we can intercept the thread constructor. I just am not yet sure how to chain to the real version.

I've been looking some more at the c++ case. I'm close to releasing libcap-2.71, but am thinking I might aim to support the c++ threads as described in this bug in libcap-2.72.

Em qui., 24 de out. de 2024 às 00:39, <bugzilla-daemon@kernel.org> escreveu:
> I've been looking some more at the c++ case. I'm close to releasing
> libcap-2.71, but am thinking I might aim to support the c++ threads as
> described in this bug in libcap-2.72.

Thank you for looking into this.

This is resolved with these two commits (that barring some sort of disaster will be in libcap-2.72 release date unknown):

https://git.kernel.org/pub/scm/libs/libcap/libcap.git/commit/?id=12e163ac21f11a8f8760305b9f60a6b7819aee7b

and this one that adds a test:

https://git.kernel.org/pub/scm/libs/libcap/libcap.git/commit/?id=db256cd8f678c2928c363f0ebf48c68d0010650c

$ cd .../libcap/contrib/bug218607/
$ make
$ ./threadcpp 
before got in:0 out:0
after got in:1 out:1
PASSED

Read the thread.cpp code to figure out why this means things are working. (Hint: in is inside the thread, out is inside the parent process main thread.)

(In reply to Andrew G. Morgan from comment #13)
> This is resolved with these two commits (that barring some sort of disaster
> will be in libcap-2.72 release date unknown):
> 
> https://git.kernel.org/pub/scm/libs/libcap/libcap.git/commit/
> ?id=12e163ac21f11a8f8760305b9f60a6b7819aee7b https://driftbossgame.co/
> 
> and this one that adds a test:
> 
> https://git.kernel.org/pub/scm/libs/libcap/libcap.git/commit/
> ?id=db256cd8f678c2928c363f0ebf48c68d0010650c
> 
> $ cd .../libcap/contrib/bug218607/
> $ make
> $ ./threadcpp 
> before got in:0 out:0
> after got in:1 out:1
> PASSED
> 
> Read the thread.cpp code to figure out why this means things are working.
> (Hint: in is inside the thread, out is inside the parent process main
> thread.)

GREAT JOB

Appreciate that, but please feel free to stress test this! I'd really like to make sure the testing I have done is sufficient. If there is something I have missed, I'd like to add more test cases and fix those before releasing lib{cap,psx}-2.72.

I guess I should have added that libpsx, as of the above commits, no longer just operates on pthread's (and their corresponding APIs), but operates on the underlying Linux threads (LWPs).

The need to link/wrap against pthread_create() is still there for now, but the reason for that is obscure and related to transparent libcap linkage. I hope to find a way to remove the need for the 'wrap'ing of that one function call at some point. Likely not in the libcap-2.72 release.

I've created this bug to explore how to do that:

https://bugzilla.kernel.org/show_bug.cgi?id=219456

So far, my searching doesn't work for both libpsx.a and libpsx.so setups. The -wrap method does.