How to share pthread synchronisation primitives between C++ and Rust?

I have a C++ program and a Rust program, and between them I have successfully got them talking over POSIX shared memory (C++ and rust).

What I am now trying to do is synchronise them. I already managed to create a working, but inefficient, primitive system using an atomic bool (creating the AtomicBool on the rust side like this).

However, I would really like to use a mutex/condvar to synchronise between the threads, and this is where I am stuck.

I seem to be able to initialise the C++ side of it, following this example pretty much word for word.

I have attempted to translate it directly into rust:

    let raw_shm = shm.get_shm();

    let mut mtx_attrs = MaybeUninit::<nix::libc::pthread_mutexattr_t>::uninit();
    if unsafe { nix::libc::pthread_mutexattr_init(mtx_attrs.as_mut_ptr()) } != 0 {
        panic!("failed to create mtx_attrs");
    };
    let mtx_attrs = unsafe { mtx_attrs.assume_init() };

    let mut cond_attrs = MaybeUninit::<nix::libc::pthread_condattr_t>::uninit();
    if unsafe { nix::libc::pthread_condattr_init(cond_attrs.as_mut_ptr()) } != 0 {
        panic!("failed to create cond_attrs");
    };
    let cond_attrs = unsafe { cond_attrs.assume_init() };

    if unsafe {
        nix::libc::pthread_mutexattr_setpshared(
            &mtx_attrs as *const _ as *mut _,
            PTHREAD_PROCESS_SHARED,
        )
    } != 0
    {
        panic!("failed to set mtx as process shared");
    };

    if unsafe {
        nix::libc::pthread_condattr_setpshared(
            &cond_attrs as *const _ as *mut _,
            PTHREAD_PROCESS_SHARED,
        )
    } != 0
    {
        panic!("failed to set cond as process shared");
    };

    // I know that these offsets are correct, having used `offsetof` on the C++ side
    let mtx_start = unsafe { &raw_shm.as_slice()[3110416] };
    let mtx = unsafe { &*(mtx_start as *const _ as *const pthread_mutex_t) };
    let cond_start = unsafe { &raw_shm.as_slice()[3110440] };
    let cond = unsafe { &*(cond_start as *const _ as *const pthread_mutex_t) };

    if unsafe {
        nix::libc::pthread_mutex_init(&mtx as *const _ as *mut _, &mtx_attrs as *const _ as *mut _)
    } != 0
    {
        panic!("failed to init mtx");
    };
    if unsafe {
        nix::libc::pthread_cond_init(
            &cond as *const _ as *mut _,
            &cond_attrs as *const _ as *mut _,
        )
    } != 0
    {
        panic!("failed to init cond");
    };

All of that passes with return values of 0... so far so good.

I can now test it in one of two ways:

1. I can set the trivial C++ program going and have it stop waiting at the condvar:

if (pthread_mutex_lock(&shmp->mutex) != 0)
    throw("Error locking mutex");
if (pthread_cond_wait(&shmp->condition, &shmp->mutex) != 0)
    throw("Error waiting for condition variable");

and in rust:

let sig = unsafe { nix::libc::pthread_cond_signal(&cond as *const _ as *mut _) };
    dbg!(sig);

Despite returning 0 (i.e. success), my C++ program is not released past the condvar; it remains waiting as if it never received a signal.

2. I can set of another trivial C++ program which endlessly signals the condition variable in a loop:

    for (unsigned int count = 0;; count++) {
        if (pthread_cond_signal(condition) != 0)
            throw("Error")
        // sleep for a bit
    }

and then in rust, something like:

    loop {
        if unsafe { nix::libc::pthread_mutex_lock(&mtx as *const _ as *mut _) } > 0 {
            panic!("Failed to acquire lock")
        };
        if unsafe {
            nix::libc::pthread_cond_wait(&cond as *const _ as *mut _, &mtx as *const _ as *mut _)
        } > 0
        {
            panic!("Failed to acquire lock")
        };
    }

Doing it this way around, the call to lock the mutex is successful, but I get an EINVAL on pthread_cond_wait defined here, which I cannot seem to rectify...

I feel like I'm close... any thoughts on how to get this to work? (this is mostly just a proof of concept).

For posterity, I have managed to get this working.

To clarify how the program is architectured, there are two binaries: one C++ and one rust. The Rust program spawns the C++ program using std::process::Command.

Error handling and imports elided for brevity.

1. The rust program starts and creates a new shared memory block (removing an existing block if it exists, to ensure the program always starts in a fresh state). I use the shared_memory crate to handle the details for me, and that also provides useful helpers such as access to a raw pointer to the start of the memory block.

The shared memory block is structured like the following:

#[repr(c)]
struct SharedMemoryLayout {
    ready: std::sync::atomic::AtomicBool,
    mutex: libc::pthread_mutex_t,
    condition: libc::pthread_cond_t,
}

Shared memory blocks are initialised with zeros, so ready will always be falseto begin with.

2. The rust program spawns the C++ program with std::process::Command::spawn and then waits in a loop until ready is true.

let proc = Command::new("/path/to/c++/binary").spawn().unwrap();

let ptr: *mut u8 = // pointer to first byte of shared memory block;
let ready: &AtomicBool =  unsafe { &*(ptr as *mut bool as *const AtomicBool) };

loop {
    if ready.load(Ordering::SeqCst) {
        break
    } else {
        thread::sleep(Duration::from_secs(1));
    }
}

3. The C++ program opens the shared memory block and mmaps it into its local address space.

struct SharedMemoryLayout
{
    std::atomic_bool ready;
    pthread_mutex_t mutex;
    pthread_cond_t condition;
};

int fd = shm_open("name_of_shared_memory_block", O_RDWR, S_IRUSR | S_IWUSR);
struct SharedMemoryLayout *sync = (SharedMemoryLayout *)mmap(NULL, sizeof(*sync), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

4. The C++ program carries on and proceeds to initialise the mutex and the condition, before marking the memory block as ready.

pthread_mutexattr_t mutex_attributes;
pthread_condattr_t condition_attributes;

pthread_mutexattr_init(&mutex_attributes);
pthread_condattr_init(&condition_attributes);

pthread_mutexattr_setpshared(&mutex_attributes, PTHREAD_PROCESS_SHARED);
pthread_condattr_setpshared(&condition_attributes, PTHREAD_PROCESS_SHARED);

pthread_mutex_init(&sync->mutex, &mutex_attributes);
pthread_cond_init(&sync->condition, &condition_attributes);

pthread_mutexattr_destroy(&mutex_attributes);
pthread_condattr_destroy(&condition_attributes);

std::atomic_bool *ready = &syncp->ready;
ready->store(true);

And then enter a loop signalling on the condition:

for (unsigned int count = 0;; count++) {
    // do something
    sleep(1);
    pthread_cond_signal(&sync->condition);
}

5. Now, the rust program will have been released from the loop in step 2). Materialise the mutex and condition that were initialised in step 4).

let mutex = unsafe {ptr.offset(4) as *mut pthread_mutex_t};
let condition = unsafe {ptr.offset(32) as *mut pthread_cond_t};

And now we can wait on the condition, getting notified by the C++ program.

loop {
    unsafe {
        pthread_mutex_lock(mutex);
        pthread_cond_wait(condition, mutex);
        pthread_mutex_unlock(mutex);
        
        // Do something
    }
}