How can Thread.Join() in .NET return before the thread has terminated?

I have a process that spawns 2 threads. One thread reads from a web socket, and the other writes to the same web socket. Once everything is initialized and the web socket has connected, the main thread blocks on the 2 threads it created in the following function:

public void BlockUntilThreadsAreDone()
{
    if (_incomingThread.IsAlive)
        _incomingThread.Join();
    if (_outgoingThread.IsAlive)
        _outgoingThread.Join();
}

Everything runs fine for a while. But after several minutes, I discovered that the main thread will return from this function. No exceptions are thrown, and I don't see any unusual logging. The function simply returns because the Thread.Join() calls returned.

I modified the function to wait on an event that both threads will set when they are shutting down as follows:

private readonly ManualResetEventSlim _messagingComplete = new(false);

public void BlockUntilThreadsAreDone()
{
    if (_incomingThread.IsAlive)
        _incomingThread.Join();
    Console.WriteLine($"Incoming thread state={_incomingThread.ThreadState}");

    if (_outgoingThread.IsAlive)
       _outgoingThread.Join();
    Console.WriteLine($"Outgoing thread state={_outgoingThread.ThreadState}");

    _messagingComplete.Wait(_cancellationToken);
}

The thread objects are created like this

_incomingMessageThread = new(HandleIncomingMessages) { Name = "Incoming" };
_outgoingMessageThread = new(HandleOutgoingMessages) { Name = "Outgoing" };

private async void HandleIncomingMessages()
    try
    {
        Console.WriteLine("PapiClient HandleIncomingMessages starting");

        do
        {
            try
            {
                Message? message = await ReceiveIncomingMessageAsync();
                // Handle each message asynchronously so we can keep receiving more messages
                _ = Task.Run(
                    () =>
                    {
                        HandleIncomingMessage(message);
                    },
                    _cancellationToken
                );
            }
            catch (OperationCanceledException)
            {
                break;
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Exception while handling messages: {ex}");
            }
        } while (!_cancellationToken.IsCancellationRequested && Connected);
    }
    catch (ObjectDisposedException) { }
    finally
    {
        _messagingComplete.Set();
        Console.WriteLine(
            $"PapiClient HandleIncomingMessages finishing: cancel={_cancellationToken.IsCancellationRequested}, connected={Connected}"
        );
    }
}

Now the code works as I would expect (i.e., the function only returns once we're done processing). Technically it's not the same since this returns once the first thread completes, but that's close enough for my purposes.

Interestingly, I see that a few minutes after the process started, both threads will be eventually log that their state is ThreadState.Stopped. The threads most definitely are not stopped, though, because I can tell the process continues to send and receive data over the web socket, and there are no other threads created to work with the web socket. In addition, all the normal processing that happens when the threads terminate doesn't occur at that point.

The documentation says that Join "blocks the calling thread until the thread represented by this instance terminates." The documentation also says that "a thread can never leave the Stopped state."

I am baffled. Why would Thread.Join() return and a thread indicate its state is Stopped when it most definitely has not stopped?

I suspect this might have something to do with networking and/or web sockets in particular. I ran a packet capture while this was happening, and I noticed a few milliseconds before the problem occurred, the websocket sends an unsolicited PONG message. The unsolicited PONG isn't a problem by itself, but those are only sent every 30 seconds, so it seems unusual that the spurious Thread.Join() would happen just a few milliseconds after a timer wakes up and sends the PONG message.

I couldn't find an issue about this on the .NET core github issues list. Any ideas?

Looking at your repo on Github (by the way your question should really be self-contained), the two functions HandleOutgoingMessages and HandleIncomingMessages are marked async void. Which means as soon as they hit their first await they are going to end, and post the continuation to the threadpool.

You seem to be expecting that the thread they are on will somehow pause, and pick up when the await continuation happens. This is not true. Your thread ends, and the threadpool will pick it up.

This is essentially a confusion between threads and tasks. You are mostly using tasks and async, I suggest you stick with that, and therefore use Task.Run to offload the Handle functions.

You need the following changes

private readonly Task _incomingMessageThread;
private readonly Task _outgoingMessageThread;

private async Task HandleIncomingMessages()
{

private async Task HandleOutgoingMessages()
{

Remove the lines in the constructor beginning _incomingMessageThread = and _outgoingMessageThread =.

Replace the lines _incomingMessageThread.Start(); _outgoingMessageThread.Start(); with

_incomingMessageThread = Task.Run(HandleIncomingMessages, _cancellationToken);
_outgoingMessageThread = Task.Run(HandleOutgoingMessages, _cancellationToken);

At various points when you need to wait for the threads to finish, just do

await Task.WhenAll(_incomingMessageThread, _outgoingMessageThread);

You do not need to check whether they have finished, they will always have finished after that.

Alternatively

try
{
    await Task
        .WhenAll(_incomingMessageThread, _outgoingMessageThread)
        .WaitAsync(TimeSpan.FromSeconds(DISCONNECT_TIMEOUT));
}
catch (OperationCanceledException)
{
    if (!_incomingMessageThread.IsCompleted)
        Console.WriteLine("Incoming task not stopping");
    if (!_outgoingMessageThread.IsCompleted)
        Console.WriteLine("Outgoing task not stopping");
}