Send a user-defined Windows-message through JNA

I want a simple but hard thing: to send a specific user-defined Windows-message from the JNA side in order this message to be catched by a message-loop (GetMessage()/DispatchMessage()) on the C++ side and that message loop will be then interrupted. Actually it should be performed by the button click in Swing GUI. My questions and considerations:

1. Suppose for example I defined on the C++ side my own message as #define WM_CUSTOM_MSG (WM_USER+42) and of course add an appropriate if-statement for interruption inside my message-loop on C++ side. But my purpose is to send this message from java.

2. To do this I wrote the following:

   public class User32Ext {
   interface User32Interface extends User32 {
   User32Interface INSTANCE = Native.load("user32",

                User32Interface.class, W32APIOptions.DEFAULT_OPTIONS);
   
        @Override
        HWND FindWindowEx(HWND lpParent, HWND lpChild, String lpClassName, String lpWindowName);
   
        HWND GetTopWindow(HWND hwnd);
   
        HWND GetParent(HWND hwnd);
   
        @Override
        HWND GetDesktopWindow();
   
        int SendMessage(HWND hWnd, int Msg, IntByReference wParam, IntByReference lParam);
   
        void keybd_event(byte bVk, byte bScan, int dwFlags, int dwExtraInfo);
   
        void SwitchToThisWindow(HWND hWnd, boolean fAltTab);
   
    }
   
    private final User32Interface u32 = User32Interface.INSTANCE;
   
    public User32Ext() {
        super();
        // TODO Auto-generated constructor stub
    }
   
    public void sendInterruptMessage(final String windowName) {
        try {
            final User32.HWND hwnd = u32.FindWindowEx(null, null, null, windowName);
            final int msg = 0x400 + 42;
            final User32.WPARAM wparam = new User32.WPARAM();
            final User32.LPARAM lparam = new User32.LPARAM();
            final LRESULT res = u32.SendMessage(hwnd, msg, wparam, lparam);
        } catch (final RuntimeException e) {
            e.printStackTrace();
        }
    }
   

   }

and then just create an object of this dummy class and call sendInterruptMessage with my JFrame's title as parameter in the button event.

Sadly, on the both sides absolutely no effect was detected. I'm pretty much sure I did some fatal error, because I'm not expirienced enough with Windows and JNA programming. So, could you tell me, at least am I right conceptually, implementing the desired result like this? Or there are just some programming errors on the Java side. Thanks!

P.S. if it could be implemented without sending a custom message, but with some standard Windows message, that's perfectly fine, I just want to be sure, the only thing this message does is to interrupt my C++ message loop.

When mapping WINAPI functions with JNA, you must take care to exactly match the Windows API definitions with appropriate Java/JNA types. Some of your mappings are wrong. The last two arguments to SendMessage are a WPARAM and LPARAM, which you appear to be using in your sendInterruptMessage() mapping. These are calling the already-mapped SendMessage function in the JNA project. It doesn't matter what you're putting in your own SendMessage mapping since you're not even calling it, as you're using different types. Delete that. Also delete the two methods you have put @Override on from the superclass. You have no reason to overwrite what they are already doing.

Actually, on further inspection, it appears as if you've copied the code someone else wrote in 2014 from this answer. However, the mappings you need were added to the JNA project in 2017. Copying code without knowing what it does is not a recipe for success. Your entire interface is not needed. Just call JNA's User32 interface.

In your call to FindWindowEx you're passing null for the first three arguments and only placing a String in the fourth argument. This seems to mismatch the API which does not appear to allow null as the third argument. Have you checked whether the returned handle is null? It probably is, and per the API you can use GetLastError to look at that error code, which is likely associated with incorrect arguments.

The docs for that third argument specify,

> If lpszClass is a string, it specifies the window class name. The
> class name can be any name registered with RegisterClass or
> RegisterClassEx, or any of the predefined control-class names, or it
> can be MAKEINTATOM(0x8000). In this latter case, 0x8000 is the atom
> for a menu class. For more information, see the Remarks section of
> this topic.

It does not appear as if you are doing this as required.

If that first call is succeeding (which would surprise me), it would be instructive for you to evaluate the LRESULT return from the existing SendMessage call and see if it indicates success or generates an error code. In fact, you are probably passing it a null handle, in which case it's no surprise that nothing is happening.

Starting out by checking method return values that indicate success/failure and evaluating the error codes should help you debug.

Finally, as ponited out by iinspectable in the comments, the code you're passing isn't a custom message. According to the documentation,

> Message-identifier values are used as follows:
>
> - The system reserves message-identifier values in the range 0x0000 through 0x03FF (the value of WM_USER – 1) for system-defined
> messages. Applications cannot use these values for private messages.
>
> - Values in the range 0x0400 (the value of WM_USER) through 0x7FFF are available for message identifiers for private window classes.
>
> - If your application is marked version 4.0, you can use message-identifier values in the range 0x8000 (WM_APP) through 0xBFFF
> for private messages.
> - The system returns a message identifier in the range 0xC000 through 0xFFFF when an application calls the RegisterWindowMessage function
> to register a message. The message identifier returned by this
> function is guaranteed to be unique throughout the system. Use of
> this function prevents conflicts that can arise if other applications
> use the same message identifier for different purposes.

Reading the WINAPI documentation to know what's expected in each parameter will help you find the right values to pass and is critical to getting expected results from your code.