How to create code for existing .exe file without knowing source code?

What is the general approach to something like the following:

Lets say I wanted to create code that would automatically trade stocks on the Thinkorswim desktop app, how can I look up stocks, set trade parameters, and execute without knowing what the functions are or even having access to those functions?

Or say I want to create code that would perform automatic google searches so functions like new tab and filling in the url field can be carried out in my own code?

Or suppose I had a mobile game on Bluestacks (Android Emulator) and I wanted to create a bot that played the game based on a certain algorithm.

How would one go about these things without knowing the source code to the .exe they want to manipulate? I know people do things like this regularly, so what is their approach?

some examples that you are asking is called reverse engineering and it's a very complicated process, its not impossible but you are trying to achieve something that will require more time than creating it by your own.

An example of what you might want to achieve:
https://github.com/zeldaret/mm

You can see here that there are still lot of tasks to work on... and this game is from 2000.
https://trello.com/b/ruxw9n6m/majoras-mask-decompilation

Also if you want to control software itself, in a custom GUI and there is no api, its gonna be a problem.

What you might be able to create is a bot to control the mouse and interact with your GUI on a given resolution.

If you are looking for a webapp solution, you can create automation bots with Selenium and other software to interact with the DOM on a browser, also developers might change the page and your bot will break.

Have fun programming those.