How does PyTorch know to differentiate functions?

I have some understanding of how PyTorch Autograd works - mostly based on the PyTorch documentation. Conceptually I get that we use the chain rule and multiply the derivatives along the way etc.

My question is how PyTorch knows to differentiate the leaf nodes. Specifically, how does PyTorch know...

• the derivative of x is 1?
• that of sin(x) is cos(x)
• that of log_a(x) is 1/ (x * ln a)

For these kind of functions does PyTorch just use a lookup table?

Also, I believe the implementation of sin, cos etc are library/system dependent. Will DL models trained on different architectures (x86, Arm, Power, GPU etc) give significantly different results (accuracy). Or is this difference just the same as train-to-train variation ?

PyTorch does not use a lookup table, it implements these utilities as autograd.Function classes. These provide the forward and backward implementations of each function (for functional interface to then utilize: eg. torch.sin, torch.log, ...). When an inference is made, the graph is kept in memory from the output tensor, and a grad_fn callback function is accessed (via a backward call, autograd.grad, or other autograph interface) to backpropagate through this implicit graph. Each node in turn calls the backward implementation of its underlying autograd Function.

For more information on Autograd's functionality I would suggest visiting the "How to extend Autograd" page which offers a clear guideline and insights as to how Autograd works under the hood.