Non complete graph as map for VRP with or tools

I have to solve a vehicle routing problem and I want to use or tools in python for that.

The problem is the map in which the routing should be done is not complete in a sense that not each node is connected to all other nodes by an edge.

From what I see in the documentation I have to create a distance matrix or distance callback but how do I encode the information of an edge not being present?
Or do I have to calculate the shortest path in the map to the respective node and so fill the distance matrix with the distance of that calculated path.

All this seems as if there was some obvious solution to this I’m not seeing.

The problem you are solving is a VRP, not a TSP.
In that sense, the distance matrix indicates the shortest path from one node to another, not if there is an edge between the two nodes.

A solution of the VRP problem will visit all nodes of the distance matrix once. But it does not forbid visiting multiple edges of the underlying road network multiple times.

Now, if two nodes are really not connected (on the complete graph), then you can put a distance that is large enough to indicate it is not possible.

Since the TSP assumes a complete graph you could use Floyd–Warshall to compute the pairwise shortest distances (if you want to find the shortest path that visits all nodes) or you add a large enough weight for all edges, that are non-existent. On this matrix you then compute the shortest path.