Enable routes to the pod/service networks to be added for multi-homed pods

There are occasions where multi-homed pods are required, using e.g. multus. Sometimes the default gateway is required to be set on those pods such that the interface controlled by calico is no longer the default gateway. In this case, those pods will no longer be able to communicate with any other pods, for example the routing table would look something like this:

default via 172.168.8.1 dev net1 
169.254.1.1 dev eth0 scope link 
172.168.8.0/22 dev net1 scope link  src 172.168.8.249 

The issue arises because Calico expects to be single homed, and by default creates a default route to 169.254.1.1. When the default route is changed, the pods no longer know how to communicate with other pods on the same network.

Expected Behavior

This problem could be solved by providing the ability in configuration to add explicit routes to the pod and service networks (v4 or v6 or for dual stack, both) to handle the situation where the pod is multi-homed and the default route is configured on the secondary interface. Calico already "knows" these networks, so the configuration would not need to be arbitrary.

Current Behavior

The issue arises because Calico expects to be single homed, and by default creates a default route to 169.254.1.1. When the default route is changed, the pods no longer know how to communicate with other pods on the same network.