polypath: Reorganise to break up ifaddr handling