Replace crash with assertion failure if we attempt to xref a non-existent