When checking if a connection comes from localhost, don't assume it's an IPv4