Tighten up a lot of casts from unsigned to int which are read by one