Maybe some alien cultures see clothes as unnecessary. The armor had a purely practical use up until he removed it.
At a certain point everyone stopped wearing armor, since it wasn't strong enough to protect from ki blasts of a fighter above a certain caliber, which everyone became.