I don’t know the full story, but that is apparently somewhat true. That’s why you’ll hear in games and manga that someone in a fairly auspicious position will get punished by cleaning toilets or something.