You can't be sexist against men in a patriarchal society, similarly you can't be sexist against women in a matriarchal society (if such a thing existed). Sexism, racism etc is all about power, those with the power can use their status to discriminate, those without power cannot. It's like saying that in aparthied South Africa that blacks could be racist against whites - it simply has no impact when a group considered to be, and treated as, inferiors can have any impact by claiming superiority - they don't have the power to exercise discrimination.