What you're describing is misogyny, in a nutshell.

[nomorenomore08]

I think some make the mistake of leaning too hard on the literal dictionary definition.

Think of homophobes saying "What homophobia? I'm not afraid of teh gheys!"

Please note that I'm not calling you (or anyone else) a bigot here. Just trying to draw an analogy.