What's so bad about dairy?


So I've seen and heard people literally avoiding milk and generally dairy like the plague, without being lactose-intolerant, and claiming that dairy is bad and unhealthy. Tbh I'd trust the opinions of FB community much more than what I'd read if I Google this potentially foolish question. Some say it's cause milk is high in sugar, others are worried about the fat, etc etc. What do you guys believe? Can someone eat healthily without cutting dairy?