"The private sector can do everything better than government." If they take that as an article of faith, I don't even want to talk with them. There are whole classes of things government does better, and I would like to discuss how to define those classes. One of them is things that are natural monopolies, like mail delivery. Nobody wants to have five mailboxes. It makes perfect sense to have one agency deliver mail to every address every day (I would add Sunday delivery). I would think packages below a certain size would fall to that agency, as well. Larger packages, more sporadically delivered, maybe to one in a hundred addresses per day, yeah, I could see a private UPS type company doing that.
Some natural monopolies, like electric service, are provided by private companies regulated by a Public Service Commission that has to rule on every rate increase. A few communities have their own power companies. From what I hear, they work pretty well.
Another class is things that have concentrated costs but diffuse benefits. Think infrastructure. Everyone benefits from the interstate highway system. Converting them all to toll roads would be a major pain in the butt, and would confine the cost of supporting them to the few who actually drive on them. (I suppose the toll payers could pass those costs on to their customers. But the convenience of being able to freely hop on and off of taxpayer supported roads increases the efficiency of using such roads tremendously. And voters have a say on maintenance standards.)
Another class is anything where health and safety concerns are paramount. Private enterprise will always, always try to skimp on such costs, in order to fatten their bottom line. There's even a kind of secret rule in economics about this, that if companies aren't polluting and aren't injuring a few customers and employees, and don't pay any legal settlements for the damage they cause, then they are spending too much on anti-pollution and safety measures.