You'll always get more money on the private market than the auction prices Carmax pays. That's just the nature of dealing with a re-seller. They're either going to auction your car or try to mark it up for resale. Disclosure: I worked there briefly as a second job, basically for a lark. I liked their basic premise of up-front pricing and the fact they didn't manipulate financing.
Buying from an individual can go great, if you get a hold of an honest car nut or the proverbial "little old man / lady who only drove to church." That's where the best cars will be, and the prices can better because there's no middle man trying to make a buck.
But if you get screwed, your recourse is asking nicely, or a lawsuit. Most people aren't good at lawsuits. As a kid, I once had a woman try to sell me what I finally realized was her ex-husband's car. She had no title, but promised to get one after the sale. She mentioned none of this until I was ready to buy and hung up when I asked more probing questions. Someone probably bought that car.
People are sometimes right to be "scared of making deals for themselves." Depends on where you go, of course -- and many new car dealerships have essentially gone to upfront pricing and no-scam financing, but the old school car sales model -- especially used cars -- was as rigged as a carny game. Not just things like chewing gum on the hose leaks, but outright dirty financing, where they make money by "selling" you the worst possible rate, and then manipulation of the trade-in, warranties, payments, etc. etc.
A lot people -- particularly a certain type of guy -- who think they're geniuses at dealing with car dealers, get screwed as much as anyone else.
If you get a lemon from Carmax -- which of course can happen -- you at least have a point of contact, and some motivation on their part to keep customer good will. They also at least used to have a limited no-questions-asked return policy. You won't get that from a private seller or the corner used-car lot. The closest thing they've got to scams are add-ons like extended warranties, leather seats, and the worst thing I ever saw, add-on DVD systems that seemed badly overpriced.
= new reply since forum marked as read