If he won the election the whole world would see what a clueless incompetent "leader" he is. His ego would be crushed & in Trump's world it's all about stroking his ego, maintaining his charade. He knows he can get away with verbal bluster on the campaign trail but if he became POTUS everyone would expect him to actually *DO* things ... & he's got nothing. He's got nothing & he knows it. His whole charade would come crashing down, & with it, his fragile ego.
He won't quit, that would be on him, he'd have to own it, & his ego won't let him be a quitter. To him it's better to lose because then he can blame the loss on someone else (the RNC, a crooked system, etc).
Look at all his actions, are they the actions of someone who really wants to win? Insulting every voter demographic needed to win (a majority of the electorate)? Doing almost no fundraising & refusing to fund his campaign (he's only given loans, not contributions)? Having no real campaign organization? Hiring just a hand full of state-level campaign staff (30 people for 50 states, with NONE in critical states like Ohio)? Turning against the press & making them his enemy? Turning against his own party (thus ensuring any help they give will be reluctant & modest)?
I could go on but I think you get the point. These aren't the actions of a person who wants to win, they're the actions of a person who's afraid of winning because winning would expose them for the incompetent huckster they really are & utterly destroy their carefully-crafted charade.
And don't forget, Trump didn't enter the race to win. He decided to run as a protest candidate & thought getting 20% would be good, that was his goal. Going all the way wasn't part of the plan.