"Patent holders have exclusive rights because giving them exclusive rights encourages the progress of science."
This is the ONLY stated reason that Congress has this power. Laws to the contrary... such as the copyright extension designed to let Disney keep control over Mikey Mouse, seem vulnerable to a constitutional challenge since it was not intended to promote progress... only protect profits.
"There is nothing in there about rights holders having to act in a pro social way."
I wonder about that. From a constitutional perspective the monopoly is granted ONLY to encourage progress in the science and useful arts. Everything comes down to that intent... and I'd argue the intent might provide some legal limitations on an inventor. But I suspect this is unexplored legal territory. But then so was the Right's ludicrous bastardization of the Second Amendment to negate the militia... and they prevailed.
"During the term of the rights they can choose not to do anything with the invention, if they want. The limited term assures that society will benefit even if the right holder is a creep."
I'd argue many patents do nothing more than create proprietary monopolies that stifle progress. When the consumer electronics manufactures got together 20 years ago to create the DV mini digital video cassette... they pooled all their best ideas. As so as soon as the consumer format was developed, Sony and Panasonic both developed incompatible pro-sumer variants to create vendor lock. It's difficult to see how such patents promote progress.