I've said for a while now (since about 2010 I think because it was in the depths of the Great Recession) that most voters who are not particularly engaged in politics except every two years, vote only with their perceptions. So, the question becomes, "Do you perceive you're better off now than you were last election?". If you do think you're better off, you vote to affirm the party in power. If you don't think you're better off, you vote for a change.
This is the problem with having only two political parties, both of which are neo-liberal on economic issues. Economic issues are the ones that people use to decide if they're better off now than they were last election. If both parties are, basically, supportive of the 1%, neo-liberalism, low business taxes, austerity, and cutting programs that people rely on, then these voters with a lower level of engagement in politics, will wind up see-sawing back and forth between the only two parties allowed to exist in an increasingly desperate attempt to elect SOMEONE, ANYONE, who will improve their situation.
Sometimes it take four years instead of two, but if you don't improve people's lives, they're going to vote for a change. The reasons don't really matter. '06 and '08 it was the Dems turn because they were the "change" party. Things didn't get significantly better, so the Repubs won in '10. Nothing significant changed for people, so the Dems won in '12. Not much has changed, so this year it's probably the Republicans turn.