Unfortunately that's just the nature of the beast with this poorly developed game. You kind of have to except the loss, stick to the lobby, and then hope that the next game shuffles the players sufficiently enough to bring more balance. In most cases the remaining stooges from the original game will also drop out at the end of the game...thusly bringing the opportunity of better players populating the next game. At least that's what I've found.
The problem with spectating out is that you can truly waste 45 mins to an hour of your life jumping from lobby to lobby in hopes of ending up in a decent game.