I'm reading, The Oxford Conspirators: A History of the Oxford Movement, 1833-1845 by Marvin R. O'Connell.

My director, Dr. John T. Ford, notes that O'Connell is somewhat anit-Oxford in this book, so note the bias.

I think I finally have this down. The Tories were the party identified with the landed aristocracy, so to speak. They were high church, Anglican, Catholic, etc. On the other hand, the Whigs were the low Church, Puritan types. Now, none of them were representative of the working classes but the Whigs had the perception of not being artistocracy. Thus during the period of the French revolutions, etc they were at a disadvantage because England did not want to become France. But after Waterloo, the economic downturn that ensued quickly turned into a politcal disadvantage for the Tories, who had kept power for 30 years and the Whigs could use this opportunity to oust the Tories.