What Speaker of the House-designate had to step down after he admitted having an extra-marital affair?

This one comes to us courtesy of U.S. History Trivia.  It's one of those fun, scandal ones too.

What Speaker of the House-designate had to step down after he admitted having an extra-marital affair?  Was it Newt Gingrich, Roy Blunt, Dennis Hastert or Robert Livingston?

Correct Answer:  Robert Livingston

Some of these guys were just plain boring.  Roy Blunt was never Speaker of the House - don't worry though, it worked out for him, he was elected to the United States Senate last week.  Dennis Hastert was Speaker of the House from 1999-2007 - he only left office when the Republican Party was booted from power after the 2006 elections.  Hastert resigned in November 2007.

Newt Gingrich isn't boring, but he wasn't booted by a sex scandal either.  After leading the Republican Revolution if 1994, Gingrich served as Speaker from 1994 - 1998 and had many high profile controversies with President Clinton.  After the Republican Party took a beating in the 1998 elections, Gingrich stepped aside as Speaker, yielding eventually to Hastert.

Ahh, and here is where Mr. Livingston comes in.  After the 1998 elections, Livingston rounded up supported was was nominated as Speaker by the Republicans without opposition.  However, during this time period, Hustler Magazine (yes, that Hustler) discovered evidence of an extra-marital affair that Livingston had.  This was at the height of the Clinton impeachment trials, during which Clinton had admitted he had an affair with Monica Lewinsky.  Subsequently, Livingston announced his resignation, not only as Speaker but from Congress.