Why do political sex scandals matter?

Ideally they shouldn't. but there's also a matter of public trust involved, and people really don't want scumbags representing them in government.