Is it bad for your health if you stop having sex?

You might think everyone is doing it more than you, but the reality is plenty of people are going without sex, often quite happily. But what are the physical impacts?