Some psychologists estimate that between 15 to 20 percent of couples claimed to have sex no more than 10 times a year, according to a Newsweek story. And a study reported by USA Today stated that 20 to 30 percent of men and 30 to 50 percent of women say they have little or no sex drive. Loss of libido is the most common complaint of women who report they have a sexual disorder, according to the Association of Reproductive Health Professionals. And CNN.com reported that 40 million Americans are not having sex with their partners—at all.

Given these stats, it’s no surprise that the loss of sexual desire—technically called hypoactive sexual desire disorder—has aroused the attention of doctors and patients alike. But what causes it—and what’s the difference between a sexual slump and a sexual disorder?

“Medically it would be when sex actually distresses you,” says Lovera Wolf Miller, MD, coauthor of Womenopause: Stop Pausing and Start Living. Stress is the deciding factor, she says, because you’re not just upset about the situation, you’re distressed by it.

But there’s clearly a difference between loss of libido and loss of pleasure. Miller gives the example of a menopausal woman who has a strong interest in sex (20 percent of women cite higher libidos after menopause) but, because low estrogen levels have reduced her natural lubrication, doesn’t physically enjoy it.

Miller also debunks the myth that aging men never lose their sex drive. “Men go through ‘manopause’ as women go through ‘womenopause’—and with that, their testosterone goes down significantly.”

Low self-esteem can also reduce a person’s sexual desire. “Many men and women have a low self-image of themselves—maybe it’s their weight, their shape, their health and whether or not they’ve been working out,” Miller says. “Their self-image is really a big factor in libido.”

What to do? Try taking time with your partner, Miller says. The goal is to avoid feeling pressured. Relax and just enjoy each other—with or without sex.