According to one study, published in Current Biology, genetics might not have much to do with it.
The researchers found that the types of faces we fancy are influenced more by our personal experiences in life than anything else.
The study found that identical twins had different ideas of who they thought was attractive, suggesting a genetic predisposition wasn't a factor. Overall, genetics explained just a fifth of the variation in people's preferences.
The authors concluded that these results could explain how models and celebrities can make money from their good looks, while friends constantly disagree about who is to their taste.
A study from 2016 found that rather than men having a preference for certain features like lips or eyes, they went for more of an overall look.
The researchers recruited 169 men in France and showed them two sets of random women's faces. They were asked to rate them on a scale of zero to 20 in terms of attractiveness.
Results showed that overall, the faces the men found most attractive were "sparsely coded" — that means the ones that were symmetrical, more plain, and had no distinguishing features.
9. Looking like your parents
Some research suggests that we tend to find people who look like our opposite sex parent attractive.
According to research from St Andrews, we are attracted to the features that our parents had when we were born, possibly because we see them as our first caregiver, and associate positive feelings with their features.
In one study from 2002, researchers asked participants to rate how attractive faces of different ages were.
"We found that women born to 'old' parents (over 30) were less impressed by youth, and more attracted to age cues in male faces than women with 'young' parents (under 30)," the authors wrote. "For men, preferences for female faces were influenced by their mother's age and not their father's age, but only for long-term relationships."
Also, in a follow up study, a sample of 697 men and women showed people were more likely to have romantic partners who had the same eye and hair colour as their opposite-sex parents.
Read next on Business Insider: Women are more attractive in groups than when they're alone because of something called the 'cheerleader effect'