Article rating: thumbs, stars or emoji on a content page?

Readers came to read, not to rate. Jakob Nielsen of NN/g put it plainly in 2004: "The highest response rates come when surveys are quick and painless." Krosnick and Presser, reviewing questionnaire research, put the difference in decisions: a two-point scale asks only for a direction, a longer one for a direction and how far. On a page where nobody has to answer, that second decision makes some readers scroll past.

More points do hold more information. Krosnick and Presser conclude that 7-point scales are probably best in many cases, and report a study where answers on a 6-point scale had a reliability of 0.83, falling to 0.71 when recoded into two groups. But you are not measuring one reader precisely. You want to know which pages fail most often, and one-tap answers from many readers show that better than precise answers from a few.

Two options carry one known bias, acquiescence: the tendency to say "yes" or "agree" whatever the question, which Krosnick and Presser note shows up in yes/no questions too. Live with it by asking every page the same question and comparing pages with each other, not with a fixed target.

A Yes / No question gives you one number per page, the share who said Yes, so you can rank pages and spot a drop after an edit. A star average hides its shape. A page where everyone gives 3 stars and a page where half give 1 and half give 5 both average 3. The first is fine; the second works for one group and fails another, and needs a different fix.

Before you act on a star average, look at how many readers chose each star. Read a Likert scale as the share who picked one of the two agree points (the top two box), and distrust a slider average to the decimal point: it looks more precise than the drag behind it.

Some star rating widgets show readers the average, like the stars under a recipe. Others collect the rating and show it only to the people who run the site. The two do different jobs.

A public average tells readers what others thought, which suits recipes, products and reviews, where readers are deciding whether to try something. It also shows each reader how others voted before they choose, so later ratings are no longer given independently.

A public widget does not bring stars to an article in Google results either. Google shows review stars only for certain types, such as recipes, products, books, courses, events and software apps, and articles are not on the list. Where stars are eligible, Google asks that the aggregate rating be visible on the page.

On articles, guides and docs pages, the rating is for you: it tells you which pages to fix. Keep it private and ask the same question on every page, so you can compare pages and nobody votes on what the crowd already said.

Stars and numbers have no words, so each reader decides what a 3 means. NN/g notes that people find scales labelled with words easier to understand than unlabelled ones, and Krosnick and Presser report that reliability is higher when every point has a word label.

A middle option can attract readers who do not want to think, but without one, readers who really are neutral must pick a side they do not hold. Krosnick and Presser weigh both and conclude that offering a midpoint is desirable. Readers use it: when Pew Research Center tested a 5-point and a 7-point scale in France, Spain and the UK, the midpoint was the most common answer on both.

With stars, put words at both ends, such as "Not useful" and "Very useful". With a Likert scale, label every point and balance the two sides; the Likert scale generator writes the labels for you. Add "Not applicable" when the question may not apply to everyone, as NN/g suggests, so those readers do not land in the middle.

Readers do not agree on what rating faces mean. A University of Minnesota study published at ICWSM in 2016 found that people looking at the same rendering of the same emoji disagreed on whether it was positive, neutral or negative 25% of the time, and more so across the versions drawn by different platforms.

Screen readers announce an emoji by its short name, which the Unicode emoji standard also calls the TTS name, for text-to-speech. A blind reader hears "slightly frowning face, neutral face, slightly smiling face". Thumbs are milder, but an icon alone still leaves readers unsure what they are rating. Give each button a word, such as Yes and No, and make that word its accessible name.

WCAG 2.2 asks for targets of at least 24 by 24 CSS pixels at level AA, and bigger for important controls. Two Yes / No buttons clear that easily, and five stars usually do. A 0 to 10 scale has eleven: on a phone 320 pixels wide that leaves about 29 pixels each before any spacing, so the row shrinks towards the minimum or wraps. A fully labelled 7-point Likert scale has to stack vertically, turning one question into a short form.

In a 2011 experiment, Funke, Reips and Thomas found that slider scales led significantly more people to give up on the survey than radio buttons. Their sliders were built in Java, so take it as a reason to test a slider on phones first, not a verdict on every slider.

Use the same scale on every page you want to compare. A star average and a Yes share cannot be set against each other, so a change of scale starts a new baseline.

Asklet asks short surveys on the page from one snippet. A Yes / No first question, worded as you like, gives each page a "found it useful" figure. Star ratings (1 to 5) show the average and the spread, and there are NPS (0 to 10) and 0 to 100 sliders. A fully labelled Likert scale is a single-answer multiple choice question. Free text answers appear as anonymous comments next to their page.

There is no emoji or thumbs scale, and ratings are never shown on the page: readers see a thank-you message, and you read the results in your dashboard. Surveys are versioned, so when you switch from stars to Yes / No, the new answers stay apart from the old ones.