There is a fair objection to any feedback widget: the readers who answer it may not be like the readers who ignore it. People with a strong opinion, or a problem, are more likely to stop and tap. If you rewrite pages for them, you may make those pages worse for everyone else.
A second objection follows from the first. Most pages on most sites get only a few answers a month. When does a handful of answers justify changing anything?
This post answers both: what you can do with any survey tool, and what Asklet does for you.
The short answer
No widget can make the readers who answer stand in for all your readers, so plan around the gap. Make the first question one tap so quiet readers answer too, ask it the same way on every page, and judge a page by how it ranks against similar pages. When a page has too few answers, Asklet shows counts and no percentages until it reaches 10. Below that, widen the date range, add the page's answers to a theme of similar pages, or act only on problems you can check yourself.
Why the readers who answer differ from the rest
Survey researchers call this self-selection, or non-response bias. The readers who choose to answer are not a random sample of the readers who saw the question. On a web page, the ones most likely to answer are:
- Readers with a strong reaction, good or bad. A guide that saved someone an afternoon and a guide that cost them one both get answers.
- Readers who reached the end. A question after the last paragraph never reaches the readers who left halfway.
- Readers with a moment to spare. Someone reading on a phone between meetings scrolls past.
No placement or wording removes this. You can make the gap smaller, and you can set up your comparisons so the gap matters less.
Make answering cheap enough for quiet readers
The more effort an answer takes, the more the answers come from people who care a lot. A one-tap question also gets answers from readers with mild opinions, and there are usually more of them.
-
Start with a Yes / No question
One tap, no typing. Readers who would never write a comment will still tap Yes.
-
Ask for the reason with a few options
A multiple choice follow-up such as "What would make this better?" takes one more tap. Every reader sees it, so the reasons come from Yes and No readers alike.
-
Put the comment box last and keep it optional
Free text draws the most motivated readers. Use it for detail once the quick answers are in.
-
Use the same question in the same place on every page
If one page asks at the top and another at the bottom, they reach different readers and you can't compare them.
Tip: Avoid wording that invites complaints, such as "Anything wrong with this page?". Ask a neutral question that a satisfied reader can answer as easily as an unhappy one.
Compare pages against each other
Say your guides get 70% Yes. Because of self-selection, the figure for all your readers could be higher or lower, and you have no way to know which. The absolute number is the weakest part of the result.
Rankings hold up better. When every guide asks the same question in the same place, similar kinds of readers answer on each one. A guide at 40% Yes among guides at 70% stands out, even if every figure is a few points off.
To keep the comparison fair:
- Compare like with like: guides with guides, news with news. Each section has its own kind of reader.
- Use the same date range for every page you compare.
- Don't move the widget or change the first question during a comparison. A new placement reaches different readers, so start a new comparison from the day you change it.
Work out a rough response rate too. Asklet doesn't count widget loads, so divide each page's answers by its page views from your analytics for the same dates. If one page's rate is far above its neighbours, something is sending readers there to answer, such as a link from a forum thread or a step that has stopped working.
Keep one reader to one voice
Sometimes the vocal minority is one loud reader. Asklet limits how often one person can answer:
- Every set of answers needs a single-use token from a real widget load.
- A 3-hour "already answered" note in the reader's browser stops the widget asking them again on the same page.
- Each network can start only a limited number of surveys an hour.
None of this identifies the reader, and it doesn't make repeat answers impossible: someone who clears their browser storage can answer again, within the hourly limit.
The rest is up to how you read the results. A reader who writes 300 words is still one reader, so count how many readers picked each option in the multiple choice question first, then read the comments on the reasons that come up most.
How many answers a page needs
Asklet shows only counts on a page, theme or website until it has 10 responses in the selected period. Then it switches to percentages, averages and NPS scores. That stops "2 of 3" from showing up as 67%.
Ten is where a percentage starts to mean something, but it is not enough to decide much on. The table shows how far a Yes share of 50% could be from the true figure, at 95% confidence, if the readers who answered were a random sample:
| Answers | Margin of error | A 50% Yes share could really be |
|---|---|---|
| 10 | ±31 points | 19% to 81% |
| 30 | ±18 points | 32% to 68% |
| 100 | ±10 points | 40% to 60% |
| 400 | ±5 points | 45% to 55% |
Readers who answer are not a random sample, so treat these margins as the best case. The real uncertainty is wider. The sample size calculator works out the numbers for your own pages.
What to do when a page has too few answers
Many pages take months to reach 10 answers, and some never will. That is normal on a site with a long tail of articles. You still have options:
-
Widen the date range
Results show the last 28 days by default. Pick 3 months, or set From and To to look at up to 52 weeks. Check that the page hasn't changed in that time, or you'll mix answers about two versions of it.
-
Add the page to a theme
Put similar pages in a theme, such as all your city guides. The theme's results add up the answers of all its pages and reach 10 much sooner. Decide for the whole group, then look at single pages inside it.
-
Read the reasons and comments anyway
Counts and comments show from the first answer. Three readers who each pick "Steps were unclear" on the same page are telling you where to look, even before there is a percentage.
-
Act on facts you can check yourself
A comment that says a link is broken or a price is out of date needs no statistics. Check it, and fix it if it's true.
-
Leave it alone
If nothing points to a problem and the page gets little traffic, you don't have to decide anything about it yet.
Match the decision to the evidence
Decide how much evidence each kind of change needs before you open the results, so a striking comment doesn't set the bar for you.
| Decision | Example | Enough evidence |
|---|---|---|
| Fix a fact | A broken link or a wrong price | One comment, once you have checked it yourself |
| Clarify a page | Readers say step 4 is unclear | The same reason from several readers, ideally on more than one page |
| Rewrite or remove a page | A guide readers say is out of date | 10 or more answers, a Yes share well below similar pages, and your analytics pointing the same way |
| Change a whole section | A new template for every recipe | Theme results with plenty of answers, and the same pattern on many pages in the theme |
Tip: Small changes you can undo need little evidence. The harder a change is to undo, the more answers it should rest on.
Example
Example: a travel site and one angry comment
Say you run a travel site with 120 city guides. Each one ends with "Did this guide help you plan your trip?", then "What would make it better?" with the options "More up-to-date prices", "Better maps", "More places to eat" and "Something else", then an optional comment.
Your Lisbon guide has 4 answers this month, 3 of them No. One comes with a long comment: the guide is useless, everything in it is wrong, and it should be rewritten from scratch. The page shows the counts and no percentage, so there is nothing to compare yet.
A rewrite would take two days. First you open your "City guides" theme, which adds up all 120 guides: 180 answers over 8 weeks, with a Yes share close to your other themes. Across the theme, the reason most No readers pick is "More up-to-date prices", on guides for many different cities.
So you make two decisions. You reread the Lisbon comment for anything specific, find that the metro fare it mentions is out of date, and fix it in five minutes. And you plan a price update for every city guide, because that is where the whole theme points. The rewrite waits until the Lisbon guide has enough answers to show whether it is behind the others.
Where Asklet fits
Asklet shows the same short survey in the same place on every page from one snippet, and keeps the path each answer came from. Results come per page, per theme and per website for the dates you choose, with counts only until there are 10 responses.
It doesn't count how many readers saw the widget, and it stores nothing that identifies a reader. So it can't tell you the response rate or weight answers by who gave them: take page views from your analytics for the first, and compare similar pages for the second. It won't help if you need to follow up with the people who answered.
Your next steps
-
Make your first question a one-tap Yes / No
Word it so a satisfied reader can answer as easily as an unhappy one.
-
Ask it the same way on every page in a section
Same wording, same place, so the pages can be compared.
-
Write down the evidence each decision needs
Before you open the results: one checked comment to fix a fact, many answers to rewrite a page.
-
Group similar pages into themes
Small pages add up, so you can decide for the group.
-
Note your page views for the same dates
Divide answers by views to spot pages where an unusual share of readers answer.
-
Check each change
Compare the same number of weeks before and after, and against similar pages over the same dates.
Try these steps on your own site. Asklet adds the questions to every page with one snippet. Free plan, no card needed. Compare the plans.
Start freeCommon questions
Does Asklet show a response rate?
No. It doesn't count widget loads, so it can't tell what share of readers answered. Divide a page's answers by its page views from your analytics for the same dates.
Can one reader answer many times?
Asklet makes it hard. Every set of answers needs a single-use token from a real widget load, readers aren't asked again on a page they answered in the last 3 hours, and each network can start only a limited number of surveys an hour. Someone determined could still answer again later, which is one more reason to compare pages instead of trusting one page's score.
Why do results wait for 10 responses?
Below 10, one answer moves a percentage by more than 10 points, and "2 of 3" would show as 67%. Ten is where a percentage is worth showing, and it is still rough: at 10 answers, a 50% Yes share could be anywhere from about 19% to 81%.
Can I weight the answers to match all my readers?
Not with Asklet. Weighting needs to know something about each reader, such as their age or how they found you, and Asklet stores nothing about the reader. Compare pages that ask the same question the same way instead: the bias tends to be similar on each, so the ranking is still useful.
Related
More from the blog
- Privacy and law EU cookie consent changes: where the Digital Omnibus stands in October 2026 9 min read
- Privacy and law Session replay and pixel lawsuits: what website owners should know 9 min read
- Privacy and law How to make a feedback widget accessible, and what the European Accessibility Act asks 9 min read