Open-ended vs closed-ended questions: which gets better feedback?

The difference shows up in how many readers answer, what you can do with the answers and what each one misses.

Nielsen Norman Group draws the same line: closed questions are easier to answer and can be analysed statistically, while a survey with many open-ended questions "will usually have a lower completion rate". Use the closed question for the figure, the open one for the reason.

Pew Research Center measured skips on its American Trends Panel, a large online survey panel in the US. On closed questions, the share of people who skip an item averages 1% to 2%. On open-ended questions, it averages about 18%, and in the 92 that Pew studied it ranged from 3% to just over 50%. Three things made a skip more likely:

"What were you looking for?" asks for a phrase. "Tell us what you think of this article" asks for an essay, and gets skipped. Ask for one reason or one missing thing, never general thoughts.

Closed questions have a cost that is harder to see. After the 2008 US presidential election, Pew asked half its respondents which one issue mattered most to their vote with five options, and the other half with no options. With the list, 58% chose the economy. In their own words, only 35% named it. And while 8% of the list group gave an answer outside it, 43% of the open-ended group gave one that was not on the list.

The list pushed people towards the options shown and hid answers nobody had thought to include. If "What was missing?" offers "More detail", "An example" and "Clearer steps", you will never learn that half your readers wanted a printable version.

Most of the bias in a multiple choice question comes from the options. Pew's questionnaire guidance and experiments give these rules:

The closed question collects a number from most readers. The open one collects reasons from the readers who have one. The free text then tells you what the list is missing.

When you change the options, compare results only within one version of the survey. A list of four and a list of five are not the same measurement.

Coding means giving each answer one or more short labels (codes) so you can count them. Pew codes open-ended answers by reviewing random samples, building a set of codes, testing them on new samples and refining them after each round. A small version works on your own feedback:

A spreadsheet is enough: one row per answer, one column for the page and one for the codes. If an AI assistant suggests codes for a first pass, check its labels against the answers before you count them.

Asklet has both question types. Multiple choice takes 2 to 12 options you write, with an optional several-answer mode. Free text takes up to 1,000 characters, removes email addresses and phone numbers before storage, and shows each answer as an anonymous comment next to its page. Readers see one question at a time, and you can let them skip any question after the first.

Surveys are versioned, so adding an option readers kept writing in means publishing a new version, and earlier answers stay matched to the questions that were asked.