Last updated January 2020
Aim: to gain insights on the impact of general form structure on respondent behaviour/responsiveness, and to share those insights with our Paperform audience (primarily for use in our Form Design eBook). We have found there to be a gap in available, quality research into the impact of pagination and number of questions used in forms and their subsequent response rates.
The general sample data for this study consists of Qualified Forms. Forms were considered to be “Qualified Forms” if they met the requirements listed below. Requirements were then translated into a query, and communicated to Tech, who then retrieved and exported raw data programmatically from existing user data.
Personal user information was not extracted, and any unique identifiers for forms were encrypted. Data is therefore not traceable by form ID, accounts, or any other unique identifier. This means the same sample data set cannot be extracted from our servers again, and is therefore not reproducible.
This general sample data was then further explored for findings relevant to the Design eBook. Claims made in the eBook that were difficult to verify with existing research were noted, and queries were formed to test those claims. Specific parameters were then formulated and applied to the data in Google Sheets to extract relevant findings from the general sample data. Those findings were then considered alongside the qualitative experience of conversations with over 8000 users, before final conclusions were drawn.
Query: How many forms in the last 30 days (20 Dec 19 - 20 Jan 19) have submissions > 20? = 4172
Claim #1: the more questions on a form, the lower the completion rate (cognitive friction theory)
Query: Is there a negative correlation between the quantity of question fields used on forms and completion rates?
Generally, there is a negative correlation between the number of question fields and completion rates. Overall, average completion rates see a progressive downtrend as the total number of form questions increase. However, the results also reveal fluctuations in completion rates that are worth noting:
Generally speaking, the higher the number of questions on a form, the lower the completion rate. However, taken in isolation, this reductionist perspective fails to highlight nuances in respondent behaviour that provide helpful insights into better form creation.
The results reveal three key sub-trends which, together with our experience working with users, allow us to presume the following:
While form completion rates generally see a decline due to higher quantities of questions, the number of questions on a form alone does not determine completion rates. Rather, it is one key factor that should be considered when creating a form. Purpose and incentive, as well as use-case also must be taken into consideration. It is possible to create longer forms that return a high completion rate, although it does present a greater challenge.
Future research: To examine the relationship between completion rates and the average number of questions used on a form within predominant use-cases (and by extension industries, where possible).
Query: what kind of impact does the quantity of pages on a form have on completion rates?
There is an overall downward trend which shows a negative correlation between page numbers and completion rates. The following results should also be noted:
Single page forms performed significantly better than forms with multiple pages, returning a completion rate approximately 15% higher than the best performing multi-paged form group (2 pages). We considered that these results could be indicative of the impact of question numbers (forms with more pages are potentially likely to have more questions, which create cognitive friction, and therefore negatively impact completion rates), however we would then also expect to see a steady and progressive downward trend as page numbers increased, which is not the case.
A breakdown of these groups by number of questions is revealing; for example, while forms with 2 pages and 7-11 pages returned similar completion rates (3% difference), the sample set of each revealed divergent groups when segmented by number of questions:
The fluctuation in completion rates for paginated forms across the groups indicates that, where a form has multiple pages, the number of pages has a low impact on completion rates (single page forms clearly outperform multi-page). As previously discussed with questions quantities, incentive to complete and purpose for engaging likely have a greater impact on completion rates than pagination alone.
By extension, use-case will likely have an impact on completion rates, as respondents will have set expectations which influence their management of cognitive friction. For example, a simple lead generation form should expect low completion rates if it contains many pages, however a law firm client intake form with more than four pages should expect higher completion rates.
This is a macro study which explores trends across large samples of data across a broad spectrum of use-cases and industries. It would benefit from being considered alongside use-case examples and A/B testing where available.