Identifying Solutions from User Testing
Aircraft Owners and Pilots Association (AOPA)
AOPA is a non-profit organization representing the largest aviation community in the world. With over 300,000 AOPA pilots across 75 countries, it’s not difficult to hear when something about their website isn’t working.
AOPA had a long list of user complaints related to website search-ability. AOPA selected Ascedia to identify and implement a new search platform for the AOPA.org website. This included evaluating the existing search solution (Google) and creating a proof of concept for how a new search solution might best integrate into Sitecore.
Solution Candidate: Coveo
Coveo Site Search emerged as a potential solution, offering best-in-class capabilities including advanced machine learning technology and native integration with Sitecore. AOPA was interested but needed validation before investing in a replacement for Google Search.
Ascedia performed two rounds of user testing to compare AOPA’s current search tool with Coveo.
User test groups were created consisting of AOPA pilots who were familiar with the current website and test participants from an online testing pool.
See the Problem
The current search tool was tested to understand how users searched for specific items and how well those results addressed their queries.
Participants were asked to perform different tasks like "how would you start to search for..." as well as very specific searches like "find the most recent article in Pilot Magazine."
Everyone had challenges
Results showed participants experienced challenges when searching for queries, often finding answers to searches in unintuitive indirect ways.
In a second round of tests, Ascedia implemented Coveo Site Search in a copy of AOPA’s Sitecore production environment.
The second round test group consisted of AOPA pilots who were familiar with the current site.
Results showed that users moved through guided tasks quickly and with greater ease within Coveo's search results.
Participants did not use the filtering features, potentially due to Coveo serving up better results for common searches based on machine learning.
Taking all the feedback and learnings from both test rounds, Ascedia created UX and designs aligned with users’ natural search processes.
After successfully implementing Coveo within in Sitecore, we provided AOPA marketing administrators with training to setup back-end features to optimize search results over time.
AOPA users have enjoyed the search experience, and the AOPA marketing team now has a manageable workflow. Coveo is a long-term investment featuring advanced artificial intelligence, so the search solution will continue to learn, adapt and benefit users over time.