Identifying Solutions from User Testing
Aircraft Owners and Pilots Association (AOPA)
AOPA is a non-profit organization representing the largest aviation community in the world. With over 300,000 AOPA pilots across 75 countries, it’s not difficult to hear when something about their website isn’t working.
The Challenge
AOPA had a long list of user complaints related to website search-ability. AOPA selected Ascedia to identify and implement a new search platform for the AOPA.org website. This included evaluating the existing search solution (Google) and creating a proof of concept for how a new search solution might best integrate into Sitecore.
Solution
Ascedia started with user experience testing to find the right website search provider. This solutions demonstrates a collaborative and iterative approach to right-size our client solutions.
Solution Candidate: Coveo
Coveo Site Search emerged as a potential solution, offering best-in-class capabilities including advanced machine learning technology and native integration with Sitecore. AOPA was interested but needed validation before investing in a replacement for Google Search.
User Testing
Ascedia performed two rounds of user testing to compare AOPA’s current search tool with Coveo.
Focus Groups
User test groups were created consisting of AOPA pilots who were familiar with the current website and test participants from an online testing pool.
See the Problem
The current search tool was tested to understand how users searched for specific items and how well those results addressed their queries.
Guided Questions
Participants were asked to perform different tasks like "how would you start to search for..." as well as very specific searches like "find the most recent article in Pilot Magazine."
Everyone had challenges
Results showed participants experienced challenges when searching for queries, often finding answers to searches in unintuitive indirect ways.
Focus Groups
User test groups were created consisting of AOPA pilots who were familiar with the current website and test participants from an online testing pool.
See the Problem
The current search tool was tested to understand how users searched for specific items and how well those results addressed their queries.
Guided Questions
Participants were asked to perform different tasks like "how would you start to search for..." as well as very specific searches like "find the most recent article in Pilot Magazine."
Everyone had challenges
Results showed participants experienced challenges when searching for queries, often finding answers to searches in unintuitive indirect ways.
Additional Insights
In a second round of tests, Ascedia implemented Coveo Site Search in a copy of AOPA’s Sitecore production environment.
Real Audience
The second round test group consisted of AOPA pilots who were familiar with the current site.
Positive Outlook
Results showed that users moved through guided tasks quickly and with greater ease within Coveo's search results.
Fine Tuning
Participants did not use the filtering features, potentially due to Coveo serving up better results for common searches based on machine learning.
User Experiences
Taking all the feedback and learnings from both test rounds, Ascedia created UX and designs aligned with users’ natural search processes.
Real Audience
The second round test group consisted of AOPA pilots who were familiar with the current site.
Positive Outlook
Results showed that users moved through guided tasks quickly and with greater ease within Coveo's search results.
Fine Tuning
Participants did not use the filtering features, potentially due to Coveo serving up better results for common searches based on machine learning.
User Experiences
Taking all the feedback and learnings from both test rounds, Ascedia created UX and designs aligned with users’ natural search processes.
Marketing Support
After successfully implementing Coveo within in Sitecore, we provided AOPA marketing administrators with training to setup back-end features to optimize search results over time.
Results
AOPA users have enjoyed the search experience, and the AOPA marketing team now has a manageable workflow. Coveo is a long-term investment featuring advanced artificial intelligence, so the search solution will continue to learn, adapt and benefit users over time.