Skip to main content

Identifying Solutions from User Testing

Aircraft Owners and Pilots Association (AOPA)

AOPA is a non-profit organization representing the largest aviation community in the world. With over 300,000 AOPA pilots across 75 countries, it’s not difficult to hear when something about their website isn’t working.

AOPA-Special-Use-Logo_Wings-Only

The Challenge

AOPA had a long list of user complaints related to  website search-ability. AOPA selected  Ascedia to identify and implement a new search platform for the AOPA.org website. This included evaluating the existing search solution (Google) and creating a proof of concept for how a new search solution might best integrate into Sitecore.

Solution

Ascedia started with user experience testing to find the right website search provider. This solutions demonstrates a collaborative and iterative approach to right-size our client solutions.

Solution Candidate: Coveo

Coveo Site Search emerged as a potential solution, offering best-in-class capabilities including advanced machine learning technology and native integration with Sitecore. AOPA was interested but needed validation before investing in a replacement for Google Search.

User Testing

Ascedia performed two rounds of user testing to compare AOPA’s current search tool with Coveo.

Focus Groups

User test groups were created consisting of AOPA pilots who were familiar with the current website and test participants from  an online testing pool.

See the Problem

The current search tool was tested to understand how users searched for specific items and how well those results addressed their queries.

Guided Questions

Participants were asked to perform different tasks like "how would you start to search for..."  as well as very specific searches like "find the most recent article in Pilot Magazine."

Everyone had challenges

Results showed participants experienced challenges when searching for queries, often finding answers to searches in unintuitive indirect ways.

User test groups were created consisting of AOPA pilots who were familiar with the current website and test participants from  an online testing pool.

The current search tool was tested to understand how users searched for specific items and how well those results addressed their queries.

Participants were asked to perform different tasks like "how would you start to search for..."  as well as very specific searches like "find the most recent article in Pilot Magazine."

Results showed participants experienced challenges when searching for queries, often finding answers to searches in unintuitive indirect ways.

Additional Insights

In a second round of tests,  Ascedia  implemented  Coveo  Site Search in a copy of AOPA’s Sitecore production environment.

Real Audience

The second round test group consisted of AOPA pilots who were familiar with the current site. 

Positive Outlook

Results showed that users moved through guided tasks quickly and with greater ease within Coveo's search results.

Fine Tuning

Participants did not use the filtering features, potentially due to  Coveo  serving up better results for common searches based on machine learning.

User Experiences

Taking all the feedback and learnings from both test rounds,  Ascedia  created UX and designs aligned with users’ natural search processes.

The second round test group consisted of AOPA pilots who were familiar with the current site. 

Results showed that users moved through guided tasks quickly and with greater ease within Coveo's search results.

Participants did not use the filtering features, potentially due to  Coveo  serving up better results for common searches based on machine learning.

Taking all the feedback and learnings from both test rounds,  Ascedia  created UX and designs aligned with users’ natural search processes.

Marketing Support

After successfully implementing Coveo within in Sitecore, we provided AOPA marketing administrators with training to setup  back-end features to optimize search results over time.

Results

AOPA users have enjoyed the search experience, and the AOPA marketing team now has a manageable workflow. Coveo is a long-term investment featuring advanced artificial intelligence, so the search solution will continue to learn, adapt and benefit users over time.

Want to Know More?