MIXED METHODS
COMPANY
ROLE
TIMELINE
YEAR
Overview
Evolution of analytics software from on-prem to cloud amid AI market disruption.
Background
Alteryx, a data analytics company, was transitioning from legacy on-premise software programs to a unified cloud-based platform.
Users & Context
Mix of data analysts, data engineers and non-analysts. Legacy product had loyal following, especially among non-analysts, for its visual approach.
Alteryx lacked a unified point of view on product health, leading to duplicated work and random prioritization. This was most acute in the data ingestion experience, a requisite task that lacked standardized performance targets.
Research Questions
What challenges, if any, do users encounter when initiating data ingestion?
What are users' perceptions of the desirability and usability of data ingestion?
How do these compare across usability standards and across Alteryx's product portfolio?
Goal
Establish a standardized, repeatable framework to measure and communicate the UX health of Alteryx products, specifically by evaluating the critical task of data ingestion.
Intercept Surveys
Collected feedback from 40 active Alteryx Cloud users via on-platform intercepts. This allowed us to measure real-time perceptions of usability and product-market fit from users naturally engaged with the platform.
Unmoderated Usability Testing
Ran sessions with 20 participants, including a mix of current customers and external data professionals recruited through a third party. They were tasked with completing high-priority onboarding workflows to identify specific functional friction.
While users reported a high product market fit, usability testing revealed critical gaps in the essential task of bringing data into the platform.
Critical Usability Gap
Site architecture and content strategy issues prevented 65% of participants from completing the data ingestion task. Importantly, many did not even realize they had not been successful.
48% System Usability Score (SUS), identifying a disconnect where users valued the tool's power but struggled with task navigation.
Validated PMF
Achieved a 59% Product Market Fit score, significantly exceeding the 40% industry benchmark and confirming high demand for cloud data capabilities.
Roadmap Prioritization
Shifted the Q1/Q2 roadmap toward addressing foundational UX debt and onboarding friction. By providing the evidence required for an architectural transformation—specifically merging the "Data" and "Connections" pages—this work removed a primary structural barrier for users initiating workflows.
Organizational Literacy
Established a shared vocabulary across 10+ product teams for what "Good" looks like.