Our Data Scientists expressed that Machine Learning benefits from training the system with more of the right data.
The Product Management team wants a tool to show customers the gaps where more data is needed, so we can add sensors for a more complete picture.
By getting to the root cause I was able to discover that gaps in our customer on-boarding process was eroding trust.
The Business Problem
The relationship with a high value customer of Machine Learning algorithms was at risk. Their critical asset suffered from downtime because an "event" they were expected to predict went unrecognized.
Our Customer Reliability Engineering team scrambled to find the root cause of the missed alert using volumes of spreadsheets and manual cross-checking.
They determine that the algorithm was in effect, however it was not being streamed enough data to provide full coverage. A lack of communication between the team configuring the analytic, and the customer lead to overconfidence in the algorithm.
Research and Analysis
As we studied this incident and spoke with the parties involved, it became clear that a tool for better understanding and communicating the full potential of the algorithm, the data fed to the algorithm, and the gap in between would be of great value to the customer and the reliability engineers... even as the team was already in QA and about to bring the system online!
As I set off to design the tool based on the spreadsheets and how the problem was articulated, it became clear that our customer on-boarding process was too opaque. Customers were losing trust in the service.
I approached my manager and requested 6-8 hours/wk to better define the customer onboarding process.
We assembled a cross-functional team from around the world and made our first attempt at mapping out the process from end-to-end. This included UX, Product Management, Service Operations, System Operations, Customer Training, Commissioning, Professional Services, and others.
It became clear that we were relying on, and making a lot of assumptions about the early stages of the on-boarding process. If those early decisions are made in isolation it can have a negative impact on the customers’ time to value.
The customer service team needed more visibility in to the progress of the entire onboarding process, but lacked the tools to do so.
Customer Service teams were getting inquires about progress, but lacked visibility in to new customer engagements as systems were installed and configured.
I proposed a customer dashboard, leveraging the product's own dashboard, that would provide them with a clear understanding of:
Interactions with this dashboard would introduce them to the core product, and act as a window in to the configuration process, so that they (and all stakeholders involved in the on-boarding process) felt more empowered and aware of what was going on.
I'm proud of this project as it exposed a significant issue which, if ignored, would have a significant impact on our ability to scale.
With the right people assembled and motivated to address the issue, I was able to return to my full-time project knowing that future customer relationships would benefit from our efforts.