Optimization dashboard
🎯 Design thinking facilitation
🖥️ Product design & leadership
📉 Data visualization
🧠 AI explainability
Design team
Workshop facilitation → Myself
Product design → Myself and McKenzie Gutshall
Content design → Emily Waldrop
Design research → Diana Sapanaro
Awards
🏆 Outstanding Technical Achievement Award for Trust in AI for Sterling Fulfillment Optimizer
🏆 Artificial Intelligence Excellence Awards, Nominee Finalist
✦ Challenge
IBM Sterling Fulfillment Optimizer uses AI to optimize order management systems by finding the best possible store, warehouse, or vendor for online order fulfillment. Although this is a great tool that can save retailers thousands, if not millions, customers have a hard time giving up sourcing controls to artificial intelligence.
“We’re not ready to jump on the machine learning bandwagon and put our kids on the school bus and hope for the best. Some day we’ll be irresponsible if we don’t put our kids on the automated school bus because it’s the safest thing, but we’re not going to be the first ones.”
“We struggle with connecting the dots between what optimizer thought was going to happen vs. what actually happened.”
✦ Action
1. Visioning workshop
As a product team, we needed to answer one question, "How do we build customer trust in Sterling Fulfillment Optimizer's AI?" Instead of immediately jumping into ideas, I proposed a design thinking workshop to our team of over 30 content designers, design researchers, product designers, product managers, data scientists, and both front and backend engineers. The design team utilized IBM’s Enterprise Design Thinking tools to plan the workshop and help find an answer to this daunting question for Fred, the fulfillment manager.
Fred
Omni-Channel Fulfillment Manager
Maximizes customer satisfaction and minimizes costs for his retail company.
1A. Identify Fred’s needs
Our goal was to determine what Fred needs to achieve his business goals and feel comfortable using AI for order fulfillment. Every workshop participant wrote multiple needs statements in the following format:
Users need a way to address this need so they can benefit in this way.
The design team then grouped the needs statements into 6 categories.
Users like Fred need to understand how their fulfillment network will be affected by their optimization settings. Each configuration could have massive implications on monetary savings, capacity, and store inventory. These decisions are not made lightly. The more knowledge Fred has about how his fulfillment network will be affected, the more confident he will be in using these AI tools.
Fred needs a way to have visibility into all fulfillment components to have confidence that the system is configured correctly and will lead to expected outcomes.
1B. Ideate solutions
Next, we needed to determine how we could alleviate Fred's concern about understanding the results of his fulfillment configurations. Workshop participants shared their ideas using the big ideas exercise. 66 new ideas were generated and grouped into nine categories. These ideas are IBM confidential.
IBM confidential1C. Prioritize our ideas
Although we had many innovative ideas, it was not feasible to design 66 potential solutions at once. The goal was to use a prioritization grid to determine which ideas to take to the next step. However, the prioritization grid felt arbitrary and did not use detailed information to prioritize items. I brought this concern and an initial idea to the design team. Our solution was for each discipline to document how long it would take to develop the idea and how much value each idea would provide Fred.
After collecting this data from front and back-end engineering, content design, product design, design research, and data science, we placed the big idea categories on a standard prioritization grid and reviewed the results.
1D. Workshop outcomes
There were zero no-brainers. Any solution we developed would need time, research, and dedication to get right. The best path forward was to focus on the ideas Fred would use frequently and take us a quarter at most to deliver.
The biggest gap in our product was the ability to monitor the fulfillment network, report on discrepancies in expectations, and take action. Creating an optimization dashboard was a clear solution.
Dashboard
A dashboard presents an opportunity to meet the monitoring and reporting needs of our customers.
Monitoring & reporting
Build trust by providing insights into why customers’ business goals have or have not been met.
The design team worked with product management to craft three hills to guide our design process.
The fulfillment manager can visualize Sterling Fulfillment Optimizer’s network performance, business outcomes, and value without needing help from IBM.
The fulfillment manager can monitor KPIs across multiple customized workspaces allowing him to be self-sufficient in finding and analyzing information.
The fulfillment manager can use optimization insights to take action according to their business strategy.
2. Internal analysis
Next, we needed to evaluate existing dashboard patterns in the IBM Carbon Design System and the rest of the IBM Order Management System suite to create a consistent experience for our customers.
IBM Sterling Order Management
IBM Sterling Order Management
IBM Sterling Fulfillment Optimizer
Before jumping into sketches, we set up a meeting with backend engineering to understand which data and KPIs were readily available to add to the dashboard.
3. Scenario map
Using the findings from the workshop, the design team mapped Fred’s current journey and ideated on a feasible near-future user experience.
4. UI design
Using our findings from the workshop and the KPIs engineering confirmed we could provide our customers, we quickly started ideating on how to visually display the data. Our customers expressed the need to see this data from a single-day and multi-date view. Their goal was to get an overall temperature check on the health of their fulfillment network during a period of time.
After reviewing the overview of a dashboard widget, as shown above, Fred wants to dig into the details. Visualizations are helpful to find anomalies, but Fred wants to see the numbers associated with each day. As a fulfillment manager, Fred spends his days reviewing Excel spreadsheets. It’s incredibly valuable for him to export this data, import it into Excel, and understand the data in his own way.
During this design process, the entire product team met at least once weekly to review the designs and give feedback. At the end of the sprint, the team signed off on the following user experience:
This is what’s in production as of April 2023. Please note that this is a demo with dummy data. Not all of the widgets have data to display.
Accessibility
To ensure accessibility, we utilized the IBM Accessibility Equal Access Toolkit. For example, we included tab order in our hand-off to development.
5. Evaluative research
Our goal was to understand if the optimization dashboard helped our users understand the health of their fulfillment network. We completed multiple 60-minute evaluative design reviews with our IBM Sponsor Users. After the sessions were complete, I worked with the design researcher and the other product designer to synthesize the feedback and play our findings back to the team.
5A. Quotes
“When is this going to be released?”
“Anything that gives us a view towards trends would be valuable and something we don’t have today.”
“We constantly feel like we’re in a state of reaction. It’s amazing to think we could be in a position of being more proactive.”
✦ Outcome
The Optimization dashboard allows users to view node and KPI data related to costs, orders, packages, and units to quickly identify anomalies. Fred is now able to be proactive rather than reactive about updating his optimization settings. We know from feedback sessions that our customers utilize the dashboard to check the health of their fulfillment network, review the 7-day view to monitor their KPIs, and often assess node backlog data. On-prem IBM Sterling Order Management customers upgraded to Order Management on Cloud due to the insights they could get out of the optimization services like this feature.
“Having the dashboard is a game changer.”
“Part of the reason why we upgraded to Order Management on Cloud was the value that we can get out of Sterling Intelligent Promising”
Awards
🏆 Outstanding Technical Achievement Award for Trust in AI for Sterling Fulfillment Optimizer
🏆 Artificial Intelligence Excellence Awards, Nominee Finalist