Management Concepts/2022
Learning Dashboards
Led the end-to-end design process of a set of dashboard for student and instructional designers separately that aims soft skill improvement. The final design is adopted by the client to be part of their product.
Intent
Master's Program Capstone Project
Timeline
2022.1-2022.8
Team size
1 UX Designer (ME)
1 UX Researcher
1 Engineer
1 Product Manager
1 UX Researcher
1 Engineer
1 Product Manager
Reponsibility
• Product Design
• Team branding
• Poster Design
• Team branding
• Poster Design
Project Overview
The context
Management Concepts is building a course to targeting learners' soft skills at work along with the basic knowledge
Management Concepts is a company that provides professional training for certifications. However, they found their learners struggle in their job even with the certification. Therefore, Management Concepts is preparing a new activity based course that targets professional - soft skills that can help them.
Problem
Learners and instructional designers lack awareness and effective strategy for soft skills building
However, the learners are still in the old mindset of rigidly memorizing by the book and instructional designers don't have any support in the workflow to see if the courses will actually help the learners improv their soft skills.
Solution
A set of soft skill based dashboards that measure learners' soft skill performance and course effectiveness
With the Learning Dashboards, learners and instructional designers can both check the soft skill mastery level from performance visualization. For learners, skill performance trend helps them identify the weaknesses in each skill. For instructional designers, course effectiveness analysis helps them find the most efficient activity for each skill.
For Learners
Filter out the under-preformed activities
For Instructional Designers
Review activity's impact on soft skill development
For Instructional Designers and Learners
Check soft skill mastery level and compare with peers
Impact
Our solution received wide appreciation within the company
✅
Approved!
Approved!
Our solution got adopted to official product by the CTO
⬆️
26%
26%
The SUS score improved from 73 to 92
⬇️
30 mins
30 mins
Reported time reduced for course revision
Research
client request
How might we help learners improve the soft skills while supporting instructional designers design better courses?
Research strategy
Find the gap between ideal status and status quo
Since we are totally new to this training area, we started from learning what the contents are & the best way of learning from experts and see what the learners lack from there.
12
Subject matter expert interview
6
Instructional designre interview
8
Learner prototype testing
Learners learning gap
Lack awareness and strategy for soft skill development
Comparing the suggested learning focus from SMEs and the frustrations during work, we noticed a inconsistency between the flexibility required in work and the limitation of cases in class.
Insight 1: Learners memorize the cases & solutions without understanding the logic behind
Insight 2: Learners treat their work as an exam rather than problem solving process
Instructional Designers' painpoint
Lack direct feedback loop to evaluate & iterate the course
Instructional designer have been "design in the dark", they can only rely on their own experiences to fulfill the requirement from the management team, without explanation or feedback.
Insight 1: Instructional designers have no access to learner's performance
Insight 2: Instructional designers don't have any data supporting their design process
Design goal
A website that raises learners' awareness on soft skills and help instructional designers validate their course
Design Process
Design Challenge1
How might we raise learners' awareness on professional skills?
Hypothesis
Personalized information on soft skill would attract users' attention to it
Based on social science, people are naturally attracted to things that are relative to themselves. So ideally highlighting information about soft skill can attract and evoke learners to reflect on their soft skills.
Ideation
Highlight soft skill performance to get attention and arise curiosity
Getting inspiration from the theory, I prototyped for three granularity of soft skill performance to get the attention of the learners.
CONCEPT VALIDATION
Score trend motivated most learners to reflect on their soft skills
I then conducted A/B test with 10 learners to see which option motivates most learners to check the weakness of their professional skills. It turned out that the score trends provides the proper amount of information to attract learners' attention on skill improvement.
Iteration
Finetune data visualization to reduce confusion
However, there was a frequently asked question during the test:
"Why is my score so fluctuated?"
I realized that the actives have different level, to make it easier to compare the improvement, I changed the graph to bar chart with activity type filter.
"Why is my score so fluctuated?"
I realized that the actives have different level, to make it easier to compare the improvement, I changed the graph to bar chart with activity type filter.
Final solution
An activity score dashboard to track & analyze underperformed skills
Design Challenge 2
How might we help instructional designers validate their course?
Ideation
Compare learner performance before and after activities
I organized a brainstorming session with the team to ideate ways of comparison, and landed on this interactive chart that allows instructional designers to filter and check learners' improvement of each skill.
Prototype testing
However, instructional designers find it hard to filter and understand the chart
After testing with 4 instructional designers, I learned that they are not familiar with chart reading, therefore having a hard time extracting helpful insight.
Iteration
Translate chart into direct score comparison
Since instructional designers are better text readers than graph reader, I switched to a table view with data that needs to be compared grouped together. With the pre-set comparison & insights, the. cognitive overload can be reduced.
Final solution
A table view for activity type impact on each skill
Result
Evaluation
A usable and useful solution that approved by client CTO
Besides thee SUS score improvement we validated through user testing, we also gained appreciation from various stakeholders of our client, from marketing to instructional designers to technicians. And our solution also got approved by the company CTO to be part of their product.
76 ➡️ 92
SUS score improvement
Key takeaways
Analyze real data before doing visualization
At the beginning, I chose line chart for showing score trend just based on the basic guidelines. However, the real data is actually confusing when shown with line chart. To avoid the same problem, always check the content before putting a placeholder during prototyping.
Common patterns are not always right
Even if enterprise products have many existing layout and patterns, they may not be effective for all kinds of users. We should understand the prior experiences of the target users before landing on any decisions.