Conversational AI Design
VR Spatial Design
Multimodal/Cross-platform experiences
Redefining Workforce Learning:
AI-Driven VR Training for Neurodivergent Talent
In association with New York's Leading organizations
Hats Worn
Accessibility UX Research • End-to-end Product Design • Usability Testing • Spatial Interaction Design
Published Accomplishments
ACM CHI 2023
ACM CHI 2024
ACM CHI 2025
Collaborators
Pinaki Babar, Ziming Lee, Roshan Peiris
Clients: Mike Barry, Director, Heritage Christian Services
Kathy Kanada, Director, The Arc Ontario
Platform
Web Responsive Interface, VR Interface
Timeline
20 months | Aug 2022 - May 2024
Context
Heritage Christian Services, The Arc Ontario, and Mozaic are at the forefront of empowering thousands of people with intellectual disabilities (PIDs) across New York State through their rehabilitation programs with the help of dedicated job coaches and caregivers.
The Hurdle
58% of PIDs struggle with workplace communication, yet the job coaches training them can only work with a handful of people at a time. These one-on-one sessions are deeply effective—patient, personalized, empathetic—but they can't scale. The question wasn't whether we could build better training technology. It was whether we could scale human empathy itself.
We built VRSpeechChatbot—
an AI-powered platform where job coaches design workplace scenarios on the web, and trainees experience them as realistic VR conversations. One coach's expertise could now become thousands of patient, adaptive training sessions.
Our breakthrough: making empathy scalable!
My contributions
As lead designer, I recognized this wasn't just a UX challenge—it was a systems design problem requiring deep empathy and technical innovation. My philosophy was simple: spend real time with users first, then build technology around their reality.
I had one north star: create something that actually works for the people who need it most. This meant asking tough questions at every turn: Can a job coach with no tech background use this easily? Will trainees feel supported, not overwhelmed? Does this genuinely solve their communication challenges?
Throughout the project, I wore multiple hats—researcher, designer, systems thinker, stakeholder translator—but always returned to the same goal: make it accessible, approachable, and effective. Not just cool technology, but technology that improves lives.
Let us peel back the layers of VRSpeechchatbot
The challenge wasn't just designing for VR—it was creating a cohesive experience across web and virtual environments while keeping diverse stakeholders aligned on something that had never been built before.
Secondary research – Uncovering the landscape
Academic papers, industry reports, and existing tools gave us more clarity about different training methodologies, rehabilitation programs, technologies used, and allied disabilities and most importantly, helped us understand the gravity of the problem.
SWOT Analysis of Job Simulator by the Oculus - VR Game for Job Skills Training
28%
working-age PIDs
have never held a job
19%
PIDs are in the labor force vs.
62% of adults without disabilities
58%
PIDs experience
communication difficulties
Key takeaway
Academic studies highlighted VR’s effectiveness for immersive, repeatable skill-building in neurodiverse populations.
Primary research – Understanding who are we solving for
We gained unique perspective on the overall training process, coaches' goals, challenges and opportunities through workshops, interviews and field studies.
Currently, how do they train?
By conducting role-playing activities!
Issues – heavy dependency on the availability of people/resources, too chaotic to handle multiple trainees simultaneously, taxing for job coaches to come up with training scenarios multiple times.
What are job coaches looking for?
"…easy-to-use and has step-by-step approach…avoid complex multi-step processes."
"…focuses on real-world skills based learning…It'll prepare them for what they would really encounter on the job."
"…adapted to each person’s unique abilities, interests, goals and support needs. "
Design direction - Defining our scope
01
Reduce the human coaching load without sacrificing empathy or iteration
02
Make training accessible, adaptive, and relevant—just like a great human coach
03
Create virtual environments that feel real and foster everyday job related skills
So, how might we build simple scenarios that feel real?
Design Decision 1 : Building scenarios and understanding learning mechanisms
My first approach was testing various training methodologies like instruction-based learning, hands-on practicals, gamified evaluation etc.
I worked closely with users, testing interaction patterns and observing how they engaged with the platform.
The platform was slow, repetitive and lacked real conversational experience and there was too much dependence on probe interactions. I was solving for immersion, but missing the human element that makes real workplace conversations unpredictable and nuanced.
Design Decision 2: Integrating Conversational AI
The breakthrough came when we integrated an LLM-based conversational agent into the VR interface. Suddenly, trainees could have dynamic, natural conversations that felt different every time.
I designed two core scenarios—information retrieval and emotion recognition—that targeted fundamental communication skills.
However, we realized soon that the training was dependent on us producing more scenarios.
It was time for democratizing the platform!
Design Decision 3: The Strategic Pivot: Empowering the Experts
Although the conversations in these scenarios felt new and refreshing in every interaction, yet they still lacked variety. The trainees needed to be trained for different situations that could not be covered in these two.
Here's where my design thinking shifted completely. Instead of asking "How do I create better scenarios?" I asked "Who knows best what scenarios trainees need?" The answer: job coaches. They understood each trainee's specific challenges, goals, and workplace contexts.
What if job coaches could create their own scenarios?
This insight transformed everything!
Redefining our scope
from trainee-focused platform to a holistic training tool that empowers job coaches
Primary Archetype - Job Coach
The job coach is a dedicated and empathetic professional committed to empowering individuals with intellectual disabilities (PIDs) to achieve their employment goals.
Train communication and interpersonal skills
Facilitate successful job placement and retention
Secondary Archtetype - Trainee
The trainee is an individual with an intellectual disability who is motivated to learn and grow. They may face challenges in communication, social interaction, and adapting to new environments. However, they are eager to develop their skills and achieve independence in the workplace.
Gain confidence and overcome anxiety
Solving the Complex Design Challenges
Making AI Feel Human
Job coaches needed to create scenarios, but trainees needed those scenarios to feel genuinely supportive. I designed non-linear dialogue flows where the AI could respond with patience and empathy—if a trainee said "Can you repeat that?" the system would respond like a caring coach, not a broken record.
Accessibility Without Compromise
I wasn't just designing for intellectual disabilities—I was designing for varying tech comfort levels, motor abilities, and cognitive load capacities. High-contrast visuals, closed captioning, minimal interactions, step-by-step guidance. Each accessibility decision had to work across both platforms without breaking the immersive experience.
Cross-Platform Consistency
Here's where my systems thinking kicked in: job coaches would design on web interfaces, but trainees would experience those designs in VR. I established design principles using Human Interface Guidelines, ensuring the "Next" button felt familiar whether you clicked it or pointed at it in virtual space.
Scaling Empathy
The hardest problem wasn't technical—it was emotional. How do you maintain the warmth and patience of human coaching in a digital system? I solved this by making the technology invisible. Job coaches input their expertise naturally, and trainees received that same caring guidance, just amplified infinitely.
Towards the solution
How can we provide a holistic training and evaluation tool that satisfies job coaches' needs?
Design Decision 4: Designing Web App User Journeys
Job coaches needed three things—
streamlining prompt creation (for VR scenario generation), enabling robust evaluation, and simplifying workload management—so they can train, track, and support more effectively than ever.
Sketching, Storyboarding
Lo-fidelity Wireframes
Moodboard
Typography
Color
Iconography
Final solution
Successfully launched VRSpeechChatbot across the U.S. driving measurable change
$1.2M
saved annually
5+
organizations pan USA
100+
user stories
40k+
neurodivergent employees
We observed a marked improvement in trainee confidence during workplace conversations and saw job coaches expand the scope and depth of their training.
Reflection
Embracing Ambiguity This project taught me that the most impactful design work happens in uncharted territory.
Listening Through Change The pivot from trainee-focused to coach-empowering wasn't failure—it was user feedback in action. Good design requires constant recaliberation based on real insights.
Scaling Empathy My biggest insight: sophisticated AI means nothing if it doesn't feel human. Technology's highest purpose isn't efficiency—it's extending our capacity to care.
Inclusive by Design Working with people with intellectual disabilities changed everything. True accessibility isn't compliance—it's rethinking how interfaces work for different minds. This inclusive approach now influences how I design for everyone.
This project proved that thoughtful design can scale empathy when we listen deeply and stay committed to the people we serve.
Let’s create experiences worth remembering *
