The above drawing from Banksy captures who we think the true heroes are. It’s not us, technologists or technology. It’s the trainers, health and front-line workers who protect and serve us everyday.
The open letter below is especially for the Australian AI Sprint Community to introduce Conja, myself and the Vision x AI x Spatial x Twins tools we craft to supercharge mobile workforces.
Hi Everyone,
I’m Anthony Chung.
Founder of Conja (conja.com), spun off from Mobile Learning (mobilelearning.io)
Intro notes below on:
my background,
the gap,
true heroes,
who we can build for,
co-design
hope
Background = Tool maker
I craft tech tools for training people who work on their feet.
Been working with technology to help scale training since early 2000s. I help by designing from the ground up and the remote edge in. I pair with Cloud architects and builders. I craft tech to support training in remote environments, regional, mixed internet connectivity and offline.
In the 2000s, worked on PC, Cloud, learning analytics to scale training for banks.
2013+ founded Mobile Learning to design training for offline workers used in Retail, NHS, Health, Police, Gov and Defence.
2023+ Conja = Crafting trainer tools with Vision x AI x Spatial x Twins.
The Gap
I love tech, but the gap between training and work is real.
Technology has always struggled to scale training in the face of labour shortages, high turn over, knowledge transfer, specialists retiring and teams losing knowledge, remote edge contexts, the tyranny of distance and the macro complexities we face with ageing populations.
We always have an “air gap” issue. The gap between when they were trained and when they work. The gap between generalised and idiosyncratic work. Difference between what was learned at Tafe/Uni/that course years ago, the seminar and what happens on the job.
What a person is looking at, contextual visual reasoning, spatial contexts, who, where and when they are - it all makes a difference to widen the gap.
We don’t have silver bullets. But we are always looking for new technologies that can improve training, quality and access to more people.
We are not they heroes, they are
I’ve been thinking about this for years, when our tech teams come to meet Health, Police, Defence, Enterprise and Gov workers. They are the front-line heroes and not us.
We think what supercharges workforces, are human trainers on the front-line. They are the mixture of experts. Their lived experience are the vector embeddings. They are the human-in-the-loop RLHF that we need for basic guidance, let alone training AI models.
They perform visual reasoning and judgment on environments, exercises and equipment. They don’t give written text, they demonstrate how to do work. They watch, give feedback, assess and guide trainees.
If as technologists, we feel like Batman, Superman or Wonder Woman, the Banksy picture above expresses who the real heroes are.
Occupational therapists protect our grandparents from falling down. Frontline workers protect and keep us safe. Facing a growing ageing population. Nurses, Physiotherapists and Allied Health workers are the heroes we need.
Opportunity to make tools for heroes
If we’re builders, we can be Lucius Fox to Batman. We can craft tech tools for front-line heroes.
Cloud and Mobile helped. I’m excited about tooling up front-line trainers and trainees with a new VAST tech stack for this next decade.
Vision x AI x Spatial x Twins (V.A.S.T.)
Co-design feedback loops cycling forward
We need to co-design these tools side by side with on-the-job trainers. Continually improve tooling based on their feedback.
Allied health and front-line workers share their experience with generosity as to what they find useful and how to improve V.A.S.T tech for Trainer product fit.
For the past year, we’ve been building Conja and demonstrating to front-line workers such as Occupational Therapists from Uniting and HammondCare. We’ve interviewed UNSW Professors that have trained Occupational Therapists over decades. Interviewed UTS Academics wrestling with systemic issues in Aged Care. Leaders in Defence Health clinical informatics have advised on architectural strategies. Learning and Development designers working with Australian and NHS hospitals are sharing their expertise.
This is not something we can solve individually. Thankful for a mixture of collaborative experts with lived experience that we co-design with.
Vision x AI
Tim Cook says, the era of Spatial computing means that we look “through” apps instead of “at” apps. We need to push past, looking at AI chat interfaces to looking through AI visual spatial interfaces.
AI needs to be aligned to visual reasoning of what trainers are looking for. There has to be Trainer AI Model Fit. Vision AI tools that guide and support their analysis.
Human-trainer-in-the-loop cycles can test computer vision models for usefulness and failure. Supercharging workforces requires Reinforcement Learning Human Trainer Feedback (RLHTF).
Then we can make Vision AI Trainer Co-pilot tools for Trainees when, trainers are not available. Trainers always use their eyes to check for safety and compliance. Visual AI auto compliance checks can be made more accessible on scale.
Spatial
Our challenge is to simplify Visual Spatial UX interfaces for AI.
Mixed reality can keep trainers in contexts to mark up their spatial environments, instead of being dislocated in virtual environments.
We love XR glasses for the future, however the feedback from allied health and the front line is that glasses will need to become more affordable to be deployed at scale.
So we prioritise designing for mobile phones and tablets instead of glasses so we can scale sooner.
Twins
AI can generate Twins of environments, exercises, people and equipment. These Twins can be built around OpenUSD just as the industry alliance has been formed last year.
The Alliance of OpenUSD supported by Apple, NVidia, Adobe, Pixar and Autodesk means tool makers can get on with it and build on an industry standard. No need to be waiting for VHS vs Betamax wars to be decided.
Jensen Huang says OpenUSD is what HTML was to the Web. We think NVidia Omniverse can function like modern TCP protocols connectors as people engage with OpenUSD.
We don’t develop in isolation, the timing of industry convergence means we can build UX for Twins based on standards instead of vendor fragmentation.
Hope
One day, we’ll scale XR trainer bots, way before we scale humanoid hardware bots. Also betting on mobile before glasses. Most of the world are just affording phones, let alone cars or hardware robots. For now, we’ll have to start by scaling visual spatial mobile AI tools, first for trainers and then for trainees.
As we move out of hospitals into homes. Out of offices into the field. Keen to see Computer Vision x AI, historically locked indoors in surgery rooms, robotics and industry cameras not just used by Surgeons but by Allied Health workers in home environments.
Professor Andrew Ng (Landing Lens) pictures scenarios of day to day use cases of computer vision, analysing pizza toppings. Pizza makers and Allied Health workers are going to have beautifully mundane daily uses of Computer Vision and AI.
Medtronic CEO mentioned that they are hoping to go from 60 million scans, to 600 million each day. I think this can happen if Computer Vision can become an accessible tool for more people.
In a hospital, there are only a few surgeons, lots more trainers, many more trainees and even more patients and family.
We think building Visual AI tools for trainers is the place to start and build from. From there, we will build Vision and AI tooling for trainees and the wider community.
Hope drives our craft over decades. We want to make tools useful for front-line heroes.