Why most AI Products and Systems currently fail people?
We are currently in the all consuming hype of AI. Every week I see dozens of new AI products being released and very few are designed well and even fewer have embedded Human-Centered AI. Many of them have a USP rooted in efficiency and sadly very few enable being able to do something brand new and exciting in a safe, reliable and trustworthy way. The latter part of that last sentence is the most pivotal point; safety, reliability and trustworthiness should be of utmost importance but often is thrown to the side. I strongly believe human-centered AI design is crucial to implementing this potentially incredible technology in a way that is beneficial to people. In this edition, I'm going to talk through some of the common pitfalls I think people make when designing and developing AI products and systems. I could write a book on this topic as there are so many reasons but i’m going to focus on a few of the common reasons I observe why AI products and systems are failing people.
It still is about humans!
The development of software is changing at astronomical speed, it is possible to go from idea to a fully functional prototype in a day or two, but building great software isn’t about code, it’s about people. Human-centered AI is an evolution of user-centered design where the needs and pains of people are at the heart of the product or system. This is still the case and will always be the case. Good products serve people. AI should serve people. This feels fairly elementary but start with a clear pain point or need people have, your goal at first is to evidence the pain or need exists and then evidence your solution solves it. AI does not change this first principle.
AI Product and system builders need to leverage what makes us human. Creativity & empathy are two distinctly human traits that AI can not do even though it will pretend it can. Please use these traits. Firstly, to employ creativity we need to reframe the question, I believe most AI product and system builders are trying to fundamentally answer in some guise, ‘How can we use AI to automate what a human currently does?’. Starting off with this question will always lead you down a path of uncreative AI solutions that just try to automate everything with little human control and this is a recipe for unsafe, unreliable and untrustworthy products and systems and who wants to use a product like this?. Not me. Who is going to pay for a product like that? I’d bet very few.
Better framing is ‘How can we solve people's pains and extend their capabilities?’. To develop AI products and systems through this framing requires empathy first. You must have a deep understanding of the pain, why is it a pain? What are the effects of the pain? What does the pain make the person feel and think? What is the person's objective?. You need to resonate with the pain.
Now onto creativity, this is where we need to think about how we can solve the pains potentially with AI. This is where a mindset change is required and why most people fall back to ‘lets use AI to automate what humans are currently doing’ thinking. The last 70 years of software has had limitations on what was possible, the interaction model of software was to enter an exact input and you get an exact output. Think about some of your most used non-AI software and ponder about all the individual tasks you may do in said software. They all require you to provide an exact input to get an exact output. Click a button to send an email, enter a value in a spreadsheet, point and click a contact in your messaging app to read your conversation. This is deterministic software which requires programmers to determine the outcomes of an input. Exact input, exact output. Understandably, this is now the mental model of software designers, builders and product people when approaching a problem. We need to break this and employ our human creativity. In another edition I will talk about a new probabilistic interaction model which you can use to develop silky AI experiences.
Developing AI products and systems mean the options are almost limitless. Think about Google Maps being able to find you the quickest route on the fly with all the possible options and scenarios, smartphone cameras that can automatically find the perfect aperture, focus, white-balance or Apple Music predicting what you want to listen to next based on your mood, time of day and listening habits.
We’re now in the era of probabilistic software. Software that can ‘think’, evolve, learn about us, predict what we want, act on our behalf. We need Human-Centered AI to facilitate such experiences and we need to embrace human creativity to think of new ways to augment, enhance and extend human capabilities to make people more empowered and capable than ever before.
Should you be using this?
Many of the pitfalls of some of the products and systems I’ve seen recently come down to one thing. They don’t need to use AI. The default answer shouldn’t be ‘lets use a LLM’. If it can be done without AI, do it without AI. Adding AI to your product or system adds a huge surface area for the potential of a poor experience to present itself. Look at the Design for AI Guide patterns and you can quickly see the amount of consideration that is needed to develop a safe, reliable, trustworthy AI experience.
Before developing a product or system I recommend running an AI Opportunity Audit which answers a fundamental question ‘Is AI the right solution here and what genuine new value could it create beyond automating existing tasks?’. This will help you justify why you’re using AI and the actual value it is bringing to the people using your product or system.
AI experiences are very different from traditional deterministic software experiences because you have a stochastic machine built into the software which can lead to many different scenarios some of which are frustrating, dangerous or erroneous. As such you need to think about a range of new things like how can the human overrule the AI when it is making a decision, how does the human reverse an action, making sure AI interactions and intent is transparent and clear to the human and much more. My point is adding AI can be a way to give people incredible powers but as spiderman says with great power comes great responsibility. Don’t add AI for marketing weight.
In-conclusion , if you don’t want to make the same mistakes a lot of existing AI products and systems are making, remember some of the fundamental lessons I mention in this writing. Firstly, it is about humans, you’re designing for the benefit of a human. If a human hasn’t been involved somewhere in the process, you’re shooting in the wind with a blindfold on. Leverage what makes you human, empathy and creativity. Secondly, when building AI products and software you are developing probabilistic not deterministic software (exact input, exact output) this means you need to carefully think about safety, how reliable the software is and what mechanism you need to implement to make it trustworthy. Finally, think deeply about if the solution to your problem actually needs AI. If it does, please do not automate what a person is already doing, give people new abilities they’ve never had before. Provide them real value in a safe, reliable and trustworthy way.