8 human-centred AI principles for trustworthy and reliable AI products and systems

I have spent many years reading and studying the latest HCAI academic research on top of 15+ years as a product designer and product leader including leading AI products. Over this time I’ve distilled my knowledge and research into key 8 principles you need to apply to make your AI products and systems safe, reliable and trustworthy. Let’s get straight into it. 

AI products and systems are stochastic in nature and therefore can often feel like an unintuitive blackbox, it is fundamental that when an AI system is taking and action like sorting things, checking numbers or spotting patterns, that it maps to how a human being would approach the task (their mental model). This makes the experience feel obvious and familiar and helps the product feel trustworthy.

Well designed AI products and systems don’t just give users an input and wait for an output. They let people take control of actions and make dual decisions. Think beyond designing a search engine with one input box, you’re limiting your ability to augment your users in a positive way.

Employing good ethical and data practices are fundamental to making your AI product or system safe and trustworthy. You should be thinking and auditing for what potential bias exists and how to offset them with techniques like fine tuning, keeping personal data anonymous, strong guardrails and please never train with someone's data without their consent, it will come back to bite you.

There is a great misconception that for systems to be autonomous you must reduce human control, this is false. Human-centred AI systems embrace and encourage high autonomy and human control. AI can complete autonomous tasks and still leave people able to step-in, override, undo or change direction when they want. 

AI systems are complex with lots of surface to fail and the potential to do harm. This is why extensive logging and observability tools should be implemented to track what when in and what came out and how the system is failing. Implementing this from the start of development will result in a safer and more reliable experience.

AI systems can do things we want to do on our behalf, and the more valuable that task is, usually there is more risk to it like writing mission-critical code or analysing financial data. Some actions the AI can take in these situations are things like deleting files, making breaking code changes, producing the wrong output, I call these, changes of consequence. Such changes of consequence should always have a way to be undone. You can do this by implementing patterns like undo/redo and version history.

AI systems have big safety risks like hallucination, misinformation, bias, false positives and many more risks. Having a clear understanding of the safety risks before shipping to production and having a strategy to mitigate them is key.

It is fundamental for AI products and systems to be transparent with present and future actions with enough time for the human to act (stop, change direction ect). Actions should be communicated in a simple yet informative manner that is not cognitively overbearing. Claude for example tries to do this with its reasoning dropdown letting you see the models thoughts however it is very cognitively demanding and ends up not being helpful.