Principles
Before designing HCAI solutions it is important to have a good understanding of the core principles, which will keep you in good stead and guide you at decision making points to make sure your system or product is safe, reliable and trustworthy.
Safety is imperative
Autonomous systems do things we want on our behalf, however, they also have a high potential to do things we don’t want (for example LLMs hallucinating) which can cause great harm to people. A clear understanding of the safety risks and a strategy to prevent them is critical.
Transparent interactions
It is fundamental for AI products and systems to be transparent with present and future actions with enough time for the human to act (stop, change direction ect). Actions should be communicated in a simple yet informative manner that is not cognitively overbearing.
AI model meet mental model
For a seamless HCAI experience, the AI’s problem type (classification, regression, etc.) must align with the user’s mental model. When the model’s logic mirrors human intent, automation feels intuitive rather than restrictive. This alignment preserves high autonomy by providing the correct form of control: validating categories, adjusting values, or interpreting patterns.
Logging & observability
AI systems are complex in nature and have a high surface of failure and the potential to do harm. To avoid this, there should be extensive logging and observability implemented to enable you to understand exactly what is happening from input to output.
Control not input
HCAI systems enable the human and AI to work in partnership with each other by enabling the human to make decisions and take action when they want to, not just with an input prompt waiting for an output.
Reversible actions
Many AI systems and products have the ability to do these that are considered changes of consequence like deleting files or making breaking code changes. HCAI products and systems should always allow for the human to revert changes of consequence in a simple and seamless way.
High autonomy & human control
A misconception of autonomous systems is that for high autonomy to be true you must take away human control. This is false. HCAI systems embrace and encourage high autonomy systems but ensure the human remains the ultimate navigator, able to intervene or redirect the system at any time.
Ethics and data
HCAI systems and products protect their people and employ ethics that are in their best interests such as making sure mission critical applications have appropriate fail safes, anonymising data, embedding strong guardrails, preventing data being used for training without consent.