By Hakimuddin Wadlawala, Founder, Aquila I
Artificial intelligence has evolved from providing answers and creating content to systems capable of performing activities autonomously. Agentic AI, as these systems are called, possess the ability to communicate with applications, retrieve data, process information and do many things without human help. The new reality should result in progress, but at the same time, it creates an important cybersecurity issue: who is responsible for the actions of the automated AI if an accident occurs? As companies implement such systems in their work, it will not be possible to look at responsibility for the results of actions through the lens of human mistakes anymore.
Agentic AI Cybersecurity Threats
Standard AI systems in most cases work according to given prompts, while agentic AI can fulfill various objectives. For instance, an AI can be ordered to run IT systems independently and continue modifying configurations and take necessary measures without requiring approvals. This gives rise to new opportunities for cyber attacks. A hacked agent may illegally use its privileges, leak sensitive data, complete forbidden transactions or shut down key systems. Hidden malicious code in emails, documents or websites can be used for forcing the agent to act against the intention. The danger mentioned above is not limited to external intruders. Wrong command and permission levels, poor instruction, or incorrect assumption may help an agent to perform many dangerous actions by itself.
Who is Accountable?
Responsibility in the case of malfunctioning AI agents is complex due to involvement of multiple parties in the agent’s operations. It is primarily the developers’ duty to create the devices that provide the necessary level of security and security testing and vulnerability mitigation. Businesses that use AI systems should make sure the agent is applied for its intended scope, given the strictest access rights and supervised during the operation. Furthermore, top management and information security teams are to create a control environment, do risk assessment and understand how to escalate the issues in case of emergencies. At the same time, tech companies may also be accountable for ruptures due to security shorts or unknown limitations of the system. Nevertheless, a multitude of responsible parties shouldn’t serve as an excuse for the absence of accountability. AI agents cannot be held liable. It is essential to keep people responsible for AI actions within their responsibilities and according to the laws of the country where they operate.
The Problem of Human Oversight
Humans have been seen as a solution to the dangers arising from AI operation. However, it is important to note that keeping a person in the system does not ensure total safety. In case of an agent operating in a rapid manner, checking every single action taken may become unfeasible. Also, countless alerts may only distract from identifying the dangerous actions.Organisations need oversight that is based on risk. Routine tasks that can be reversed can be automated within set limits.. Sensitive actions. Like financial transfers, access to privileged systems handling personal data or managing critical infrastructure. Should always require extra checks or direct human approval.
As important is the ability to step in. AI agents must operate within boundaries. They should have emergency shutdown features and built-in restrictions that stop them from gaining permissions on their own. Any change in access must be approved by a person.
Building Security Into Autonomous Systems
Securing AI means moving beyond protecting single applications. It means taking control of the chain of automated decisions. Organizations need to focus on limiting access. They should give agents the minimum permissions they need. This is called least-privilege access. Where possible keep agents separate from systems. This isolation helps prevent damage if an agent acts unexpectedly. Each agent should have its set of credentials. This way if one agent is compromised it does not give access to others.
Continuous monitoring is key. It should track every action an agent takes. It must also record what data the agent accesses. Any change in behavior should be flagged. Detailed audit logs are essential. They help investigators go back and understand what happened during an incident. They show where the security safeguards broke down. Adversarial testing and red-team exercises should be used regularly. These tests check how agents react to harmful instructions. They also test how agents handle tools that are compromised. Unexpected situations should be tested to see how agents respond under pressure.
Accountability Needs to Change Along With Autonomy
AI brings many chances to make cybersecurity better, automate usual defense work and react to dangers quicker. However the same abilities can make errors worse and open up ways for people to take advantage if autonomy is added without proper controls. The main idea is that more independence needs to come with responsibility.
Developers need to make systems, companies need to manage how they are used and people in charge need to make sure that real control is still possible. In the end the question is not if AI agents should be allowed to work on their own but under what rules they should be allowed to do. Wise use means having ownership, the right level of access, clear choices and ways to stop things if needed. When independent AI makes a mistake, responsibility should not get lost in the technology. It should go back to the people and companies who allowed and controlled what it did.
Express Computer is one of India's most respected IT media brands and has been in publication for 24 years running. We cover enterprise technology in all its flavours, including processors, storage, networking, wireless, business applications, cloud computing, analytics, green initiatives and anything that can help companies make the most of their ICT investments. Additionally, we also report on the fast emerging realm of eGovernance in India.