Author: Niharika Rusia
College: Balaji Law College, Pune
Abstract
The use of generative artificial intelligence in legal research, document drafting, and assistance in decision-making is changing how legal services are provided in India.
However, this shift has exposed an important issue in the legal system: India does not yet have a single comprehensive framework that clearly allocates responsibility if AI-generated legal advice causes harm. AI systems may create inaccurate or false information, such as incorrect legal documents or non-existent legal sources, which can pose risks for individuals, lawyers, businesses, and the justice system. Existing legal scholarship has not fully addressed the specific responsibility gaps that arise from AI-generated legal advice within India’s legal and professional framework.
This research looks into who is legally responsible when AI-generated legal advice causes harm—whether it is the developers of AI, the technology companies, the legal professionals, or the users. It examines if existing legal ideas like negligence, consumer protection, professional duty, contract law, and information technology law are enough to handle these risks. The research suggests that neither complete protection for AI developers nor strict liability for every mistake is the right solution. Instead, it recommends, as a proposed reform, a risk-based approach that involves shared responsibility.
This framework would include transparency, clear information, human control, professional accountability, the ability to audit AI systems, and fair remedies for those affected. The study concludes that clearly assigning responsibility is necessary to protect people, maintain the quality of legal services, and support responsible AI development in India.
To The Point
Artificial Intelligence (AI) is quickly changing the legal field. Lawyers and people involved in legal cases are increasingly using AI tools for research, document writing, summarizing, translating, and finding information. However, AI can sometimes create legal statements that are convincing but incorrect, such as fake legal rulings, wrong legal rules, or false citations. These mistakes are often called AI hallucinations.
The main legal question now is: who is responsible if AI-generated legal advice is wrong and causes damage?
This issue has become especially important in India because of the Supreme Court’s judgment in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, decided on 2 July 2026. The Court addressed AI-generated citations that were found to be non-existent or improperly attributed. It stressed that while AI can support legal processes, it cannot replace human judgment. The Court stated that it is misconduct on the part of an advocate to cite AI-generated precedents without verification and that courts must adopt a zero-tolerance approach.
The new principle emerging is clear: AI can help a lawyer, but it cannot replace them. Therefore, India needs a regulatory system that includes human checks, professional responsibility, transparency, confidentiality, and proportional liability.
Use of Legal Jargon
The regulation of AI-generated legal advice can be examined using established principles of professional duty, care, liability, and regulatory responsibility.
Professional Responsibility: Under Section 35 of the Advocates Act, 1961, lawyers may face disciplinary actions for professional or other misconduct. In Bar Council of Maharashtra v. M.V. Dabholkar, (1976) 2 SCC 291, the Supreme Court emphasized the high ethical standards expected of lawyers. Similarly, N.G. Dastane v. Shrikant S. Shivde, (2001) 6 SCC 135 is often cited on professional accountability and standards expected in legal practice. These principles also apply when misconduct involves careless use of AI.
Professional Misconduct: A lawyer’s duty to the court is independent and cannot be passed on. A lawyer must not mislead the court, as reflected in the Bar Council of India Rules relating to duties to the court.
Consumer Protection: Section 2(42) of the Consumer Protection Act, 2019 defines “service.” However, in Bar of Indian Lawyers v. D.K. Gandhi, PS National Institute of Communicable Diseases, 2024 INSC 410, the Supreme Court held that legal services by advocates do not fall within the Consumer Protection Act. Therefore, consumer remedies against advocates may be limited, although other remedies may still arise depending on the facts, including contractual, negligence-based, or regulatory claims against non-advocate AI service providers.
Human-in-the-Loop: AI should function as a tool to support decision-making. The final legal judgment and responsibility must remain with a human professional.
Regulatory Gap: Currently, India does not have a single, comprehensive law specifically for AI-generated legal advice.Regulation for AI-related legal work is presently spread across the Information Technology Act, 2000, the IT Rules, and policy guidance and advisories issued by the Ministry of Electronics and Information Technology. This gap makes a case for a new, risk-based framework for AI-assisted legal practices.
Thus, existing legal principles provide some basis, but India needs clearer rules on verification, professional responsibility, law firm liability, and AI provider accountability.
The Proof
The strongest evidence comes from Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668. The Supreme Court reviewed cases where adjudication bodies had used AI-generated legal authorities that were later found to be false or wrong. The Court overturned the affected decisions and stressed the need for human control in AI-assisted legal processes.
This judgment is especially important because it treats reliance on unverified AI-generated sources as a serious issue of professional responsibility, not merely a technical error.
A helpful comparison is Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023). Lawyers submitted AI-generated fabricated case authorities, leading to sanctions from the United States District Court. The decision set a clear rule: technological help does not excuse lawyers from verifying information given to the court.
India already has some relevant laws. The Advocates Act, 1961 governs professional discipline, and the Bar Council of India Rules outline duties towards courts and clients.
Data protection is another concern. Legal professionals may use AI platforms to handle private legal documents, financial details, and personal information. The Digital Personal Data Protection Act, 2023is therefore relevant in cases where AI systems process personal data.
Collectively, these cases show that India has some pieces of an AI-liability system but still lacks a full framework specifically addressing AI-generated legal advice.
Case Laws
Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668
The Supreme Court held that a decision based on non-existent, fake, or hallucinated material generated through Artificial Intelligence cannot be sustained. The Court set aside the NCLT and NCLAT orders, stressed that adjudication must remain under human control at every stage, and declared that it is misconduct for an advocate to cite AI-generated precedents without verification.
Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023)
The U.S. court sanctioned lawyers for submitting fabricated authorities generated with the help of ChatGPT. The case illustrates the broader principle that lawyers remain responsible for verifying every authority filed before a court.
Bar of Indian Lawyers v. D.K. Gandhi, PS National Institute of Communicable Diseases, 2024 INSC 410
The Supreme Court held that legal services provided by advocates do not fall within the Consumer Protection Act’s definition of “service.” This is relevant when considering remedies for harm caused by incorrect legal advice.
Conclusion
AI is neither inherently harmful nor inherently reliable. Its legal significance depends on how it is used, who relies on it, and what safeguards are applied.
A complete prohibition on AI in legal practice would be impractical and could deprive the profession of substantial benefits. AI can accelerate research, assist drafting, improve translation and potentially increase access to legal information. However, permitting unrestricted reliance on AI-generated legal advice would undermine professional standards and public confidence in the justice system.
India should therefore adopt a human-accountability and risk-based regulatory model.
First, every AI-assisted legal opinion or court submission should have an identifiable human professional responsible for its final content.
Second, AI-generated judgments, statutory provisions, quotations and legal propositions must be independently verified against authoritative sources.
Third, the level of scrutiny should correspond to the potential harm. A formatting error is fundamentally different from an incorrect criminal-law provision or limitation period.
Fourth, lawyers and AI providers must protect confidential and personal information processed through AI systems.
Finally, liability should be allocated according to control, reliance, foreseeability and breach of duty. A lawyer who blindly relies upon fabricated AI authorities should not escape responsibility by blaming the software. Equally, a technology provider that makes misleading claims or fails to implement reasonable safeguards should not automatically escape scrutiny.
The future of legal practice should therefore not be framed as human versus AI. The appropriate model is human judgment enhanced by AI, subject to human accountability.
AI can generate an answer. It cannot bear the professional consequences of giving that answer.
FAQs
Q1. Who is liable when AI gives incorrect legal advice?
Liability depends on the circumstances. Responsibility may potentially arise for the lawyer, technology provider or another responsible party depending upon control, reliance, duty, negligence and resulting harm.
Q2. Can an advocate cite an AI-generated judgment?
An advocate should never cite a judgment merely because an AI system identifies it. The actual judgment must be located and verified through an authoritative source.
Q3. Can AI hallucination amount to professional misconduct?
Potentially, yes. The Supreme Court’s 2026 decision in Pooja Ramesh Singh makes clear that submitting unverified AI-generated authorities can raise serious professional-misconduct concerns.
Q4. What is the most important safeguard?
Independent verification. AI should be treated as a legal research assistant, not as an authoritative source of law.
