Author: Arpita Kamble, a student at HVPS College of Law, Mumbai University.
LinkedIn Profile Link : https://www.linkedin.com/in/arpita-kamble-2953a1334
Artificial Intelligence (AI) is increasingly being used in the legal field for research, drafting and analysis. While AI can make legal work faster and more efficient, it can also generate inaccurate or completely non-existent case laws, commonly known as AI hallucinations. The use of such material in judicial proceedings can create serious concerns regarding the accuracy and reliability of legal decision-making.
This issue came before the Supreme Court of India in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, decided on 2 July 2026. The Court found that the NCLT had relied upon several case citations that were either non-existent, incorrectly identified, or contained paragraphs that could not be found in the actual judgments. The Supreme Court consequently set aside the orders of the NCLT and NCLAT and directed the matter to be reconsidered in accordance with law.
The judgment highlights an important principle: AI may assist lawyers and courts in legal research, but it cannot replace human verification and judicial application of mind. Every legal authority relied upon in court must be genuine, relevant and properly verified. The case therefore raises significant questions about judicial accountability, professional responsibility and the responsible use of AI within the Indian legal system.
The issue of AI-generated fake case law involves several important legal concepts, including judicial accountability, professional responsibility, legal research, judicial discretion, procedural fairness, and the rule of law. Case law refers to legal principles established through judicial decisions, which courts rely upon while interpreting and applying laws. Therefore, the use of fabricated or inaccurate authorities can undermine the reliability of the adjudicatory process.
The term “AI hallucination” refers to a situation where an Artificial Intelligence system produces information that appears accurate but is factually incorrect, misleading, or entirely fabricated. In the legal context, this may include non-existent judgments, incorrect case citations, or statements falsely attributed to courts. Such errors become particularly serious when they are presented before a court without proper verification.
Judicial accountability requires courts and legal professionals to ensure that judicial proceedings are based on authentic and reliable legal material. The principles of natural justice and due process further require that parties receive a fair opportunity to present and challenge the material relied upon during a proceeding. At the same time, advocates have a professional responsibility to assist the court by presenting accurate legal submissions and genuine authorities.
The emergence of AI therefore creates a need to balance technological assistance with human oversight. AI may assist in legal research, drafting and analysing information, but it cannot replace the independent application of judicial mind. Verification of authorities, accuracy of submissions and adherence to professional ethics must remain essential requirements of legal practice.
The case of Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, arose from insolvency proceedings involving Essel Infraprojects Ltd., of which Pooja Ramesh Singh was a suspended director. The company had acted as a corporate guarantor for a loan obtained by Pan India Utilities Distribution Company Ltd. from Jammu and Kashmir Bank. The Bank initiated proceedings under Section 7 of the Insolvency and Bankruptcy Code, 2016, before the National Company Law Tribunal (NCLT), Mumbai.
The NCLT admitted the application and its decision was subsequently challenged before the National Company Law Appellate Tribunal (NCLAT). While deciding the matter, the NCLT relied upon several judicial authorities in support of its findings. However, during the appeal before the Supreme Court, it was brought to the Court’s attention that a number of the cited authorities were either non-existent or contained paragraphs that could not be found in the actual judgments.
The Supreme Court independently examined the cited authorities and found that six citations relied upon by the NCLT were problematic. Some had incorrect citations, some contained non-existent paragraphs, and some judgments themselves did not exist. The Court found that AI-generated material had been wrongly attributed to genuine citations and had been relied upon without proper verification.
This raised a serious concern regarding the reliability of AI-generated legal material and its use in judicial proceedings. The case therefore brought the issue of AI hallucinations directly before the Supreme Court and required the Court to examine the effect of relying upon fabricated legal authorities in the administration of justice.
The Supreme Court was required to consider whether a judicial or quasi-judicial decision can remain valid when it is based on fake, non-existent or inaccurate case law. The Court also examined the consequences of relying upon legal authorities that had not been properly verified before being used in judicial proceedings.
Another important issue was the responsibility of legal professionals and adjudicating authorities while using Artificial Intelligence for legal research. The Court had to consider whether AI can be used as a supporting tool in legal work without compromising the accuracy and reliability of judicial proceedings.
The case also raised broader concerns relating to judicial discipline, professional responsibility, accuracy of legal research and the rule of law. The Supreme Court emphasised that technological tools may assist legal professionals and courts, but the responsibility for verifying legal authorities and applying judicial mind cannot be transferred to Artificial Intelligence.
The judgment therefore highlights the need for responsible use of AI in the legal profession and establishes that technological assistance must always remain subject to human verification, legal ethics and judicial accountability.
Decision and Legal Proceedings
The Supreme Court carefully examined the orders passed by the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT). It found that the NCLT had relied upon several authorities that were either non-existent or did not contain the legal principles attributed to them. The NCLAT subsequently failed to detect these errors while deciding the appeal. The Supreme Court treated this as a serious defect because judicial decisions must be based on authentic and verifiable legal authorities.
The Supreme Court therefore set aside the orders passed by the NCLT and NCLAT dated 28 August 2024 and 11 September 2025. It restored the Section 7 application to its original number and directed the NCLT to reconsider the matter in accordance with law. The Court also directed that the application be disposed of expeditiously and ordered the parties to maintain status quo until its disposal. The Court made it clear that it had not expressed any opinion on the merits of the insolvency proceedings.
The decision is significant for the emerging relationship between Artificial Intelligence and the legal system. The Supreme Court declared that courts should adopt a zero-tolerance approach towards producing, citing or using AI-generated precedents without verification. It also stated that citing such material without verification amounts to professional misconduct for an advocate, while reliance upon fake or hallucinated AI-generated material by a judge is a serious lapse. The judgment therefore establishes that AI may assist legal work, but it cannot substitute human verification, legal reasoning and judicial application of mind.
The judgment in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. is significant because it addresses an emerging challenge created by the use of Artificial Intelligence in the legal system. It demonstrates that the use of technology in legal research must not compromise the authenticity and reliability of judicial authorities.
The case highlights the importance of verification in judicial proceedings. A fabricated case citation may appear genuine and may contain convincing legal reasoning, but reliance on such material can result in an incorrect application of law and may adversely affect the rights of the parties. This makes verification of legal authorities an essential part of responsible legal practice.
The judgment also reinforces the principle of judicial accountability. Courts must independently examine the material placed before them, while advocates and other legal professionals have a responsibility to ensure that the authorities cited before a court are genuine, relevant and accurately represented.
At the same time, the judgment does not suggest that Artificial Intelligence should be completely excluded from legal practice. AI can assist with legal research, drafting and organisation of information when used responsibly. However, its output must remain subject to human scrutiny. The case therefore represents an important step towards balancing technological advancement with judicial integrity, professional responsibility and the rule of law.
Artificial Intelligence (AI) is rapidly becoming a part of modern legal practice, assisting in legal research, drafting and analysis. While these technological developments can improve efficiency, they also create new risks when AI-generated information is treated as authentic legal material. One such concern is the generation of fake or non-existent case laws, commonly known as AI hallucinations.
The Supreme Court of India’s decision in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, brings this issue into sharp focus. The case demonstrates how reliance on unverified AI-generated legal authorities can affect the integrity of judicial proceedings and may result in decisions being based on incorrect legal propositions.
This article examines the legal and practical concerns arising from AI-generated fake case law, with particular emphasis on judicial accountability, professional responsibility, verification of legal authorities and the rule of law. It also considers the appropriate role of Artificial Intelligence in the Indian legal system and the need to maintain human oversight while adopting emerging technologies.
1. Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. (2026)
In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, the Supreme Court of India dealt with the serious issue of relying upon inaccurate and non-existent case law in judicial proceedings. The Court found that certain authorities relied upon by the lower forums could not be verified and did not contain the legal propositions attributed to them.
The Supreme Court held that judicial decisions must be founded upon authentic and verifiable legal authorities. The judgment highlighted the dangers of relying upon AI-generated material without proper verification. The Court made it clear that Artificial Intelligence may assist in legal research, but the responsibility to verify the accuracy and existence of cited authorities remains with the person relying upon them.
The decision is particularly important in the present technological environment because it establishes that the convenience offered by AI cannot override the basic requirements of judicial accuracy, professional responsibility and application of judicial mind.
2. M. Subramaniam v. S. Janaki (2020)
In M. Subramaniam v. S. Janaki, (2020) 16 SCC 728, the Supreme Court delivered a genuine judicial decision. The case became relevant in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. because one of the citations relied upon by the NCLT was presented under an incorrect case name, although the citation number actually corresponded to M. Subramaniam v. S. Janaki.
The Supreme Court found that the citation had been wrongly associated with another case name. This demonstrated how inaccurate or AI-generated legal material can combine genuine citation details with an incorrect case title and create a misleading appearance of legal authority.
The case highlights an important lesson for legal research: verifying only the case name or citation is not sufficient. The actual judgment, relevant paragraph and legal proposition must be checked before an authority is relied upon in judicial proceedings.
The emergence of Artificial Intelligence in the legal sector presents both opportunities and challenges for the Indian judicial system. While AI can significantly assist in legal research, drafting and analysis, the possibility of generating fake or inaccurate case law cannot be ignored. The decision in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. demonstrates the serious consequences that may arise when unverified AI-generated material is relied upon in judicial proceedings.
The judgment reinforces that technology must remain a tool for assisting the administration of justice and cannot replace human reasoning, verification or judicial application of mind. Lawyers and other legal professionals must exercise caution while using AI-based tools and ensure that every authority relied upon is genuine and relevant.
Going forward, the Indian legal system may need clearer guidelines governing the responsible use of Artificial Intelligence in courts. Proper verification mechanisms, professional accountability and technological awareness can help minimise the risks associated with AI-generated legal content. The objective should not be to reject technological advancement, but to ensure that its use remains consistent with accuracy, fairness, judicial integrity and the rule of law.
