Why AI-generated legal advice may be misleading — even when the AI is not biased.

Artificial Intelligence (AI) has rapidly become one of the most accessible sources of legal information. Platforms such as ChatGPT, Claude, Gemini and Grok allow users to obtain explanations of legal principles, analyse disputes, draft documents and explore litigation strategies within seconds. Yet, as more people turn to AI for legal advice, an important concern has emerged: 

Can an AI system provide an objective legal opinion when the person requesting that opinion presents only their own subjective version of events? 

The answer reveals a problem that is frequently misunderstood. We often blame AI for producing biased or misleading answers without considering whether the real problem originated with the information supplied by the user. 

What if the AI is answering the question correctly, but the user has asked the wrong question, omitted crucial facts or presented a distorted version of events?

THE USER'S VERSION IS NOT NECESSARILY THE TRUTH

Consider a common situation. A person approaches ChatGPT and explains:

"My employer dismissed me without any warning or disciplinary hearing. I have worked for the company for five years and have never received a written warning. Surely my dismissal is unfair?" 

The AI might explain that South African labour law generally requires dismissals to be substantively and procedurally fair and identify possible grounds for challenging the dismissal. Based on the information provided, that response may be entirely reasonable — but what has the employee omitted? Perhaps the employee was caught stealing company property. Perhaps the employee admitted to the theft. Perhaps the employer conducted an investigation, afforded the employee an opportunity to respond and dismissed the employee after considering the circumstances. Alternatively, perhaps the dismissal arose from operational requirements rather than misconduct. 

Each additional fact may materially alter the legal analysis. The AI did not necessarily provide a biased opinion. It provided an answer based on an incomplete and potentially biased account. This distinction is fundamental — the user may sincerely believe that their version is complete and accurate. However, people naturally describe disputes from their own perspectives, often emphasising facts supporting their position while minimising or overlooking facts that undermine it. They may not deliberately conceal information. They may simply not appreciate its legal significance. Unfortunately, what the user considers irrelevant may be precisely what determines the legal outcome.

THE MISSING PIECE: THE ATTORNEY'S CONSULTATION

This is where the distinction between consulting an attorney and independently prompting an AI system becomes particularly important. An experienced attorney does not ordinarily sit silently while a client recounts their story, accept every assertion as established fact and immediately formulate a legal opinion. 

The attorney interrogates the narrative. The attorney asks questions, identifies contradictions, requests supporting documents and explores facts the client may have overlooked. More importantly, the attorney's questions change dynamically as the consultation progresses. Consider a client who explains:

"I lent someone R500,000, and now they refuse to repay me."

An attorney might initially consider a straightforward contractual claim — but the consultation continues, the attorney answers to the following questions:

"Was the loan agreement recorded in writing?"

   "No."

"Who actually transferred the money?"

   "My company."

"Was the defendant aware that the money came from your company?"

   "Not really. We agreed personally."

"Was the money reflected as a loan in the company's accounting records?"

   "I am not sure."

"Has the defendant made any repayments?"

   "Yes, into the company's bank account."

Suddenly, a seemingly straightforward claim raises questions regarding the identity of the creditor, the parties' contractual intentions, the company's involvement, the effect of subsequent payments and the evidence required to establish the cause of action. The attorney has not merely collected information. The attorney has actively transformed an initially subjective account into a more comprehensive factual enquiry. That is one of the most valuable functions of a proper legal consultation.

An AI system can also ask these questions, provided it is appropriately instructed to do so. However, users frequently bypass this process by immediately requesting a legal conclusion or a document supporting their version. They ask the AI to provide an answer before allowing it to establish whether the question rests on reliable factual foundations.

WHEN THE SAME DISPUTE PRODUCES OPPOSITE ANSWERS

Imagine two parties involved in a contractual dispute. The first approaches ChatGPT:

"I paid a contractor R150,000 to renovate my property. He abandoned the project without completing the work. Can I recover my money?"

The contractor independently approaches the same AI system:

"I completed substantial renovations, but the homeowner refused to approve necessary additional costs and prevented me from finishing. Can I claim damages?"

Both parties are describing the same dispute.Both may receive legal explanations that appear to support their respective positions. Does this mean the AI is biased towards whoever asks the question? Not necessarily — the two parties have supplied different factual accounts, each emphasising circumstances favourable to their position. 

One has omitted the alleged additional costs and obstruction, while the other has omitted the allegation that the work was abandoned. The AI is not necessarily contradicting itself —  it is analysing two different versions of the facts. 

If both parties supplied the same complete, accurate and independently established facts, the legal conclusions should be substantially more consistent. The difficulty is that neither party may possess, recognise or willingly disclose the complete factual picture. This phenomenon can be described as input bias: the distortion that arises when the information supplied to a decision-making system is incomplete, selective or unrepresentative. It is not unique to artificial intelligence — attorneys, judges, investigators and other professionals can also reach incorrect conclusions when presented with misleading or incomplete evidence. The difference lies in how effectively the decision-maker identifies and challenges those deficiencies.

CONFIRMATION BIAS: ASKING AI TO TELL YOU WHAT YOU WANT TO HEAR

Another frequent mistake is the manner in which users formulate their questions. Consider these two instructions: 

Instruction A: "My landlord has unlawfully cancelled my lease. Explain why I am entitled to damages and identify the laws supporting my claim. "

Instruction B: "My landlord has cancelled my lease. I believe the cancellation was unlawful. What information would you need to determine whether the cancellation was legally justified? Please evaluate the possible arguments available to both parties."

The first instruction assumes the landlord acted unlawfully and directs the AI towards establishing a claim for damages. The second invites an independent legal assessment. This matters because AI systems can sometimes exhibit sycophancy, a tendency to reinforce a user's expressed views or preferred conclusions. OpenAI has publicly acknowledged this risk. However, we must distinguish that genuine technological limitation from a user supplying incomplete facts. 

An inaccurate conclusion does not automatically mean the AI was biased. It may simply mean the user unintentionally created the conditions for an unreliable answer. When you ask AI to prove that you are right, you should not be surprised when the resulting response concentrates on arguments supporting your position. That is different from asking whether you are right.

HALLUCINATIONS: A DIFFERENT PROBLEM ALTOGETHER

AI systems have another recognised limitation: hallucinations. A hallucination occurs when an AI generates information that appears convincing but is inaccurate, unsupported or entirely fabricated.In legal research, this may include nonexistent judgments, incorrect statutory references or legal principles falsely attributed to actual authorities.Hallucinations are not necessarily caused by biased users. 

They arise from limitations in the technology itself, including how language models are trained and evaluated. South African courts have already confronted this problem. In Mavundla v MEC: Department of Co-operative Government and Traditional Affairs, KwaZulu-Natal and Others [2025] ZAKZPHC 2, the High Court criticised the reliance on nonexistent legal authorities and emphasised the responsibility of legal practitioners to independently verify their research.

The matter illustrates why legal professionals cannot simply reproduce AI-generated authorities without examining their authenticity and relevance. Importantly, hallucinations and input bias are different problems. One concerns the reliability of information generated by the technology. The other concerns the reliability and completeness of the information supplied to it.Neither should be confused with the other.

ARTIFICIAL INTELLIGENCE IS A TOOL, NOT YOUR ATTORNEY

The greatest mistake users make is treating an AI-generated response as a definitive legal opinion rather than an analysis based on the information available. AI cannot independently establish the truth of disputed allegations merely because someone has described them convincingly. Nor can it automatically identify every important fact that a user has omitted. It can, however, become considerably more effective when instructed to adopt an investigative approach.

Instead of asking: "Do I have a case?" 

Consider asking:

"Before providing a legal opinion, question me as an experienced South African attorney would during an initial consultation. Identify missing facts, request clarification, consider the opposing party's likely version and distinguish what I believe happened from what I can prove."

This instruction changes the nature of the interaction. Rather than merely processing the user's preferred narrative, the AI is encouraged to challenge its completeness. Users should also ask AI to identify weaknesses in their cases, examine opposing arguments and distinguish established legal principles from uncertain propositions. Even then, important legal conclusions and authorities must be independently verified.

THE REAL LESSON: BETTER QUESTIONS PRODUCE BETTER LEGAL ANALYSIS

Artificial intelligence is neither inherently biased towards every person who uses it nor guaranteed to provide an impartial, accurate legal opinion.It is a powerful analytical tool whose usefulness depends partly on the quality of the information and instructions it receives. 

An attorney's professional advantage lies not simply in knowing the law, but in understanding which questions must be asked before the law can properly be applied. That skill is developed through experience, legal knowledge, critical thinking and the ability to recognise the significance of facts that clients themselves may consider irrelevant. AI can assist with that process. 

It cannot automatically replace the professional judgment required to establish a reliable factual foundation. We should therefore be cautious about blaming the technology every time an AI-generated legal opinion proves inaccurate. Sometimes the technology genuinely fails — but sometimes the AI answers the question it was asked, while the user fails to provide the information necessary to answer the question that should have been asked. 

This that brings us to a fundamental principle of legal practice: The quality of a legal opinion depends not only on the knowledge of the person — or system — providing it, but also on the completeness and accuracy of the facts upon which that opinion is based. So — before blaming AI for giving you the wrong answer, ask yourself a more important question:

"Did I give it the whole story?"


Copyright © 2026 Rohan Lamprecht. Disclaimer: The information in this article is of a general nature for educational purposes only, relevant to the publishing date. Any opinions expressed are solely those of the author and do not necessarily reflect the views or opinions of Grobler Malope Inc. The content is not intended to constitute professional or legal advice, and you are encouraged to call and consult with our attorneys to discuss your specific situation before making any decisions. Grobler Malope Inc - 087 057 1790 - info@gmilaw.co.za