Built-In Bias: What Every Lawyer Needs to Know About AI’s Hidden Prejudices

Scenario: A junior associate at a mid-sized firm runs a contract through an AI reviewer. It flags “standard force majeure language,” gives it a green check, and moves on. Weeks later, the client discovers the clause excluded pandemics, exposing them to millions in liability. The AI wasn’t malicious. It had simply learned from a training set heavy with seller-favorable deals.

Bias is woven into every layer: data, design, deployment. When lawyers rely on AI without understanding it, they risk client harm, professional exposure, and erosion of justice. AI isn’t neutral. It carries our assumptions, our histories, our blind spots. Bias cannot be removed. It can only be refined, revealed, and managed. This article explains what that means for lawyers: how bias arises in legal AI, why it matters under ethics and discovery rules, when advocacy-aligned models may be defensible, and how to supervise and prompt them responsibly.

II. What “Bias” in AI Really Means

A. Definition in Plain English

Bias in AI refers to systematic distortion or skew in predictions or outputs. It’s not a random error. It’s when the system consistently favors one outcome, perspective, or group over another.

B. Bias Is a Reflection, Not a Glitch

Every human carries bias shaped by upbringing, culture, and experience. That’s why two juries can reach different verdicts from identical evidence. AI is no different. It reflects the data fed to it and the design choices made by its creators. Bias in AI isn’t a bug. It’s a mirror held up to us.

C. The Three Layers of Bias

Data Bias – The training set may underrepresent certain demographics, jurisdictions, or fact patterns.
Algorithmic Bias – The mathematical rules may amplify patterns or suppress nuance.
User Bias – Lawyers’ prompts, expectations, and interpretations shape outputs as much as any dataset.

D. The Measurement Challenge

There’s no single way to measure fairness. Calibration is where the same predicted probability equals the same actual outcome across groups, often conflicts with equal error rates, where false positives and false negatives are identical. You can’t satisfy both simultaneously. In criminal risk tools, enforcing calibration can worsen disparities in error rates, and vice versa.[1][2] A vendor may claim “fairness” based on one metric while ignoring grave disparities under another.

Key takeaway: Lawyers must ask vendors which fairness standard they use and which tradeoffs they’ve accepted.

III. How Bias Manifests in Legal AI Tools

A. Legal Research and Case Summaries

AI research tools may overemphasize majority opinions or high-citation cases, marginalizing minority or dissenting reasoning. The result: you miss the outlier precedent that could reshape your argument.

B. Document Review and Contract Analysis

Review systems trained mainly on large commercial agreements may treat one-sided clauses as “standard.” That subtle skew translates into costly blind spots for consumer or plaintiff firms.

C. Litigation Analytics and Risk Tools

In 2016, ProPublica analyzed more than 7,000 defendants in Broward County, Florida, and found that the COMPAS risk-assessment tool used in sentencing and bail—produced unequal error rates between racial groups.[3] For defendants with the same risk score, actual recidivism rates were roughly equal across races. The model was calibrated. Yet Black defendants were twice as likely to be labeled “high risk” when they did not reoffend.

White defendants were more often misclassified “low risk.” That clash between calibration and equal error rates remains one of the thorniest fairness problems in AI. For lawyers, COMPAS shows that even mathematically consistent models can encode social inequity. The same dynamic plays out when discovery tools prioritize certain document patterns over others, embedding hidden assumptions about what matters.

D. Hiring and HR Algorithms

Amazon’s now-defunct recruiting AI revealed a parallel problem in corporate settings. In 2018, Reuters reported that Amazon discontinued an internal hiring algorithm after discovering it downgraded resumes containing female-coded terms such as “women’s chess club.”[4] The model had been trained on ten years of resumes from a male-dominated workforce. It taught itself to replicate that imbalance. No one told it to discriminate. The bias emerged naturally from the data.

Practice implication: Bias doesn’t require intent. Once deployed, even “objective” AI can perpetuate inequality quietly, at scale.

IV. Why Bias in AI Is a Legal Problem

A. Ethical Duties

Under Model Rule 1.1, competence includes technological understanding. Formal Opinion 512 (2024) reminds lawyers to know the risks and benefits of AI tools.[5] Rule 5.3 requires supervision of non-lawyer assistants, which includes technology. Rule 1.6 safeguards client information; careless prompt-sharing could expose confidential data. Rules 3.1 and 3.3 prohibit false statements and require disclosure of adverse authority. AI hallucinations can trigger both.

B. Liability Risk

Judges have already sanctioned lawyers for fabricated citations. Delegating judgment to algorithms doesn’t dilute responsibility. It amplifies it.

C. Discovery Implications

If AI outputs shaped your litigation strategy, it’s possible in due course that opposing counsel may seek discovery on your methodology, datasets, or model versions. Privilege arguments may hinge on documentation. Keep version logs and prompts as part of your file.

D. Regulatory Oversight

The NIST AI Risk Management Framework (2023) emphasizes transparency and fairness. The EU AI Act (2024) designates legal-decision systems as “high risk.” Lawyers can’t plead ignorance of these standards.

Practice takeaway: Bias isn’t abstract. It creates malpractice exposure, discovery burdens, and regulatory risk.

V. Bias Can’t Be Eliminated, Only Refined

No juror or judge is perfectly neutral, yet the justice system aspires to fairness. AI deserves the same treatment. Neutrality may be impossible, but unexamined bias is inexcusable. Lawyers must measure, document, and correct bias as a matter of competence.

Practice takeaway: Treat bias like precedent. It’s always there, but your job is to interpret and refine it responsibly.

VI. Advocacy-Aligned AI: When Bias Becomes a Feature

A. The Proposition

Law firms now experiment with custom GPTs tuned for plaintiff or defense advocacy. A PlaintiffGPT might emphasize causation and damages. A DefenseGPT might prioritize statutory limits and procedural defenses. The question isn’t whether they can. It’s whether they should.

B. Ethical Analysis

Zealous advocacy allows partiality, not falsehood. A biased model that omits controlling law crosses into misrepresentation. Supervision and documentation remain essential. Advocacy bias cannot become discrimination bias.

C. Client Consent

If an advocacy-tuned model could materially influence a case, client consent may be warranted. Clients should know the model’s role and limitations; particularly if its framing could tilt advice or tone. Transparency builds trust and mitigates ethical risk. Informed consent isn’t just good practice. It’s a shield against claims that AI-driven strategy betrayed client interests.

D. The AI Arms Race

Imagine a near future where both sides deploy advocacy-tuned GPTs in discovery or motion practice, each reinforcing its own worldview. Plaintiff counsel’s model surfaces every sympathetic precedent and frames facts for maximum liability. Defense counsel’s model emphasizes causation breaks, statutory bars, and Daubert challenges.

The result: escalating rhetorical extremes, drafted at machine speed.

Courts may respond by mandating disclosure of AI use, requiring neutral audit modes before filings, or applying Daubert-style admissibility tests to algorithmic reasoning. If the profession doesn’t self-regulate advocacy AI, the judiciary may eventually do so.

Practice takeaway: The question isn’t whether advocacy-aligned tools will proliferate. It’s whether lawyers will govern them before courts impose governance from the bench.

E. Helpful, Not Mandatory

Advocacy-aligned tools aren’t necessary, but they’re quickly becoming integral in high-volume practice areas. Used responsibly with oversight, adverse-authority checks, and version control, they can amplify competence. Without those safeguards, they erode it.

VII. The Lawyer’s Role: Governance and Oversight

A. Treat AI Like a Junior Associate

Trust but verify. Every AI draft requires human review.

B. Vendor Due Diligence

Ask AI software vendors how they detect and mitigate bias, what fairness metrics they monitor, and who oversees model integrity.

Exit strategy: Prepare for vendor drift or discontinuation. Models change. Vendors shut down. If a tool you’ve relied on for months suddenly behaves differently or vanishes, you need archived versions, prompts, and outputs to defend your work. Document everything.

C. Version Control

Models evolve. Behavior shifts with updates. Maintain logs of prompts, model versions, and output changes.

D. Firm Governance

Adopt an AI policy outlining usage, confidentiality, review standards, and escalation paths. Periodically audit outputs for bias.

E. Judicial AI Adoption

Courts may soon rely on AI not only for administrative tasks but, eventually, for case management, analysis, and even the drafting of judicial reasons. While no court of record currently delegates decision-making to AI, early signs suggest the judiciary is already experimenting at the margins. Judges, clerks, and administrative staff are beginning to use generative tools informally for scheduling, summarization, and research support—often without explicit authorization or oversight.

In October 2025, Reuters reported that a U.S. senator had questioned federal judges about whether AI had been used to assist in withdrawn court rulings.[7] The example linked above is a warning to the profession: judicial personnel will use AI, whether sanctioned or not. That reality makes governance, not prohibition, the more prudent path.

Governing bodies must now decide whether to formally sanction AI use at the judicial level—and if so, how to supervise it. Transparent policies and audit trails could prevent the quiet proliferation of unsupervised tools within the court system. It is arguably better for AI to be used safely and openly than for judges to go rogue in the shadows of innovation.

Key takeaway: Lawyers should anticipate, not merely react to, judicial AI. The profession must help set the standards for explainability and oversight before courts adopt these tools at scale. Efficiency without transparency is not progress—it is abdication.

VIII. Practical Guidance

It’s all well and good to discuss these issues in theory, but the real value lies in guidance for putting it into practice. Lawyers can’t afford to treat AI bias as an abstract ethical dilemma. It’s a day-to-day operational issue, one that affects vendor selection, client communication, and even how you draft a simple prompt.

It’s easy to rely on and trust AI software vendors, and in many cases, it’s reasonable to do so. But that doesn’t mean attorneys or firms should blindly sign up and deploy. Every firm has an obligation to understand the bias (and alignment) baked into the tools they use. You wouldn’t outsource legal research to an associate you’ve never met or supervised. The same standard applies here.

A. AI Software Vendor Questions

Ask vendors:

What data trained the model, and how is bias tested?
Which fairness metrics do you use?
How often are audits performed, and are they independent?
Can you trace reasoning and cite sources?
How do you communicate model updates?
Who oversees bias internally?
For your practice area, are datasets balanced by plaintiff/defense or buyer/seller orientation?

B. Prompting for Fairness and Advocacy

Prompts are the lawyer’s new form of instruction. Just as phrasing can sway a witness or frame a cross-examination, the way you phrase an AI query determines what perspective the system adopts. Mastering this skill means understanding when to ask for balance and when to advocate.

Neutral / Research Mode:

“List both majority and dissenting opinions on [issue].”
“Summarize circuit splits and explain the reasoning.”
“Identify what opposing counsel might argue.”

Advocacy Mode:

“Draft arguments supporting the plaintiff assuming favorable facts.”
“Include adverse controlling cases the court would expect.”
“Write a defense motion emphasizing procedural issues; verify citations.”

Two-pass workflow:

Many firms are adopting a two-pass workflow: one system generates the initial draft, and another independently reviews it for accuracy and bias. It’s the digital equivalent of a second set of eyes.

Generate in advocacy mode.
Run a neutral review prompt to surface omissions or bias.

Practice implication: Precision in prompting shapes fairness and credibility.

IX. Case Studies

A. COMPAS (Criminal Sentencing)

In 2016, ProPublica found that the COMPAS risk-assessment tool in Broward County, Florida, was statistically calibrated but racially skewed.[3] Defendants with equal scores reoffended at similar rates. Yet Black defendants were twice as likely to be labeled “high risk” without reoffending. White defendants were more often misclassified “low risk.” The takeaway: technical accuracy does not guarantee justice. Fairness is a choice, not an output.

B. Amazon Recruiting AI (Employment)

Amazon’s internal recruiting model downgraded resumes containing female-coded terms like “women’s chess club.”[4] Trained on male-dominated historical data, it learned patterns that excluded women without explicit instruction. For employment lawyers, that case is a reminder that bias can emerge without intent. Intent is irrelevant when outcomes are discriminatory.

C. Predictive Policing (Law Enforcement)

Predictive policing models often amplify disparities. Over-policed neighborhoods yield more crime data, which reinforces future patrols.[6] The bias becomes cyclical. Lesson: Bias propagates through feedback loops. Unchecked, it becomes precedent masquerading as probability.

X. The Path Forward

Lawyers must embed algorithmic literacy into professional competence. Bias will always exist, but it can be understood, monitored, and corrected. The rule should be simple: Explain or abstain. If you can’t explain a model’s reasoning, don’t rely on it.

XI. Conclusion

When I began studying legal AI, I believed bias was a technical flaw that could be coded away. I was wrong. Bias is a reflection of us—our data, our priorities, our judgments. Eliminating it is impossible. Refining it is our duty. The tools we build mirror the system we serve. Lawyers must ensure that reflection bends toward fairness—and that justice never cedes to the algorithm.

My Take

I’m going out on a limb here because it’s not talked about much. Lawyers have an ethical obligation to use biased AI when that bias mirrors zealous advocacy for their clients. I’m not suggesting manipulation, suppression of evidence, or anything unethical. I’m talking about aligning AI’s analytical leanings with a lawyer’s legitimate role of presenting a client’s case in the best possible light. A pro-plaintiff or pro-defendant bias is not malpractice; it is strategy.

Firms that invest in custom-built AI tuned to their advocacy style may gain a real edge. These systems can frame arguments, interpret evidence, and prioritize precedents through the lens of client advantage. Neutral AI will not do this as effectively.

And it will not stop there. Smart firms will also deploy mirror models, AI trained to think like the opposition, to anticipate counterarguments and simulate adversarial reasoning. This adds cost and complexity, but nuanced advocacy-aligned AI will soon become a competitive moat for firms that take representation seriously.

For now, only customized systems can achieve this level of alignment. In time, the major legal AI platforms will introduce similar advocacy modes. When that happens, the firms that already understand how to wield bias ethically will be the ones ahead of the curve.

What do you think? Leave a comment below.

References

[1] Alexandra Chouldechova, “Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments,” Big Data 5(2): 153–163 (2017). Published by Mary Ann Liebert, Inc. https://doi.org/10.1089/big.2016.0047

[2] Pleiss et al., “On Fairness and Calibration,” Advances in Neural Information Processing Systems 30 (2017). https://papers.nips.cc/paper/7151-on-fairness-and-calibration.pdf

[3] Julia Angwin et al., “Machine Bias,” ProPublica (May 23 2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

[4] Jeffrey Dastin, “Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women,” Reuters (Oct 10 2018). https://www.reuters.com/article/amazon-com-jobs-automation-insight-idUSKCN1MK08G

[5] American Bar Association, Formal Opinion 512: Generative Artificial Intelligence Tools (July 2024). https://www.americanbar.org/content/dam/aba/administrative/professional_responsibility/aba-formal-opinion-512.pdf

[6] Kristian Lum & William Isaac, “To Predict and Serve?” Significance 13(5): 14-19 (2016). Published by Wiley on behalf of the Royal Statistical Society. https://doi.org/10.1111/j.1740-9713.2016.00960.x

[7] David Thomas, “U.S. Senator Asks Judges if They Used AI in Withdrawn Court Rulings,” Reuters (Oct. 6, 2025). https://www.reuters.com/legal/government/us-senator-asks-judges-if-they-used-ai-withdrawn-court-rulings-2025-10-06/

I. Introduction