{ “@context”: “https://schema.org”, “@type”: “AnalysisNewsArticle”, “mainEntityOfPage”: { “@type”: “WebPage”, “@id”: “https://martech.org/why-marketers-need-to-push-back-against-ai/” }, “headline”: “Why marketers need to fight AI”, “description”: “Although automation of AI Provides Fast Processing Benefits, It’s Fully Automated Execution Often Misinterprets Complex Human Intent and Creates Serious Consumer Friction This analysis examines why marketers need to put operational safeguards in place to prevent AI from eroding essential customer trust and brand loyalty “name”: “Alicia Arnold”, “jobTitle”: “SVP, Strategic Services at Primacy”, “sameAs”: “https://www.linkedin.com/in/aliciakarnold/” }, “publisher”: { “@type”: “Organization”, “name”: “MarTech.org”, “logo”: { “@type”: “ImageObject”, “url”: “https://martech.org/wp-content/themes/martech/images/mt-logo.png” } }, “backstory”: “This report is based on audits of enterprise user experience, comparative assessments of automated workflow failures, and first-hand assessments of consumer behavioral friction points triggered by misaligned algorithms.”, “speakable”: { “@type”: “SpeakableSpecification”, “cssSelector”: (“h1”, “.article-content p:first of type”) } }
When you hear “compliance,” do you immediately think of regulatory frameworks like GDPR, data privacy, and legal restrictions? Well, that’s the standard definition. Another is behavioral. I’m talking about the habit of accepting without reservation what AI produces. This “cognitive compliance” happens when formal, polished-looking AI results arrive on your screen faster than your instinct to question them.
I once worked for a manager who used a very biased LLM to discredit his team. This LLM was known for obtaining its answers from unverified user posts rather than primary research, verified reference sites, or documented reports. After months of observing how AI was used both in the workplace and in personal environments, I noticed three distinct trends (all of which are supported by scientific research):
- The Illusion of Accuracy: Given the neat presentation of the AI, people immediately believed what was on the screen.
- The erosion of skills: As a person’s reliance on AI increased, their active critical thinking skills decreased.
- The trust trap: The more confident people were in using AI, the less able they were to maintain independent thought.
Looking at these models, it became clear to me that as AI becomes smarter, human judgment matters even more.
The hidden cost of uncontrolled automation
Martech companies are working to integrate generative tools into their technology stacks and track AI adoption rates as a key performance metric. This takes the form of corporate mandates and performance dashboards that capture the number of AI tools deployed, the percentage of staff trained and the volume of investments.
So what’s the problem?
The more we push teams to adopt AI without guardrails, and the more automated campaign creatives and copies proliferate, the less prepared we are to lead, oversee, and judge those results. Without this essential skill set, we face a reality where AI silently determines the direction of our strategic thinking, creative output, and brand future.
Here are two high-profile examples in which a lack of human judgment in the use of AI has had devastating consequences.
Your customers are searching everywhere. Make sure your brand introduces himself.
The SEO toolkit you know, plus the AI visibility data you need.
@media (maximum width: 768 px) { .headline-responsive { font-size: 30px !important; line-height: 1.3 !important; } }
Hallucination and the law
Attorney Steven Schwartz of the firm Levidow, Levidow & Oberman used ChatGPT to conduct legal research and wrote a brief citing six cases in opposition to a motion to dismiss. The problem ? None of these cases actually existed. ChatGPT completely hallucinated them – fabricating names, quotes and legal opinions.
As if that wasn’t enough, when confronted with evidence that the research might be wrong, the lawyer returned to the exact same AI tool to verify it, blindly accepting the AI’s confirmation as sufficient.
Although the resulting court fine was only $5,000, the case paved the way for strict disclosure of AI use in legal filings. Lawyers must now certify either that generative AI was not used in the preparation of briefs or that all AI-generated content has been verified by a human. As we can see, intentionally integrating a human into the supervisory role is the only way to keep critical thinking current. (There is another case hereif you are interested.)
- The Martech parallel: If a seasoned lawyer can blindly trust a crazy legal brief, it’s awfully easy for a marketer to blindly trust crazy competitive intelligence, flawed personal data, or fabricated performance metrics when crafting a high-stakes campaign.
The black box and denial of health care requests
Imagine finding out that an algorithm, not your doctor, decides how long you stay in rehab after major surgery. This is the heart of an ongoing class-action lawsuit against UnitedHealth Group. The lawsuit claims UnitedHealth used an AI tool called “nH Predict” to automatically deny necessary medical care to elderly Medicare Advantage patients.
Instead of assessing each patient’s needs, the AI used a predetermined generalized schedule to interrupt coverage for essential services like nursing homes and physical therapy. Worse still, the trial highlights a complete lack of significant human intervention. Rather than having medical professionals review cases, UnitedHealth allegedly used AI to systematically ignore the explicit recommendations of patients’ treating doctors.
The consequences are devastating. Patients are forced to leave healthcare facilities before they are fully cured, leading to serious medical setbacks – which, ironically, often cost the company more than the initial treatment. Families who refuse to leave find themselves facing massive, unexpected medical bills. As if all that wasn’t enough, the Senate Permanent Subcommittee on Investigations found that as UnitedHealth aggressively automated its processes, its post-acute care denial rate more than doubled, from 10.9% in 2020 to 22.7% in 2022.
The families suing UnitedHealth are asking the court to force the company to stop using AI for automated medical decisions and to completely overhaul how claims are reviewed. Hopefully, this will not only force the company to reintegrate doctors’ judgment, but also expose how its AI works behind the scenes.
- The Martech parallel: While the stakes in healthcare are life and death, the underlying technical flaw is the same as what we risk in marketing: setting an algorithmic model to aggressively optimize for a single metric (like immediate cost reduction or short-term conversion peaks) while completely ignoring human nuances, long-term brand health, and customer trust.
What is happening to critical thinking?
We’ve seen what happens when people take automated data at face value, but why does it happen so easily?
A global study carried out in 2025 by the University of Melbourne and KPMG interviewed 48,000 people in 47 countries and found that two-thirds of AI users do not evaluate the accuracy of AI results before acting on them. It’s not because they don’t know how to do it; it’s because they choose not to. The output looks clean. He looks confident. This looks like something a competent colleague produced. So, they reached the launch.
To discover what’s happening beneath the surface, Researchers at the MIT Media Lab placed EEG sensors on participants’ heads while they wrote with the help of AI. They found measurable reductions in brain connectivity compared to people who wrote without technological intervention. Although this study is preliminary, they coined a powerful term for what they observed: “cognitive debt.”
The idea is that we borrow mental effort from the future, outsourcing the heavy cognitive work that builds our judgment, creativity, and critical thinking. In doing so, we accrue a mental cost that persists even after the AI application is closed.
Lately, Microsoft Research and Carnegie Mellon questioned 319 knowledge workers and found that the more people trust AI, the less effort they put into critical thinking. Trust in the tool actively suppresses self-confidence. Additionally, researchers have found that relying heavily on generative AI tools produces a much less diverse set of results than independent, human-led thinking.
In modern marketing, where differentiation is your only real defense against market noise, this algorithmic homogenization is a quiet killer. If everyone uses the same prompts and templates, we condemn our brands to a “sea of sameness,” where every blog post, ad headline, and email campaign sounds exactly like the competitor next door.
The Modern Marketer’s Handbook
Rather than succumbing to automation bias and outsourcing your strategy to an algorithm, you can maintain your expertise, edge, and creativity by putting a few fundamental principles into practice:
- Do: Lead AI using your industry expertise, unique brand voice, and professional training.
- Don’t do it: Allow AI to drive strategy. You’ve accumulated years of hard-earned knowledge: use it as an anchor.
- Do: Rigorously pressure test your AI inputs and outputs.
- Don’t do it: Let’s assume the machine is correct. Critically question AI findings by taking a skeptical stance. Looking at data through a critical lens will save your team from a public overhaul and brand embarrassment.
- Do: Construct the first draft, defining your main hypotheses and proof arguments yourself.
- Don’t do it: Give the AI a blank topic and let it do some fundamental thinking for you.
The position Why marketers should oppose AI appeared first on MarTech.
