Almost every agency now calls itself “AI-powered.” Some of that is real: AI tools genuinely speed up parts of building a website. Much of it is a new label on the same work. This guide covers what AI changes in practice, where it falls short, and the questions that tell you which kind of pitch you’re looking at.

The tools change quickly, so this is about kinds of work rather than specific products or benchmarks.

What AI actually changed

Writing code

Developers can now describe what they want, get a starting point from an AI assistant, then refine and test it. On boilerplate (forms, standard components, configuration, tests for well-understood functions) that saves real time.

It doesn’t replace thinking about architecture, understand your business, or make a weak developer strong. Generated code still needs review by someone who could have written it, because it is often plausible and wrong in ways that only show up later.

Content

AI is useful for outlines, first drafts, tone rewrites and variations. It is weak at the parts that make content worth reading: first-hand expertise, your specific stories and a voice that sounds like you.

Compare two openings for an About page:

We are a leading digital agency providing innovative solutions to help businesses succeed in the digital landscape.

We look after the Totallyamaha forums, and one of us has administered one of them since 2013: three forums, with threads from 2003 still online.

The first could be anyone. The second could only be one company, because it is built on facts only that company has. AI can polish the second sentence. It can’t supply the facts.

Search engines draw the same line. Google’s spam policies treat mass-producing pages to manipulate rankings as scaled content abuse, however the pages are made. One of their examples is using AI tools to generate many pages without adding value for users. The problem is pages that add nothing, not the tool that wrote them.

Design

AI can suggest layouts, generate palettes, produce placeholder imagery and speed up asset production. It can’t decide what your brand should stand for, who it is for, or how a system of pages should work together. It speeds up execution; it doesn’t replace design judgment.

Testing and QA

Automated checks were useful before AI and are more useful with it: missing alt text, contrast failures, broken links, absent security headers, skipped heading levels. AI helps write test cases and explain failures. None of it replaces testing with real people, and automated accessibility checks catch only part of what WCAG covers. Our accessibility guide explains what still needs a human.

Decoding “AI-powered” claims

“AI-designed website”

Often this means a template generator. Ask what the AI decided and what a person decided. If the result looks like every other site, you already have your answer.

“AI-optimized SEO”

Often this means content run through a language model to work in keywords. Rankings depend on usefulness, expertise and a technically sound site, and a model has no expertise of its own about your business. Pages produced at scale with nothing added are exactly what Google’s spam policies target.

“AI chatbot integration”

A generic chat widget that can’t answer “What are your prices?” loses the visitor. A useful assistant has to be grounded in your actual content and data, with limits on what it will answer, and someone has to maintain it. That is real AI integration work, not a widget, and it is worth paying for only when it’s done properly.

“AI-powered performance”

Ask how, exactly. Speed comes from code quality, page weight and infrastructure. A slow site with an AI label is still a slow site.

“Guaranteed top rankings”

Treat any ranking guarantee as a red flag, with or without AI. Nobody controls Google’s results, and an agency that promises a position is either misinformed or misleading you.

“Our proprietary AI model”

Ask what it was trained on and how its performance is measured. Many “proprietary” tools are a commercial model behind a custom interface. That is fine if it is described honestly and a problem if it is not.

Where AI genuinely helps

In a well-run project, AI earns its place in a handful of jobs:

  • Explaining and reviewing code. Asking what an unfamiliar function does, or for a second look at a change, catches mistakes earlier. A person still decides what ships.
  • Documentation. Turning working notes into readable docs, checked for accuracy by the people who know the system.
  • Asset processing. Background removal, upscaling and batch conversion to modern image formats.
  • Test coverage. Drafting test cases, including edge cases people tend to miss.
  • Accessibility pre-checks. Suggesting alt text for a person to edit, and flagging heading and labeling problems.

The pattern is the same every time: AI produces a draft or a flag, and a person who understands the context accepts, corrects or rejects it. On our projects, every client-facing word is written or approved by a person.

A realistic workflow

  1. People work out the business requirements.
  2. People design the information architecture.
  3. People decide the visual design; AI can suggest options.
  4. People and AI write the code: AI drafts boilerplate, people own the architecture and the review.
  5. People and AI produce content: AI can help draft, people supply the substance and edit.
  6. Automated tools, AI-assisted or not, run the checks.
  7. People test the experience with real users.
  8. People deploy and monitor, with tooling to help.

The time AI saves sits mostly in steps 4 to 6. Strategy, structure and judgment take as long as they always did, and they are where most of the value is.

Questions to ask an agency about AI

  1. Which tools do you use, and for what? A good answer names tools and tasks. “Advanced AI technology” is not an answer.
  2. How does AI change my project specifically? A good answer ties AI to particular tasks, such as drafting product descriptions that a copywriter then edits, and says what that saves you.
  3. What do people still do? A good answer: strategy, architecture, design decisions, review and accountability.
  4. Who checks the AI’s output, and how? A vague answer here is the biggest red flag.
  5. Where won’t AI help on my project? An honest agency has an answer: compliance requirements, unusual integrations, anything that depends on context a model doesn’t have.
  6. What happens to my data? Ask whether your content, code or customer data goes to third-party AI services, under what terms, and whether you can opt out.

If the answers are vague, ignore the AI claims and judge the agency on its record: relevant work, the quality of what it has shipped, and how well it understands your business. Our guide to vetting a web agency covers the rest of that check, and we look at the bigger question in whether AI agents will replace agencies.

What’s likely to change

Code generation, design suggestions, automated testing and drafting will keep improving, probably quickly. What won’t change is the need for someone to decide what to build, understand the people who will use it, and answer for the result. Predictions about exactly when AI will do a senior developer’s job have dated badly, so plan around what the tools can do today and revisit the plan as they change.

The bottom line

AI is a tool. A good hammer doesn’t make a good carpenter, and a good model doesn’t make a weak agency strong. AI’s best use is mostly invisible: experienced people working faster, with more checks, on the work that matters.

A simple test for any pitch: strike “AI-powered” from it. If you would still hire the agency, AI is a bonus. If you wouldn’t, you were being sold a label.