Workshop · back to the section

The website no longer read by people

Some of your visitors are programs. They don't look, they fetch — and what they fetch is often not what the browser shows.

This piece isn't about why it's worth preparing for machine reading — the article on AI visibility covers that. This is about what the machine actually sees, and how to check it.

The most important difference

A browser downloads the HTML, runs the scripts, and displays the result. Many machine readers stop at the first step: they download the HTML and process that.

If the content only assembles at runtime — the server sends an empty shell and a script fills in the text — then for such a reader the page is empty. Not broken, not slow: empty.

What the browser gets and what a program gets The browser downloads the HTML, runs the scripts and shows the final content; a reader that runs no scripts only sees the HTML as sent. server HTML as sent browser runs it what a visitor sees what a program gets whatever only comes into being on the right · does not exist for the reader on the left
  1. 01 server
  2. 02 HTML as sent
  3. 03 browser runs it
  4. 04 what a visitor sees
  5. 05 what a program gets

whatever only comes into being on the right · does not exist for the reader on the left

The check, in one command

No tooling required. Fetch the page the way a program does — with curl, or via view source in the browser, which likewise shows the HTML as sent, not the post-execution state.

Then look in it for whatever you consider important: the service list, the opening hours, the prices, the address. What you can't find there isn't there — at least not for anyone who doesn't run scripts.

The most common things this catches: content inside collapsible elements loaded by script on click; text set inside an image; and whatever sits behind a “load more” control.

Semantics: not a styling question

A machine reader sees no layout. It doesn't know a piece of text is larger and therefore more important — only what element it sits in. A bold, larger paragraph is a heading to the eye and a paragraph to the program.

The same goes for lists, tables and quotations. If a list is made of paragraphs with dashes, a person sees a list and a machine doesn't. This is the oldest advice in HTML, and in the age of machine reading it is more useful than ever.

Structured data: what it's worth, and what it isn't

Structured data is neither magic nor a substitute. It does one thing: it names relationships without ambiguity.

A date in running text could be a publication, an event or an expiry. A name could be an author, a client or a mentioned third party. From prose that follows by inference — in structured data it is stated.

What it doesn't do is repair missing content. If the page doesn't say what you do, then calling it a service in the structured data still leaves nothing to quote. Schema fixes the meaning of what exists.

One rule follows from this, which I've handed to a machine: text in the structured data must match the visible text verbatim. When the two diverge it isn't only a rule violation — it drifts apart unnoticed during maintenance. That's the subject of the piece on the checking script.

The self-contained paragraph

There's an editorial consequence to text being read in extracts. A machine reader rarely quotes a whole chapter — it lifts a paragraph or two, without their surroundings.

Which is why it pays to write so that every paragraph stands on its own. Not “as I wrote above”, but the thing named. Not “this”, but what it points at. That's no loss for human readers either: text using fewer back-references is easier to read.

What can't be hidden

Finally a clarification, because it comes up. There are standard signals for telling a reader what not to use — but those are requests, not barriers, and they bind only cooperating parties.

Anything publicly downloadable should realistically be assumed to be read. The question therefore isn't how to hide something, but what is worth publishing at all — and that is no longer a technical decision.

Questions on this topic

What does a reader that runs no JavaScript see?

Exactly what the server sends: the raw HTML. If the content only assembles in the browser at runtime, then emptiness is what stands in its place. That is the most important difference between what a visitor sees and what a program receives.

How can this be checked simply?

By fetching the page and looking at the raw response — with curl, for instance, or with view-source in the browser, which likewise shows the HTML as sent rather than the post-execution state. What you can't find there, a reader without JavaScript won't find either.

What is structured data for if the text is there anyway?

For making the relationships readable without ambiguity. A date in running text could be a publication, an event or an expiry; in structured data it is named. It doesn't substitute for content, it fixes its meaning.

Can content be hidden from machine readers?

Partly, and only with cooperating readers. There are standard signals for restricting use, but they are requests, not barriers. Anything publicly downloadable should realistically be assumed to be read — the question is rather what is worth publishing at all.

← Back to the Workshop