Every check was green. The site had been broken for eleven days.

What I learned running a news publication that writes and publishes itself, with no human in the editorial loop.

For six months I have been running Ai2India, a daily economic-intelligence publication for India. An AI agent researches it, writes it, builds the HTML, generates the images, publishes to social, and verifies the result. No human writes or reviews an edition before it goes out.

It works. It has also failed in ways I did not anticipate, and the failures have a shape worth describing, because I do not think they are specific to my setup.

The shape is this: every monitoring check I built eventually reported success while the thing it was monitoring was broken. Not once. Five separate times, in five different ways. Each time the check was correct by its own definition and useless in practice.

Here they are, because the specifics are the useful part.

1. The scheduler was green. The edition had one section out of thirteen.

The daily job reported ok. The file existed. The site returned 200.

The edition contained one of its thirteen sections.

The model had ended its turn by announcing the next step — "Now let me create today's edition" — without calling the tool. To the orchestration layer that is a clean completion: stopReason: stop, no error, exit zero. The fallback logic had nothing to fall back from, because nothing had failed.

I had been checking does the job report success. I started checking does the file contain thirteen <section> elements.

2. The section count was green. The page had no image.

Image generation runs through a third-party service whose OAuth session expires about every eight weeks. When it died, the pipeline degraded exactly as designed: it fell back to a placeholder, the edition published, thirteen sections, 200 OK, no error anywhere.

Three editions went out with no lead image before anyone noticed — and only because the layout looked wrong, not because anything told us.

Degrading gracefully is correct behaviour. It is also indistinguishable from working, which is the problem.

3. The social check was green. Nothing had been posted.

The check was:

if [ -f "$BRIEF" ] && [ ! -f "${BRIEF}.posted.json" ]; then
  alert "brief generated but not posted"
fi

Read it carefully. It catches "brief written, not posted". If the brief is never written at all, the first condition is false and the check is silent by construction.

Two editions published complete and reached no one. The check could not have caught it. I had written a guard that tested the failure I imagined rather than the outcome I wanted.

4. The page returned 200. The hero image was unrenderable.

The daily data-visualisation is an SVG. SVG is XML, and browsers refuse a malformed one outright — no partial render, no console error a reader would see. The <img> collapses to a sixteen-pixel sliver with a broken-image icon.

One HTML entity did it. → is fine in HTML and fatal in XML.

When I finally checked, eleven of a hundred editions had been shipping a dead hero image, going back four months. Every one returned 200. Every section count passed.

5. The backup ran six times a day for five months and produced nothing.

This is my favourite, because it is so stupid and so quiet.

H=$(date -u +%H)
[ "$H" = "01" ] && tar czf backups/claw-$TS.tar.gz ...

Cron ran it 0 */4 * * * — hours 00, 04, 08, 12, 16, 20. 01 is not in that set. The gate was never true. The job executed on schedule roughly nine hundred times, exited zero every time, logged nothing, and the backups directory was empty.

Nothing alerted, because nothing was checking the artefact. The schedule was green.

The pattern

Every one of these checks verified that something existed. None verified that it worked.

• The job ran → but produced a fragment
• The sections were counted → but the image was missing
• The brief was posted → but only if it existed
• The page returned 200 → but did not render
• The backup executed → but wrote nothing

The fix in each case was the same move: stop asking the system how it feels, and go look at what it made. Does the file parse. Does the asset load. Is the output newer than yesterday. Did the thing arrive at the other end.

That sounds obvious written down. It was not obvious while building, because at every step the check I wrote was the natural one, and it passed.

Two corollaries I did not expect

Self-reported fields lie, and your logs are full of them.

A raw grep of my access logs showed Googlebot receiving 404s on roughly half its requests — which looks catastrophic. Split by source IP, the genuine crawler (66.249.x) was at 118 × 200, 13 × 301, 5 × 404. Healthy. The rest were vulnerability scanners wearing the Googlebot user-agent while probing for /.env.

The referrer lies too. Forty-four visits claimed https://www.google.com/ as their referrer, which reads as organic search traffic. Their landing pages were /wallet.json, /private_keys.txt, /storage/.env. Filtering probe paths removed 2,616 requests — 22% of all traffic — from every statistic I had been reading.

Anything additive compounds daily.

The pipeline instructs the agent to use the previous edition as its structural baseline. That is sensible and it means any field that grows slightly each run grows without bound.

A kicker label meant to read "Lead Analysis" went 16 words → 36 → 81 → 94 over a fortnight, rendering as an eight-line wall of uppercase monospace above the headline it was duplicating. A section intro on another page reached 1,377 words. Neither was a bug anyone introduced. Each run inherited yesterday's and added a clause.

Prompt wording never held that line. Mechanical passes that trim the field after generation did.

What I would tell someone starting this

Write the check against the artefact, not the status. If your monitoring can only tell you a process ran, it will eventually tell you that while the output is garbage, and you will believe it, because it has been right every day until then.

And budget for the failures being invisible rather than loud. Not one of these five announced itself. Every single one was found by a human looking at the thing and going that's odd.

Ai2India publishes daily at ai2india.com. It states on its About page that it is produced by an automated system, because labelling the images and not the writing seemed like the wrong place to draw that line.