0 of 40 HN links died in 13 months. That's the wrong sample

2026-11-03来源 dev.to DiscussEnglish

Zero of 40 Hacker News links from October 2025 are dead today, 394 days later — and 38 of the 40 have Wayback Machine snapshots. That sounds like the web archives itself. It doesn't; my sample is the most durable slice of the web that exists.

The prompt for this measurement is a post that hit HN's front page this week, "If you love a website, back it up" (Misty De Meo, October 29, 2026). Her argument is that personal web archiving is not just for institutions: individuals should save the sites they care about, in WARC/WACZ format, with tools like ArchiveWeb.page and Browsertrix, instead of hoping someone else does it. Her motivating example is a fansite for the Ochaken anime toy line that had been online since 2004, unchanged since 2005, and was gone when she went back — its domain now serving gambling ads. She could have archived it. She didn't.

So I tested the optimistic version of her claim: how fast does the web actually rot? I took 40 outbound links from HN stories posted on October 4, 2025 — a full front-page day, excluding HN-internal links — and re-fetched every one today, November 2, 2026, with curl following redirects and a browser user agent. Then I asked the Wayback Machine's availability API whether each URL has a snapshot.

The numbers

sample: 40 outbound links, HN stories from 2025-10-04 checked: 2026-11-02 (394 days later) alive, HTTP 200: 37 (36 full GETs + 1 HEAD; see note) bot/paywall wall: 3 (2x 403, 1x 401 — the pages exist, they block bots) confirmed dead: 0 Wayback snapshot: 38 of 40 (95.0%) 

The three walled responses are worth naming, because a lazy checker would count them as dead: McMaster University's health sciences news page (403), The Economist (403), and the Wall Street Journal (401). All three are bot or paywall walls, not missing pages. The one ambiguous fetch, a PagedOut PDF, failed on body download in my sandbox but returned a clean HEAD 200 with a Last-Modified of October 4, 2025 — the file is intact; my download pipeline is what failed. I count it alive and say so, because the alternative is counting my own tooling bug as link rot.

The two URLs with no Wayback snapshot are both still live: the McMaster article above and a Perspectives piece on Microsoft 365 Copilot. Archive absence and page death are different things, and in this sample they never coincide.

Why zero dead is the wrong lesson

If you stopped here, the conclusion would be "don't bother archiving; the web keeps itself." That conclusion is a property of my sample, not of the web. A link that reached HN's front page has already survived a brutal filter: it was posted, upvoted, discussed, linked from newsletters, and crawled by every archiving bot within hours. The Wayback timestamps in my results prove it — most snapshots are from October 2026, meaning the archive re-crawls these pages constantly. Popular pages are archived redundantly and kept alive redundantly.

De Meo's Ochaken fansite is the opposite shape: a single-author site, unchanged since 2005, no inbound links from anywhere that matters, never on a front page. Nothing in my measurement says anything about that class of site, because that class never enters my sample. That is exactly the selection effect her post is warning about — the sites you personally love are, almost by definition, not the sites the whole internet already saved.

The honest version of my result is narrower and still useful: for anything popular enough to hit HN, link rot over 13 months is ~0% and archival coverage is 95%. The residual 5% — two live pages with no snapshot — is the crack that matters. A page can be fully alive today and one domain-expiry away from being an Ochaken story tomorrow, and "the archive will have it" is only true for pages the archive noticed.

So the measurement ends up agreeing with the post it started from, just through the back door: archive the small, weird, personally important sites yourself, because the 0% death rate I measured is what happens to pages everyone already cares about. The pages only you care about are not in that denominator. ArchiveWeb.page will save a single page in a browser session; Browsertrix will crawl a whole small site. Either one produces a WACZ file you can keep and share, and that file is the only copy whose existence you control.

Sources: "If you love a website, back it up," Misty De Meo, October 29, 2026 — https://blog.mistys-internet.website/post/2026/10/29/if-you-love-a-website-back-it-up/ ; link sample via the Hacker News Algolia API (stories created 2025-10-04); live checks and Wayback availability checks run by me on November 2, 2026. All counts above are from my own runs; the Ochaken example and tool descriptions are as reported in De Meo's post.

原文:https://www.a8q9.cn/en/article/0-of-40-hn-links-died-in-13-months-that-s-the-wrong-sample-932016.html

相关文章