For site owners
About the Linkamend checker
If you are here because you saw "Linkamend" in your server log: a WordPress site that links to you uses the Linkamend plugin to find out whether its links still work. The request came from that site's own server, not from us. We do not run a crawler.
What it does
- It asks for one address at a time, first with HEAD, and with GET only when HEAD was refused or failed. A GET is cut off after 16 kB, so a large file is not downloaded. An error page is read up to 1 MB, so that a Retry-After you send is seen.
- It waits between requests to the same site: two seconds at the normal setting, five at the gentle one, one at the fast one.
- When your server answers 429, even to HEAD, it asks nothing more and stays away for as long as the Retry-After header says, or an hour when there is none.
- It checks each link again about once a week, and a link that failed, once more after an hour.
- It sends no cookies, runs no scripts and follows at most five redirects.
- Its User-Agent is
Mozilla/5.0 (compatible; Linkamend/1.0.0; +https://linkamend.tinsmithy.com/checker). It does not pretend to be a browser.
How to stop it
Block the User-Agent, or answer it with 403. Linkamend then files the link under "could not verify" and tells the site's owner that it was refused, not that your page is broken. It does not try to get past a block.
Linkamend does not read robots.txt, because it is not a crawler: it only asks for the exact addresses a site already links to, the way a visitor's click would.