Re: Auto-checking dead links in the manual (was: http: links in the manual)

[Max Nikulin <manikulin@gmail.com>]

On 22/08/2022 09:46, Ihor Radchenko wrote:
> Juan Manuel Macías writes:
> 
>> Maybe, instead of repairing the links manually, we could think of some
>> code that would do this work periodically, and also check the health of
>> the links, running a url request on each link and returning a list of
>> broken links. I don't know if it is possible to do something like that
>> in Elisp, as I don't have much experience with web and link issues. I
>> think there are also external tools, like Selenium Web Driver, but my
>> experience with it is very limited (I use Selenium from time to time
>> when I want to take a screenshot of a web page).
> 
> This is a good idea.
> 
> Selenium is probably an overkill since we should better not link JS-only
> websites from the manual anyway. What we can do instead is a make target
> that will use something like wget.
> 
> Patches are welcome!

I hope that selenium is currently overkill, however more sites are 
starting to use anti-DDOS shields like cloudflare and HTTP client may be 
banned just because it does not fetch other resources like JS scripts.

I do not have a patch, just an idea: export backend that ignores 
everything besides link and either send requests from lisp code or 
generate file for another tool.

#+attr_linklint: ...

may be used to specify regexp that target page is expected to contain. 
There are some complications like e.g. "info:" links having special code 
to generate HTML with URL derived from original path. So it may be more 
robust to parse HTML document (without checking of linked document text).