bug#43566: 27.1; url-http and excorporate does not authenticate

[Thomas Fitzsimmons <fitzsim@fitzsim.org>]

Hi,

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Domingo Gómez Pérez <domingo.gomez@unican.es> writes:
>
>>> Is wget using NTLM authentication, though?  And how have you set up
>>> url.el to use NTLM?
>>
>> Dear Lars Ingebrigtsen,
>>
>> First of all, thanks for your time and help.
>>
>> I would say so, because in the terminal it reads "Autenticación
>> seleccionada: NTLM" (sorry, it is in spanish but I think the meaning is clear).
>
> Thanks for checking.
>
> I'm absolutely completely unfamiliar with NTLM (I assume that's a
> Windows-only thing?), so somebody else will have to try debugging this.

It turns out that this is a regression in Emacs 27.1 introduced by the
fix for bug 27022:

   https://debbugs.gnu.org/27022

The NTLM protocol requires a three step handshake with the server, where
handshake information is passed from the client to the server via
Authorization headers.  I'm not sure if the 27022 scenario would result
in url-http-ntlm inflooping too; if it would, then the obvious
fix-of-the-fix would probably be wrong (i.e., to disable the check when
NTLM is present in the authorization string).

It also turns out that another user found the regression and filed a bug
report with a workaround a month ago:

   https://debbugs.gnu.org/44439

(From now on I'll search for "excorporate" using debbugs-gnu-search
instead of just scanning subject lines.)

I've merged bug#44439 and another report of the same problem, bug#44195,
into this one.

I no longer use the NTLM mode myself, but I did manage to independently
figure out the issue by implementing an NTLM server in elisp.  I'm going
to add it to the test suite, since otherwise it's not feasible for Emacs
maintainers to test ntlm.el properly (which is what happened in Emacs
27.1).  I can test inflooping against it too once I have the changes
ready.

Thomas