Async DNS redux

Async DNS redux

[Lars Ingebrigtsen]

I was reading a blog just now, and apparently CloudFlare has a hickup:

larsi@rusty:~/mgnus$ time host www.thisiscolossal.com
www.thisiscolossal.com is an alias for www.thisiscolossal.com.cdn.cloudflare.net.
www.thisiscolossal.com.cdn.cloudflare.net is an alias for cf-ssl4799-protected-www.thisiscolossal.com.cdn.cloudflare.net.
cf-ssl4799-protected-www.thisiscolossal.com.cdn.cloudflare.net has address 141.101.125.141
cf-ssl4799-protected-www.thisiscolossal.com.cdn.cloudflare.net has address 141.101.126.141
;; connection timed out; no servers could be reached
;; connection timed out; no servers could be reached

real	0m20.134s
user	0m0.012s
sys	0m0.000s

This made using Emacs impossible.  The resolving is totally synchronous,
and there were many pictures from this host.  Every one halting Emacs
for 20 seconds.

I'm going to try to fix up url-retrieve-queue somewhat -- if resolving
failed for host foo, then just remove all entries from the queue that
are trying to do the same.  Hm.  But since it fetches a lot of stuff in
parallel, that may not be trivial.  Uhm...

Anyway, what I wanted to ask was whether anybody has had any further
ideas on async DNS resolving in Emacs?  For stuff like this to work in a
satisfactory manner, we really need a kind of
(dns-resolve "foo.bar" 'callback) call in Emacs.

Does gnulib or anything have anything handy here?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: Async DNS redux

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I'm going to try to fix up url-retrieve-queue somewhat -- if resolving
> failed for host foo, then just remove all entries from the queue that
> are trying to do the same.

I've now committed that fix, and is helps for the use case I'm seeing.
But it's achieved by postponing all the fetches a bit, which may or may
not be a good thing.  At the moment, it looks OK to me, but feedback on
the usability would be nice.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: Async DNS redux

[Leo]

On 2012-02-06 08:32 +0800, Lars Ingebrigtsen wrote:
> Anyway, what I wanted to ask was whether anybody has had any further
> ideas on async DNS resolving in Emacs?  For stuff like this to work in a
> satisfactory manner, we really need a kind of
> (dns-resolve "foo.bar" 'callback) call in Emacs.

As a user of fragile unreliable slow network, I think that would make a
huge difference.

Leo

Re: Async DNS redux

[Stefan Monnier]

> Anyway, what I wanted to ask was whether anybody has had any further
> ideas on async DNS resolving in Emacs?  For stuff like this to work in a
> satisfactory manner, we really need a kind of
> (dns-resolve "foo.bar" 'callback) call in Emacs.

Can you explain why you think we need to export the async interface
to Elisp?


        Stefan

Re: Async DNS redux

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Can you explain why you think we need to export the async interface
> to Elisp?

When doing async URL retrievals, like

(url-retrieve "http://fsf.org" #'our-callback)

To implement this, `url-retrieve' needs to do the resolving
asynchronously, so it would first decompose the URL, then call
(dns-resolve "fsf.org" #'continue-doing-the-retrieval), and then contact
the host, before finally calling `our-callback'.

I don't quite see how to get async chain going without exposing it to
Elisp...

An alternative would be to make `make-network-process' fully
asynchronous, and do the DNS resolving in an asynch manner there.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: Async DNS redux

[Stefan Monnier]

> An alternative would be to make `make-network-process' fully
> asynchronous, and do the DNS resolving in an asynch manner there.

That seems like the more direct solution.

Providing a proof-of-concept async Elisp-level resolution isn't hard if
you use something like (start-process "dns-resolve" nil "host"
"the.name"), but I think that fixing it at the C code level in
make-network-process makes a lot more sense.


        Stefan

Re: Async DNS redux

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Providing a proof-of-concept async Elisp-level resolution isn't hard if
> you use something like (start-process "dns-resolve" nil "host"
> "the.name"), but I think that fixing it at the C code level in
> make-network-process makes a lot more sense.

If :nowait would result in `make-network-process' returning
immediately -- even before doing the DNS resolving -- then that would be
ideal, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Rome

Re: Async DNS redux

[Stefan Monnier]

>> Providing a proof-of-concept async Elisp-level resolution isn't hard if
>> you use something like (start-process "dns-resolve" nil "host"
>> "the.name"), but I think that fixing it at the C code level in
>> make-network-process makes a lot more sense.
> If :nowait would result in `make-network-process' returning
> immediately -- even before doing the DNS resolving -- then that would be
> ideal, I think.

Exactly,


        Stefan