Asynchronously downloading a file

Asynchronously downloading a file

[Deniz Dogan]

I'm currently using find-file to view images fetched over HTTP.
However, sometimes the server hosting the image is kind of slow and
thus Emacs just hangs waiting for the image to fully download before I
can view it.

Is there any existing Emacs functionality which asynchronously gets a
file from any of the protocols that e.g. find-file (tramp?) supports
which lets me provide a callback function?

Thanks,
Deniz Dogan

Re: Asynchronously downloading a file

[Qiang Guo]

Emacs adheres to single thread fervently, so it may be
impossible to do this within emacs. 

Re: Asynchronously downloading a file

[Deniz Dogan]

2010/7/2 Qiang Guo <mcknight0219@gmail.com>:
>
> Emacs adheres to single thread fervently, so it may be
> impossible to do this within emacs.
>

I was not necessarily looking for threading, but just spawning a
process, e.g. wget, which gets the file for me. :)

-- 
Deniz Dogan

Re: Asynchronously downloading a file

[Daniel Pittman]

Qiang Guo <mcknight0219@gmail.com> writes:

> Emacs adheres to single thread fervently, so it may be impossible to do this
> within emacs.

That isn't actually all that true: while there is only a single thread of
execution within Emacs you can address this in one of two ways:

The obvious, and sane, way is to take advantage of the event-driven I/O
functionality present in Emacs as part of the process / network-connection
filter infrastructure.

In this mode you tell the system to start the download, then your function is
called (or data collected in a buffer) as it is available to be read.  While
there is only a single thread, this will jump over to handling your download
transparently to doing other activities.


The second way is to use an external process to do the same; you can even
invoke 'emacs --batch --eval' if you really want to. :)

        Daniel

The second way is kind of wasteful given that `open-network-stream' is
actually entirely asynchronous and all.
-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

Re: Asynchronously downloading a file

[Deniz Dogan]

2010/7/2 Daniel Pittman <daniel@rimspace.net>:
> Qiang Guo <mcknight0219@gmail.com> writes:
>
>> Emacs adheres to single thread fervently, so it may be impossible to do this
>> within emacs.
>
> [snip]
>
> The second way is to use an external process to do the same; you can even
> invoke 'emacs --batch --eval' if you really want to. :)
>
>        Daniel
>
> The second way is kind of wasteful given that `open-network-stream' is
> actually entirely asynchronous and all.
>

I'm looking for something like that, but with some layer taking care
of the HTTP stuff for me. I am specifically looking for something
which lets me make a GET or POST request and easily extract the
response body. Error handling such as 404, 500, etc. would be nice as
well.

-- 
Deniz Dogan

Re: Asynchronously downloading a file

[Daniel Pittman]

Deniz Dogan <deniz.a.m.dogan@gmail.com> writes:
> 2010/7/2 Daniel Pittman <daniel@rimspace.net>:
>> Qiang Guo <mcknight0219@gmail.com> writes:
>>
>>> Emacs adheres to single thread fervently, so it may be impossible to do this
>>> within emacs.
>>
>> [snip]
>>
>> The second way is to use an external process to do the same; you can even
>> invoke 'emacs --batch --eval' if you really want to. :)
>>
>>        Daniel
>>
>> The second way is kind of wasteful given that `open-network-stream' is
>> actually entirely asynchronous and all.
>
> I'm looking for something like that, but with some layer taking care of the
> HTTP stuff for me.

Oh.  You mean `url-retrieve'.
        Daniel

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

Re: Asynchronously downloading a file

[Deniz Dogan]

2010/7/2 Daniel Pittman <daniel@rimspace.net>:
> Deniz Dogan <deniz.a.m.dogan@gmail.com> writes:
>> 2010/7/2 Daniel Pittman <daniel@rimspace.net>:
>>> Qiang Guo <mcknight0219@gmail.com> writes:
>>>
>>>> Emacs adheres to single thread fervently, so it may be impossible to do this
>>>> within emacs.
>>>
>>> [snip]
>>>
>>> The second way is to use an external process to do the same; you can even
>>> invoke 'emacs --batch --eval' if you really want to. :)
>>>
>>>        Daniel
>>>
>>> The second way is kind of wasteful given that `open-network-stream' is
>>> actually entirely asynchronous and all.
>>
>> I'm looking for something like that, but with some layer taking care of the
>> HTTP stuff for me.
>
> Oh.  You mean `url-retrieve'.
>

That looks very interesting. Thank you.

-- 
Deniz Dogan

Re: Asynchronously downloading a file

[Lennart Borgman]

On Fri, Jul 2, 2010 at 12:11 PM, Deniz Dogan <deniz.a.m.dogan@gmail.com> wrote:
>>
>> Oh.  You mean `url-retrieve'.
>>
>
> That looks very interesting. Thank you.

There is code using this in pause.el in nXhtml if you want an example.

Re: Asynchronously downloading a file

[David Engster]

Deniz Dogan writes:
> I'm currently using find-file to view images fetched over HTTP.
> However, sometimes the server hosting the image is kind of slow and
> thus Emacs just hangs waiting for the image to fully download before I
> can view it.
>
> Is there any existing Emacs functionality which asynchronously gets a
> file from any of the protocols that e.g. find-file (tramp?) supports
> which lets me provide a callback function?

I don't think there's something in Emacs proper which does that right
away.

At least for http, you could use emacs-w3m, which support asynchronous
retrieval of URLs:

(w3m-retrieve URL &optional NO-UNCOMPRESS NO-CACHE POST-DATA REFERER
HANDLER)

Retrieve web contents pointed to by URL.
It will put the retrieved contents into the current buffer.
[...]
If HANDLER is a function, this function will come to an end in no time.
In this case, contents will be retrieved by the asynchronous process
after a while.  And after finishing retrieving contents successfully,
HANDLER will be called on the buffer where this function starts.
[...]

Regards,
David

Re: Asynchronously downloading a file

[Alberto Luaces]

Deniz Dogan writes:

> Is there any existing Emacs functionality which asynchronously gets a
> file from any of the protocols that e.g. find-file (tramp?) supports
> which lets me provide a callback function?

Asynchronously, you could take a look at try wget-el. I'm not sure about
the callback part, though.

-- 
Alberto

Re: Asynchronously downloading a file

[Andreas Politz]

Alberto Luaces <aluaces@udc.es> writes:

> Deniz Dogan writes:
>
>> Is there any existing Emacs functionality which asynchronously gets a
>> file from any of the protocols that e.g. find-file (tramp?) supports
>> which lets me provide a callback function?
>
> Asynchronously, you could take a look at try wget-el. I'm not sure about
> the callback part, though.

There is also `url-retrieve'.

-ap