From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Richard M. Stallman" Newsgroups: gmane.emacs.devel Subject: [happy@mcplaksin.org: HTTP redirects make url-retrieve-synchronously asynchronous] Date: Tue, 10 Jan 2006 10:21:32 -0500 Message-ID: Reply-To: rms@gnu.org NNTP-Posting-Host: main.gmane.org Content-Type: text/plain; charset=ISO-8859-15 X-Trace: sea.gmane.org 1136914015 14383 80.91.229.2 (10 Jan 2006 17:26:55 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Tue, 10 Jan 2006 17:26:55 +0000 (UTC) Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jan 10 18:26:52 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1EwNFU-0004zu-Lu for ged-emacs-devel@m.gmane.org; Tue, 10 Jan 2006 18:25:33 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1EwNHT-000612-UG for ged-emacs-devel@m.gmane.org; Tue, 10 Jan 2006 12:27:36 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1EwLNh-0002p6-10 for emacs-devel@gnu.org; Tue, 10 Jan 2006 10:25:53 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1EwLNd-0002ni-PH for emacs-devel@gnu.org; Tue, 10 Jan 2006 10:25:50 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1EwLNY-0002mm-6K for emacs-devel@gnu.org; Tue, 10 Jan 2006 10:25:44 -0500 Original-Received: from [199.232.76.164] (helo=fencepost.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1EwLQ1-0007St-G4 for emacs-devel@gnu.org; Tue, 10 Jan 2006 10:28:17 -0500 Original-Received: from rms by fencepost.gnu.org with local (Exim 4.34) id 1EwLJU-0005JH-21; Tue, 10 Jan 2006 10:21:32 -0500 Original-To: emacs-devel@gnu.org X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:48905 Archived-At: Would someone please look at this, and then ack? ------- Start of forwarded message ------- X-Injected-Via-Gmane: http://gmane.org/ To: emacs-pretest-bug@gnu.org From: Mark Plaksin Date: Mon, 09 Jan 2006 13:18:46 -0500 X-Gmane-NNTP-Posting-Host: stone.tss.usg.edu Cancel-Lock: sha1:UaHMftiRGaDChvZKQsoPebVNGFs= Subject: HTTP redirects make url-retrieve-synchronously asynchronous Sender: emacs-pretest-bug-bounces+rms=gnu.org@gnu.org X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on monty-python X-Spam-Level: X-Spam-Status: No, hits=0.0 required=5.0 tests=HTML_MESSAGE autolearn=no version=2.63 url-retrieve-synchronously becomes asynchronous when HTTP redirects are involved. When it encounters a redirect, url-http-parse-headers calls url-retrieve instead of url-retrieve-synchronously. Naively switching to the latter doesn't solve the problem and I haven't been able to find a fix.. I encountered the problem in Gnus using nnrss. I have an old URL for Slashdot's RSS feed and was experimenting with setting mm-url-use-external to nil. To reproduce, evaluate this in *scratch*: (mm-url-insert "http://slashdot.org/index.rss") You'll get this: ("http://slashdot.org/index.rss" 316) 301 Moved Permanently

Moved Permanently

The document has moved here.

Apache/1.3.33 Server at slashdot.org Port 80

Moved Permanently

The document has moved here.

Apache/1.3.33 Server at slashdot.org Port 80

To make the problem go away, add a breakpoint before "(when redirect-uri" in url-http.el. Then re-run the test and wait a few seconds after hitting the breakpoint. Tell the debugger to continue and you will get the expected contents of Slashdot's RSS feed instead of the redirect message above. I'll keep trying to find a way to fix this but maybe it's trivial for somebody who already understands the URL library. _______________________________________________ emacs-pretest-bug mailing list emacs-pretest-bug@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-pretest-bug ------- End of forwarded message ------- From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Tue, 17 Jan 2006 22:13:11 -0500 Message-ID: <87y81ep6wf.fsf-monnier+emacs@gnu.org> References: NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1137554024 2815 80.91.229.2 (18 Jan 2006 03:13:44 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Wed, 18 Jan 2006 03:13:44 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Jan 18 04:13:39 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1Ez3lD-0005mK-MB for ged-emacs-devel@m.gmane.org; Wed, 18 Jan 2006 04:13:24 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ez3na-0001tr-BQ for ged-emacs-devel@m.gmane.org; Tue, 17 Jan 2006 22:15:50 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Ez3nT-0001tZ-83 for emacs-devel@gnu.org; Tue, 17 Jan 2006 22:15:43 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Ez3nR-0001tC-DS for emacs-devel@gnu.org; Tue, 17 Jan 2006 22:15:42 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ez3nR-0001t9-67 for emacs-devel@gnu.org; Tue, 17 Jan 2006 22:15:41 -0500 Original-Received: from [209.226.175.93] (helo=tomts36-srv.bellnexxia.net) by monty-python.gnu.org with esmtp (Exim 4.34) id 1Ez3rD-0000jY-Cs; Tue, 17 Jan 2006 22:19:35 -0500 Original-Received: from alfajor ([67.71.26.73]) by tomts36-srv.bellnexxia.net (InterMail vM.5.01.06.13 201-253-122-130-113-20050324) with ESMTP id <20060118031312.MFGF5032.tomts36-srv.bellnexxia.net@alfajor>; Tue, 17 Jan 2006 22:13:12 -0500 Original-Received: by alfajor (Postfix, from userid 1000) id C7818D7377; Tue, 17 Jan 2006 22:13:11 -0500 (EST) Original-To: rms@gnu.org In-Reply-To: (Richard Stallman's message of "Tue, 17 Jan 2006 14:59:54 -0500") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (gnu/linux) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49228 Archived-At: > [I sent this message a week ago but did not get a response.] > Would someone please look at this, and then ack? I've looked at it and understand the problem, but I can't think of a quickfix, and the real fix will take some work (and enough changes that it'll risk introducing bugs that'll need more testing than I can do myself). Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Richard M. Stallman" Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Thu, 19 Jan 2006 12:44:18 -0500 Message-ID: References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> Reply-To: rms@gnu.org NNTP-Posting-Host: main.gmane.org Content-Type: text/plain; charset=ISO-8859-15 X-Trace: sea.gmane.org 1137702305 3238 80.91.229.2 (19 Jan 2006 20:25:05 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Thu, 19 Jan 2006 20:25:05 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jan 19 21:25:04 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1EzgKG-0000nN-CA for ged-emacs-devel@m.gmane.org; Thu, 19 Jan 2006 21:24:08 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1EzgMi-0001Oq-Dl for ged-emacs-devel@m.gmane.org; Thu, 19 Jan 2006 15:26:40 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Ezdv7-00065D-Fn for emacs-devel@gnu.org; Thu, 19 Jan 2006 12:50:01 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Ezdv5-00064K-NP for emacs-devel@gnu.org; Thu, 19 Jan 2006 12:50:00 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ezdv5-000646-8I for emacs-devel@gnu.org; Thu, 19 Jan 2006 12:49:59 -0500 Original-Received: from [199.232.76.164] (helo=fencepost.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1Ezdz9-000251-2D for emacs-devel@gnu.org; Thu, 19 Jan 2006 12:54:11 -0500 Original-Received: from rms by fencepost.gnu.org with local (Exim 4.34) id 1Ezdpa-0008Fg-J7; Thu, 19 Jan 2006 12:44:18 -0500 Original-To: Stefan Monnier In-reply-to: <87y81ep6wf.fsf-monnier+emacs@gnu.org> (message from Stefan Monnier on Tue, 17 Jan 2006 22:13:11 -0500) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49263 Archived-At: I've looked at it and understand the problem, but I can't think of a quickfix, and the real fix will take some work (and enough changes that it'll risk introducing bugs that'll need more testing than I can do myself). Could you explain the issue to us? Maybe someone will have an idea. From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Fri, 20 Jan 2006 16:56:55 -0500 Message-ID: <87wtgu8trl.fsf-monnier+emacs@gnu.org> References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1137800189 6836 80.91.229.2 (20 Jan 2006 23:36:29 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Fri, 20 Jan 2006 23:36:29 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Jan 21 00:36:29 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1F05nq-0006fQ-7b for ged-emacs-devel@m.gmane.org; Sat, 21 Jan 2006 00:36:22 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F05qL-0003MY-PK for ged-emacs-devel@m.gmane.org; Fri, 20 Jan 2006 18:38:57 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1F04Ig-0004Aj-NV for emacs-devel@gnu.org; Fri, 20 Jan 2006 17:00:06 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1F04If-00049V-9r for emacs-devel@gnu.org; Fri, 20 Jan 2006 17:00:05 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F04Ie-00048h-AH for emacs-devel@gnu.org; Fri, 20 Jan 2006 17:00:04 -0500 Original-Received: from [209.226.175.34] (helo=tomts13-srv.bellnexxia.net) by monty-python.gnu.org with esmtp (Exim 4.34) id 1F04Mg-0006Xg-ID; Fri, 20 Jan 2006 17:04:28 -0500 Original-Received: from alfajor ([67.71.26.73]) by tomts13-srv.bellnexxia.net (InterMail vM.5.01.06.13 201-253-122-130-113-20050324) with ESMTP id <20060120215659.RBSX20927.tomts13-srv.bellnexxia.net@alfajor>; Fri, 20 Jan 2006 16:56:59 -0500 Original-Received: by alfajor (Postfix, from userid 1000) id DFE47D73B3; Fri, 20 Jan 2006 16:56:55 -0500 (EST) Original-To: rms@gnu.org In-Reply-To: (Richard M. Stallman's message of "Thu, 19 Jan 2006 12:44:18 -0500") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (gnu/linux) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49334 Archived-At: > I've looked at it and understand the problem, but I can't think of a > quickfix, and the real fix will take some work (and enough changes that > it'll risk introducing bugs that'll need more testing than I can do > myself). > Could you explain the issue to us? Maybe someone will have an idea. Here is what I believe to be the relevant information: 1. when an http server returns a redirect, url-http.el currently calls url-retrieve to retrieve the target of the redirect (i.e. transparently follow the redirection). 2. url-retrieve takes a url and a callack function and returns a new buffer into which the requested "page" will be asynchronously inserted. When the buffer is complete, it calls the callback function (in that buffer). 3. some backends (at least url-http, maybe others) sometimes decide not to call the callback, presumably as a way to signal an error (the operation can't be completed so the callback can't be called, basically). This is a bug, but I don't know of anyone who's tried to tackle it yet. 4. url-retrieve-synchronously is implemented on top of url-retrieve by busy-looping with accept-process-input waiting for a variable to be set to t by the callback function. Now, because of number 3 above url-retrieve-synchronously can't assume that the callback will eventually be called, so it also stops the busy-waiting if it notices that there's no more process running in the buffer that url-retrive returned. So when a redirect is encountered, the inner call to url-retrieve creates a brand new buffer, different from the one returned by the original url-retrieve call, and the subsequent async process runs in that buffer and the callback will be called in *that* buffer as well. So url-retrieve-synchronously gets all confused: in the buffer in which it expects the output, after the redirect there's no more process running (it's running the newly generated buffer), so it stops busy-waiting and returns, thinking the download has completed whereas it's actually still going on, just in another buffer. So there are fundamentally two bugs in the code: 1. sometimes the callback doesn't get called. 2. sometimes the callback gets called in another buffer than the one returned by url-retrieve. I think the first is due to a bug in the API because there's no documented way for a backend to indicate to the callback function that the operation failed. And the second is an internal bug, but which I think is due to a bug in the internal API (the one used between the generic part of the URL lib and each backend) where the url- function necessarily returns a new buffer. They (and url-retrieve) should either take an optional destination buffer as parameter or they should simply not return any buffer at all and the destination buffer should only be made known when the callback is called. This second option is simpler but would cause problems for code that wants to start processing data before it's fully downloaded. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Richard M. Stallman" Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Sat, 21 Jan 2006 22:59:07 -0500 Message-ID: References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> <87wtgu8trl.fsf-monnier+emacs@gnu.org> Reply-To: rms@gnu.org NNTP-Posting-Host: main.gmane.org Content-Type: text/plain; charset=ISO-8859-15 X-Trace: sea.gmane.org 1137902686 18698 80.91.229.2 (22 Jan 2006 04:04:46 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 22 Jan 2006 04:04:46 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Jan 22 05:04:45 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1F0WT0-0006hI-5F for ged-emacs-devel@m.gmane.org; Sun, 22 Jan 2006 05:04:38 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F0WVZ-0002eQ-Gh for ged-emacs-devel@m.gmane.org; Sat, 21 Jan 2006 23:07:17 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1F0WTa-00011p-Bx for emacs-devel@gnu.org; Sat, 21 Jan 2006 23:05:14 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1F0WTY-00010N-Ot for emacs-devel@gnu.org; Sat, 21 Jan 2006 23:05:13 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F0WTX-0000zz-DI for emacs-devel@gnu.org; Sat, 21 Jan 2006 23:05:11 -0500 Original-Received: from [199.232.76.164] (helo=fencepost.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1F0WY2-0005Mz-5G for emacs-devel@gnu.org; Sat, 21 Jan 2006 23:09:50 -0500 Original-Received: from rms by fencepost.gnu.org with local (Exim 4.34) id 1F0WNf-0006gY-NW; Sat, 21 Jan 2006 22:59:07 -0500 Original-To: Stefan Monnier In-reply-to: <87wtgu8trl.fsf-monnier+emacs@gnu.org> (message from Stefan Monnier on Fri, 20 Jan 2006 16:56:55 -0500) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49387 Archived-At: 3. some backends (at least url-http, maybe others) sometimes decide not to call the callback, presumably as a way to signal an error (the operation can't be completed so the callback can't be called, basically). This is a bug, but I don't know of anyone who's tried to tackle it yet. What exactly is the bug? It isn't clear to me. Are you saying that they should also call the callback function to report failure? That seems like a good idea on general principles. I don't know how much of a change it would be. Is the calling convention easy to extend for this? 2. sometimes the callback gets called in another buffer than the one returned by url-retrieve. One solution would be to give the first buffer a local variable that would, in this case, point to the second buffer. Then url-retrieve-synchronously could check the local variable, which would tell it to check the process in the other buffer. That solution would be safe, since it would not involve changing general calling conventions. They (and url-retrieve) should either take an optional destination buffer as parameter or they should simply not return any buffer at all and the destination buffer should only be made known when the callback is called. I don't much like the idea of that big a change in calling conventions. From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Mon, 23 Jan 2006 11:40:33 -0500 Message-ID: References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> <87wtgu8trl.fsf-monnier+emacs@gnu.org> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1138041239 16142 80.91.229.2 (23 Jan 2006 18:33:59 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 23 Jan 2006 18:33:59 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jan 23 19:33:54 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1F16VM-0002I7-NG for ged-emacs-devel@m.gmane.org; Mon, 23 Jan 2006 19:33:30 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F16Y0-0003LN-Ai for ged-emacs-devel@m.gmane.org; Mon, 23 Jan 2006 13:36:12 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1F14nC-0005w7-DE for emacs-devel@gnu.org; Mon, 23 Jan 2006 11:43:46 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1F14n9-0005vC-CI for emacs-devel@gnu.org; Mon, 23 Jan 2006 11:43:45 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F14n9-0005v6-1X for emacs-devel@gnu.org; Mon, 23 Jan 2006 11:43:43 -0500 Original-Received: from [132.204.24.67] (helo=mercure.iro.umontreal.ca) by monty-python.gnu.org with esmtp (Exim 4.34) id 1F14rZ-0005x0-QR; Mon, 23 Jan 2006 11:48:17 -0500 Original-Received: from hidalgo.iro.umontreal.ca (hidalgo.iro.umontreal.ca [132.204.27.50]) by mercure.iro.umontreal.ca (Postfix) with ESMTP id E723533E612; Mon, 23 Jan 2006 11:40:36 -0500 (EST) Original-Received: from asado.iro.umontreal.ca (asado.iro.umontreal.ca [132.204.24.84]) by hidalgo.iro.umontreal.ca (Postfix) with ESMTP id 431934AC017; Mon, 23 Jan 2006 11:40:33 -0500 (EST) Original-Received: by asado.iro.umontreal.ca (Postfix, from userid 20848) id 27F54E6C1E; Mon, 23 Jan 2006 11:40:33 -0500 (EST) Original-To: rms@gnu.org In-Reply-To: (Richard M. Stallman's message of "Sat, 21 Jan 2006 22:59:07 -0500") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (gnu/linux) X-DIRO-MailScanner-Information: Please contact the ISP for more information X-DIRO-MailScanner: Found to be clean X-DIRO-MailScanner-SpamCheck: n'est pas un polluriel, SpamAssassin (score=-4.854, requis 5, autolearn=not spam, AWL 0.05, BAYES_00 -4.90) X-MailScanner-From: monnier@iro.umontreal.ca X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49445 Archived-At: > 3. some backends (at least url-http, maybe others) sometimes decide > not to call the callback, presumably as a way to signal an error (the > operation can't be completed so the callback can't be called, > basically). This is a bug, but I don't know of anyone who's tried to > tackle it yet. > What exactly is the bug? It isn't clear to me. Are you saying that > they should also call the callback function to report failure? Yes. > That seems like a good idea on general principles. I don't know how > much of a change it would be. Is the calling convention easy to > extend for this? Not directly. The most natural way to do that is to pass the success/failure information as a parameter to the callback function, but if we don't want to break existing code, we could pass the info via a local variable in the destination buffer. > 2. sometimes the callback gets called in another buffer than the one > returned by url-retrieve. > One solution would be to give the first buffer a local variable that > would, in this case, point to the second buffer. > Then url-retrieve-synchronously could check the local variable, which > would tell it to check the process in the other buffer. Yes, sounds like a good quick-fix. > They (and url-retrieve) should either take an optional destination > buffer as parameter or they should simply not return any buffer at all > and the destination buffer should only be made known when the callback > is called. > I don't much like the idea of that big a change in calling conventions. Neither do I, especially not that close to a release. OTOH we haven't released URL yet so it's the last opportunity to clean up the API. Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Mon, 23 Jan 2006 15:38:40 -0500 Message-ID: References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> <87wtgu8trl.fsf-monnier+emacs@gnu.org>

NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1138050580 5311 80.91.229.2 (23 Jan 2006 21:09:40 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 23 Jan 2006 21:09:40 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jan 23 22:09:36 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1F18wI-0004sB-VW for ged-emacs-devel@m.gmane.org; Mon, 23 Jan 2006 22:09:28 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F18yx-0005oI-H2 for ged-emacs-devel@m.gmane.org; Mon, 23 Jan 2006 16:12:11 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1F18Ve-0007db-87 for emacs-devel@gnu.org; Mon, 23 Jan 2006 15:41:54 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1F18Vc-0007bt-52 for emacs-devel@gnu.org; Mon, 23 Jan 2006 15:41:53 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F18VH-0007Ct-HS for emacs-devel@gnu.org; Mon, 23 Jan 2006 15:41:34 -0500 Original-Received: from [132.204.24.67] (helo=mercure.iro.umontreal.ca) by monty-python.gnu.org with esmtp (Exim 4.34) id 1F18a4-00030V-OI for emacs-devel@gnu.org; Mon, 23 Jan 2006 15:46:28 -0500 Original-Received: from hidalgo.iro.umontreal.ca (hidalgo.iro.umontreal.ca [132.204.27.50]) by mercure.iro.umontreal.ca (Postfix) with ESMTP id A82DD2CE9BE; Mon, 23 Jan 2006 15:38:45 -0500 (EST) Original-Received: from asado.iro.umontreal.ca (asado.iro.umontreal.ca [132.204.24.84]) by hidalgo.iro.umontreal.ca (Postfix) with ESMTP id B55D24AC015; Mon, 23 Jan 2006 15:38:40 -0500 (EST) Original-Received: by asado.iro.umontreal.ca (Postfix, from userid 20848) id 91B08E6C1E; Mon, 23 Jan 2006 15:38:40 -0500 (EST) Original-To: Mark Plaksin In-Reply-To: (Stefan Monnier's message of "Mon, 23 Jan 2006 11:40:33 -0500") User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (gnu/linux) X-DIRO-MailScanner-Information: Please contact the ISP for more information X-DIRO-MailScanner: Found to be clean X-DIRO-MailScanner-SpamCheck: n'est pas un polluriel, SpamAssassin (score=-4.854, requis 5, autolearn=not spam, AWL 0.05, BAYES_00 -4.90) X-MailScanner-From: monnier@iro.umontreal.ca X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49457 Archived-At: >> 2. sometimes the callback gets called in another buffer than the one >> returned by url-retrieve. >> One solution would be to give the first buffer a local variable that >> would, in this case, point to the second buffer. >> Then url-retrieve-synchronously could check the local variable, which >> would tell it to check the process in the other buffer. > Yes, sounds like a good quick-fix. The patch below uses this approach. Mark, does it work well for you? Stefan Index: lisp/url/url.el =================================================================== RCS file: /sources/emacs/emacs/lisp/url/url.el,v retrieving revision 1.20 diff -u -r1.20 url.el --- lisp/url/url.el 10 Jan 2006 19:31:15 -0000 1.20 +++ lisp/url/url.el 23 Jan 2006 20:37:41 -0000 @@ -114,6 +114,13 @@ ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Retrieval functions ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + +(defvar url-redirect-buffer nil + "New buffer into which the retrieval will take place. +Sometimes while retrieving a URL, the URL library needs to use another buffer +than the one returned initially by `url-retrieve'. In this case, it sets this +variable in the original buffer as a forwarding pointer.") + ;;;###autoload (defun url-retrieve (url callback &optional cbargs) "Retrieve URL asynchronously and call CALLBACK with CBARGS when finished. @@ -189,10 +189,14 @@ (url-debug 'retrieval "Spinning in url-retrieve-synchronously: %S (%S)" retrieval-done asynch-buffer) + (if (buffer-local-value 'url-redirect-buffer asynch-buffer) + (setq proc (get-buffer-process + (setq asynch-buffer + (buffer-local-value 'url-redirect-buffer + asynch-buffer)))) (if (and proc (memq (process-status proc) '(closed exit signal failed)) - ;; Make sure another process hasn't been started, as can - ;; happen with http redirections. + ;; Make sure another process hasn't been started. (eq proc (or (get-buffer-process asynch-buffer) proc))) ;; FIXME: It's not clear whether url-retrieve's callback is ;; guaranteed to be called or not. It seems that url-http @@ -200,7 +204,7 @@ ;; clear that it's a bug, but even then we need to decide how ;; url-http can then warn us that the download has completed. ;; In the mean time, we use this here workaround. - (setq retrieval-done t) + (setq retrieval-done t)) ;; We used to use `sit-for' here, but in some cases it wouldn't ;; work because apparently pending keyboard input would always ;; interrupt it before it got a chance to handle process input. Index: lisp/url/url-http.el =================================================================== RCS file: /sources/emacs/emacs/lisp/url/url-http.el,v retrieving revision 1.23 diff -u -r1.23 url-http.el --- lisp/url/url-http.el 18 Nov 2005 16:55:54 -0000 1.23 +++ lisp/url/url-http.el 23 Jan 2006 20:37:41 -0000 @@ -1,6 +1,6 @@ ;;; url-http.el --- HTTP retrieval routines -;; Copyright (C) 1999, 2001, 2004, 2005 Free Software Foundation, Inc. +;; Copyright (C) 1999, 2001, 2004, 2005, 2006 Free Software Foundation, Inc. ;; Author: Bill Perry ;; Keywords: comm, data, processes @@ -35,10 +35,8 @@ (require 'url-cookie) (require 'mail-parse) (require 'url-auth) -(autoload 'url-retrieve-synchronously "url") -(autoload 'url-retrieve "url") +(require 'url) (autoload 'url-cache-create-filename "url-cache") -(autoload 'url-mark-buffer-as-dead "url") (defconst url-http-default-port 80 "Default HTTP port.") (defconst url-http-asynchronous-p t "HTTP retrievals are asynchronous.") @@ -509,10 +509,17 @@ (let ((url-request-method url-http-method) (url-request-data url-http-data) (url-request-extra-headers url-http-extra-headers)) + ;; Put in the current buffer a forwarding pointer to the new + ;; destination buffer. + ;; FIXME: This is a hack to fix url-retrieve-synchronously + ;; without changing the API. Instead url-retrieve should + ;; either simply not return the "destination" buffer, or it + ;; should take an optional `dest-buf' argument. + (set (make-local-variable 'url-redirect-buffer) (url-retrieve redirect-uri url-callback-function (cons :redirect (cons redirect-uri - url-callback-arguments))) + url-callback-arguments)))) (url-mark-buffer-as-dead (current-buffer)))))) (4 ; Client error ;; 400 Bad Request From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: "Richard M. Stallman" Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Tue, 24 Jan 2006 11:47:42 -0500 Message-ID: References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> <87wtgu8trl.fsf-monnier+emacs@gnu.org>

Reply-To: rms@gnu.org NNTP-Posting-Host: main.gmane.org Content-Type: text/plain; charset=ISO-8859-15 X-Trace: sea.gmane.org 1138123493 30142 80.91.229.2 (24 Jan 2006 17:24:53 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Tue, 24 Jan 2006 17:24:53 +0000 (UTC) Cc: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Jan 24 18:24:44 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1F1Rt3-00008d-UV for ged-emacs-devel@m.gmane.org; Tue, 24 Jan 2006 18:23:22 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F1Rvl-0006yF-7l for ged-emacs-devel@m.gmane.org; Tue, 24 Jan 2006 12:26:09 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1F1RNN-0006qo-Cj for emacs-devel@gnu.org; Tue, 24 Jan 2006 11:50:37 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1F1RNM-0006px-NH for emacs-devel@gnu.org; Tue, 24 Jan 2006 11:50:36 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1F1RNM-0006pq-JS for emacs-devel@gnu.org; Tue, 24 Jan 2006 11:50:36 -0500 Original-Received: from [199.232.76.164] (helo=fencepost.gnu.org) by monty-python.gnu.org with esmtp (Exim 4.34) id 1F1RSJ-0001KO-2D for emacs-devel@gnu.org; Tue, 24 Jan 2006 11:55:43 -0500 Original-Received: from rms by fencepost.gnu.org with local (Exim 4.34) id 1F1RKY-0000jW-JD; Tue, 24 Jan 2006 11:47:42 -0500 Original-To: Stefan Monnier In-reply-to: (message from Stefan Monnier on Mon, 23 Jan 2006 11:40:33 -0500) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:49485 Archived-At: Neither do I, especially not that close to a release. OTOH we haven't released URL yet so it's the last opportunity to clean up the API. In that case, I'd say it is a good idea to pass the success/failure info as another argument to the callback function. From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark Plaksin Newsgroups: gmane.emacs.devel Subject: Re: Fwd: HTTP redirects make url-retrieve-synchronously asynchronous Date: Sun, 19 Feb 2006 15:15:18 -0500 Message-ID: References: <87y81ep6wf.fsf-monnier+emacs@gnu.org> <87wtgu8trl.fsf-monnier+emacs@gnu.org>

NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1140392258 17994 80.91.229.2 (19 Feb 2006 23:37:38 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 19 Feb 2006 23:37:38 +0000 (UTC) Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Feb 20 00:37:37 2006 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1FAy7K-0006zH-AL for ged-emacs-devel@m.gmane.org; Mon, 20 Feb 2006 00:37:27 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FAy7J-0002Nh-TH for ged-emacs-devel@m.gmane.org; Sun, 19 Feb 2006 18:37:25 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1FAuxy-0006RK-ET for emacs-devel@gnu.org; Sun, 19 Feb 2006 15:15:35 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1FAuxr-0006PP-CC for emacs-devel@gnu.org; Sun, 19 Feb 2006 15:15:27 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1FAuxp-0006Og-PJ for emacs-devel@gnu.org; Sun, 19 Feb 2006 15:15:25 -0500 Original-Received: from [80.91.229.2] (helo=ciao.gmane.org) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.52) id 1FAv3h-0001mB-Sy for emacs-devel@gnu.org; Sun, 19 Feb 2006 15:21:30 -0500 Original-Received: from list by ciao.gmane.org with local (Exim 4.43) id 1FAuxn-000507-Hs for emacs-devel@gnu.org; Sun, 19 Feb 2006 21:15:23 +0100 Original-Received: from stone.tss.usg.edu ([168.24.82.77]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 19 Feb 2006 21:15:23 +0100 Original-Received: from happy by stone.tss.usg.edu with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 19 Feb 2006 21:15:23 +0100 X-Injected-Via-Gmane: http://gmane.org/ Original-To: emacs-devel@gnu.org Original-Lines: 17 Original-X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: stone.tss.usg.edu User-Agent: Gnus/5.110004 (No Gnus v0.4) Emacs/22.0.51 (gnu/linux) Cancel-Lock: sha1:3MoLP7ety/qrTKd1xXPqWy1aRTY= X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:50773 Archived-At: Stefan Monnier writes: >>> 2. sometimes the callback gets called in another buffer than the one >>> returned by url-retrieve. > >>> One solution would be to give the first buffer a local variable that >>> would, in this case, point to the second buffer. >>> Then url-retrieve-synchronously could check the local variable, which >>> would tell it to check the process in the other buffer. > >> Yes, sounds like a good quick-fix. > > The patch below uses this approach. Mark, does it work well for you? It does; thanks! Are applications supposed to be aware of the possibility of redirect buffers and clean up both the original buffer and the redirect buffer? You could easily say "It's a work around, so yes!" :)