Emacs 21 and w3 on Debian

Emacs 21 and w3 on Debian

[Tim X]

This weekend, I thought it was about time I tried to work out why I
have not been able to get emacs w3 working reliably. It seems that
there is a problem with relative links within w3 and I wanted to know
if anyone else has seen this and possibly has a solution. 

I'm running -

Linux Debian Sid (Testing)
Emacs 21.4
w3-el-e21 (version 4.0pre.2001.10.27-16)
w3-url-e21 (version 2001.11.08-7)

When I try to follow a link in a page, it seems w3 is parsing the
information icnorrectly. For example, if I'm at the page

http://www.some.host/dir/index.html

and that page has a url of the form <a href="about/index.html">next
link</a>, tabbing to that link and hitting enter causes w3 to try and
retrieve the url http://www.some.host/about/index.html when it should
be trying to retrieve http://www.some.host/dir/about/index.html. It
seems that instead of forming the relative link so that it is relative
to the current page, it forms one which is relative to the document
root of the remote server. 

Has anyone (especially anyone using Debian) seen this problem? If so,
does anyone have a fix? If not, I'll try to track it down further. 

Tim

-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Tim X <timx@spamto.devnul.com> writes:

> It seems that
> there is a problem with relative links within w3 and I wanted to know
> if anyone else has seen this and possibly has a solution. 

I have never experienced such a problem ; do you have an example
that is accessible on internet ?

> For example, if I'm at the page
>
> http://www.some.host/dir/index.html
>
> and that page has a url of the form <a href="about/index.html">next
> link</a>, tabbing to that link and hitting enter causes w3 to try and
> retrieve the url http://www.some.host/about/index.html when it should
> be trying to retrieve http://www.some.host/dir/about/index.html.

A wild guess: is there an erroneous
<base href="http://www.some.host/index.html">
in http://www.some.host/dir/index.html ?

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Tim X]

Thierry Emery <see.sig@spamfoil.invalid> writes:

> Tim X <timx@spamto.devnul.com> writes:
> 
> > It seems that
> > there is a problem with relative links within w3 and I wanted to know
> > if anyone else has seen this and possibly has a solution. 
> 
> I have never experienced such a problem ; do you have an example
> that is accessible on internet ?

Well, it seems pretty much any page with a relative link which is not
relative to the document root of the server. An example is

http://www.une.edu.au/itd/index.html

If you try to follow the "About ITD" link on that page, instead of
getting 

http://www.une.edu.au/itd/about/index.html
 
I get

http://www.une.edu.au/about/index.html

which doesn't work (obviously). 

I will check the page for incorrect HTML - however, the above pages
work with every other browser I've tried - w3m, firefox, opera, IE. 

> A wild guess: is there an erroneous
> <base href="http://www.some.host/index.html">
> in http://www.some.host/dir/index.html ?

Will check, but doubt its a page specific problem as all other
browsers are dealing with this page (and other) fine. Note that w3
under xemcas also works fine. 

Tim
-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Tim X]

Tim X <timx@spamto.devnul.com> writes:

> Thierry Emery <see.sig@spamfoil.invalid> writes:
> 
> > Tim X <timx@spamto.devnul.com> writes:
> > 
> > > It seems that
> > > there is a problem with relative links within w3 and I wanted to know
> > > if anyone else has seen this and possibly has a solution. 
> > 
> > I have never experienced such a problem ; do you have an example
> > that is accessible on internet ?
> 
> Well, it seems pretty much any page with a relative link which is not
> relative to the document root of the server. An example is
> 
> http://www.une.edu.au/itd/index.html
> 
> If you try to follow the "About ITD" link on that page, instead of
> getting 
> 
> http://www.une.edu.au/itd/about/index.html
>  
> I get
> 
> http://www.une.edu.au/about/index.html
> 
> which doesn't work (obviously). 
> 
> I will check the page for incorrect HTML - however, the above pages
> work with every other browser I've tried - w3m, firefox, opera, IE. 
> 

Some more information has come to light. If I attempt to go to the
page http://www.une.edu.au/itd/index.html and then follow a link in
that page to the "About ITD" it works fine. However, if I go to the
link http://www.une.edu.au/itd I get the page, but when I try to
follow relative links from that page, the link is made relative to the
document root rather than the page the link was followed from. 

So, it seems that what *may* be happening under Emacs/W3 is that it is
incorrectly removing the last element of the original url i.e. itd in
the above example, possibly mistakenly beleiving that is the last
element of the url (instead of recognizing it is the defualt
index.html) and then appending the relative link - resulting in the
wrong link. I will see if I can debug it further. 

Tim

-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Tim X]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Tim X <timx@spamto.devnul.com> wrote:
> 
> > An example is
> >
> > http://www.une.edu.au/itd/index.html
> >
> > If you try to follow the "About ITD" link on that page, instead of
> > getting
> >
> > http://www.une.edu.au/itd/about/index.html
> >
> > I get
> >
> > http://www.une.edu.au/about/index.html
> >
> > which doesn't work (obviously).
> 
> With Emacs/W3 from its CVS and Emacs/URL from the Emacs CVS, I get the
> correct link. So I think you have an outdated Emacs/W3 or something
> else is not working correctly.

Hi Klaus,

Can you try http://www.une.edu.au/itd and then try one of the links on
the page you get i.e. "About ITD" and see if that works for you. I've
found that if I use that link instead of
http://www.une.edu.au/itd/index.html, it doesn't work, but if I 
use the full link with the index.html on the end, relative links work
OK.  

I'm guessing the problem is that Emacs/W3 is interpreting a link that
ends without the explicit .html page (e.g. index.html) incorrectly -
its stripping the last element off the link and adding the relative
link to that, which is incorrect. 

regards,

Tim


-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Tim X <timx@spamto.devnul.com> writes:

> Well, it seems pretty much any page with a relative link which is not
> relative to the document root of the server. An example is
>
> http://www.une.edu.au/itd/index.html
>
> If you try to follow the "About ITD" link on that page, instead of
> getting 
>
> http://www.une.edu.au/itd/about/index.html

Hmm, strange, this works for me when using the same environment as you have:

Debian GNU/Linux sid
GNU Emacs 21.4.1 (i386-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2005-03-17 on trouble, modified by Debian
w3-el-e21 4.0pre.2001.10.27-16
w3-url-e21 2001.11.08-7

with the following procedure:

/usr/bin/emacs -q &
M-x w3-fetch
http://www.une.edu.au/itd/index.html
<click on "About ITD">

this does get http://www.une.edu.au/itd/about/index.html ...


However:

1. Errors are signaled when loading images ("Text is read-only"),
   this is solved with:

(defadvice w3-finalize-image-download (around set-inhibit-read-only activate)
  (let ((inhibit-read-only t))
    ad-do-it))

(defadvice widget-image-value-create (around set-inhibit-read-only activate)
  (let ((inhibit-read-only t))
    ad-do-it))

2. Images are not inserted where they belong, this is fixed with:
   
(eval-after-load "w3-widget" (quote
(defun widget-image-value-set (widget value)
  ;; Recreate widget with new value.
  (save-excursion
    (let* ((where (widget-get widget 'where)))
      (widget-image-delete widget)
      (if (or (eq 'image (car-safe value)) ; Emacs 21
	      (widget-glyphp value))
	  (widget-put widget 'glyph value)
	(widget-put widget :value value))
      (and where
	   (goto-char where))
      (put-text-property (point)
			 (progn
			   (widget-apply widget :create)
			   (point))
			 'inaudible
			 widget-image-inaudible-p))))
))

3. There are ">" glitches, which are eliminated with:

(add-hook 'w3-parse-hooks
	  (lambda ()
	    (while (search-forward "/>" nil t) 
	      (replace-match ">"))))


Hoping this helps,

Thierry
-- 
thierry |point| emery |chez| free |point| fr

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Tim X <timx@spamto.devnul.com> writes:

> Can you try http://www.une.edu.au/itd and then try one of the links on
> the page you get i.e. "About ITD" and see if that works for you. I've
> found that if I use that link instead of
> http://www.une.edu.au/itd/index.html, it doesn't work, but if I 
> use the full link with the index.html on the end, relative links work
> OK.  

Oh, i hadn't though of that !

What happens is that `url-http-parse-headers' gets back a "301"
redirection, with URI "http://www.une.edu.au/itd/", but it does not
pass it on to `w3-fetch-callback' ...

This can be solved with the following patch to url-http.el:

--- url-http.el~
+++ url-http.el
@@ -461,7 +461,8 @@
 		(url-request-data url-http-data)
 		(url-request-extra-headers url-http-extra-headers))
 	    (url-retrieve redirect-uri url-callback-function
-			  url-callback-arguments)
+			  (list redirect-uri) ;;url-callback-arguments
+			  )
 	    (url-mark-buffer-as-dead (current-buffer))))))
      ((= class 4)			; Client error
       ;; 400 Bad Request


HTH,

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Thierry Emery <see.sig@spamfoil.invalid> wrote:
>
>> (defadvice w3-finalize-image-download (around set-inhibit-read-only activate)
>>   (let ((inhibit-read-only t))
>>     ad-do-it))
>
> Thank you for your improvements. They work good for me.
> Do have any possibility to bring them into the CVS version?

I have a CVS client installed, if that is what you are asking ?
Who is in charge of the CVS version you mention ?

>> (defadvice widget-image-value-create (around set-inhibit-read-only activate)
>>   (let ((inhibit-read-only t))
>>     ad-do-it))
>
> Is this really necessary?

Oops, no -- i had written it chronologically before the first `defadvice'
and have not checked afterwards whether it was still useful ...
thanks for testing it !


> I have been able to achieve good results with the following:
>
>   (defadvice widget-image-value-set (around set-point activate)
>     (save-excursion
>       (let ((p (widget-get (ad-get-arg 0) 'where)))
>         (if p (goto-char p)))
>       ad-do-it))
>
> This avoids redefining the whole function and using eval-after-load.

Yes, this is more elegant on the user side !

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Did you suggest that improvement to the Emacs development list?
> If not yet, I think that would be a good idea. The URL library
> is maintained there (in parallel?).

I have proposed it both through M-x report-emacs-bug and to Debian,
as they maintain two slightly different versions of the URL package.

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Thierry Emery <see.sig@spamfoil.invalid> wrote:

>> Who is in charge of the CVS version you mention ?
>
> I don't know. That's part of the problem.

In the meantime, i have submitted modifications to
w3-finalize-image-download and widget-image-value-set
as Debian GNU/Linux bug reports.

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Thierry Emery <see.sig@spamfoil.invalid> wrote:

> I have proposed it both through M-x report-emacs-bug and to Debian,
> as they maintain two slightly different versions of the URL package.

Maybe someone should suggest to Debian that they work with Emacs
development instead of doubling the maintenance effort.

Does Debian maintain (officially or not) the W3 package? This package
is not (yet?) in the Emacs development tree.

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Tim X]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Tim X <timx@spamto.devnul.com> wrote:
> 
> > Can you try http://www.une.edu.au/itd and then try one of the links on
> > the page you get i.e. "About ITD" and see if that works for you.
> 
> That does not work, indeed. I have found that the problem has its root
> in the handling directories without the trailing "/". If you try
> "http://www.une.edu.au/itd/" (note the trailing slash) you will see the
> difference.
> 
> > I've found that if I use that link instead of
> > http://www.une.edu.au/itd/index.html, it doesn't work, but if I use
> > the full link with the index.html on the end, relative links work OK.
> 
> That will work, too, yes.
> 
> > I'm guessing the problem is that Emacs/W3 is interpreting a link that
> > ends without the explicit .html page (e.g. index.html) incorrectly -
> > its stripping the last element off the link and adding the relative
> > link to that, which is incorrect.
> 
> I have my own suspicion, see above. However, I have not had enough
> energy to fix it. Maybe it is widespread but incorrect HTML usage
> and Emacs/W3 is not to blame.
> 
Thanks Klaus. I think your right in that without the / it is incorrect
(technically). Unfortunately, most other browsers I've tried do work
without the slash, so I guess its one of those examples where the
technical spec and common use differ. 

Tim

-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Tim X]

Thierry Emery <see.sig@spamfoil.invalid> writes:

> Tim X <timx@spamto.devnul.com> writes:
> 
> > Can you try http://www.une.edu.au/itd and then try one of the links on
> > the page you get i.e. "About ITD" and see if that works for you. I've
> > found that if I use that link instead of
> > http://www.une.edu.au/itd/index.html, it doesn't work, but if I 
> > use the full link with the index.html on the end, relative links work
> > OK.  
> 
> Oh, i hadn't though of that !
> 
> What happens is that `url-http-parse-headers' gets back a "301"
> redirection, with URI "http://www.une.edu.au/itd/", but it does not
> pass it on to `w3-fetch-callback' ...
> 
> This can be solved with the following patch to url-http.el:
> 
> --- url-http.el~
> +++ url-http.el
> @@ -461,7 +461,8 @@
>  		(url-request-data url-http-data)
>  		(url-request-extra-headers url-http-extra-headers))
>  	    (url-retrieve redirect-uri url-callback-function
> -			  url-callback-arguments)
> +			  (list redirect-uri) ;;url-callback-arguments
> +			  )
>  	    (url-mark-buffer-as-dead (current-buffer))))))
>       ((= class 4)			; Client error
>        ;; 400 Bad Request
> 
> 
> HTH,
> 
> Thierry
> -- 
> thierry |dot| emery |at| free |dot| fr

Thierry,

Thanks a lot for that - you saved me a lot of work and have gone above
and beyond with this one. Greatly appreciated!

regards,

Tim
-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Tim X]

Thierry Emery <see.sig@spamfoil.invalid> writes:

> Tim X <timx@spamto.devnul.com> writes:
> 
> > Well, it seems pretty much any page with a relative link which is not
> > relative to the document root of the server. An example is
> >
> > http://www.une.edu.au/itd/index.html
> >
> > If you try to follow the "About ITD" link on that page, instead of
> > getting 
> >
> > http://www.une.edu.au/itd/about/index.html
> 
> Hmm, strange, this works for me when using the same environment as you have:
> 
> Debian GNU/Linux sid
> GNU Emacs 21.4.1 (i386-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
>  of 2005-03-17 on trouble, modified by Debian
> w3-el-e21 4.0pre.2001.10.27-16
> w3-url-e21 2001.11.08-7
> 
> with the following procedure:
> 
> /usr/bin/emacs -q &
> M-x w3-fetch
> http://www.une.edu.au/itd/index.html
> <click on "About ITD">
> 
> this does get http://www.une.edu.au/itd/about/index.html ...
> 
> 
> However:
> 
> 1. Errors are signaled when loading images ("Text is read-only"),
>    this is solved with:
> 
> (defadvice w3-finalize-image-download (around set-inhibit-read-only activate)
>   (let ((inhibit-read-only t))
>     ad-do-it))
> 
> (defadvice widget-image-value-create (around set-inhibit-read-only activate)
>   (let ((inhibit-read-only t))
>     ad-do-it))
> 
> 2. Images are not inserted where they belong, this is fixed with:
>    
> (eval-after-load "w3-widget" (quote
> (defun widget-image-value-set (widget value)
>   ;; Recreate widget with new value.
>   (save-excursion
>     (let* ((where (widget-get widget 'where)))
>       (widget-image-delete widget)
>       (if (or (eq 'image (car-safe value)) ; Emacs 21
> 	      (widget-glyphp value))
> 	  (widget-put widget 'glyph value)
> 	(widget-put widget :value value))
>       (and where
> 	   (goto-char where))
>       (put-text-property (point)
> 			 (progn
> 			   (widget-apply widget :create)
> 			   (point))
> 			 'inaudible
> 			 widget-image-inaudible-p))))
> ))
> 
> 3. There are ">" glitches, which are eliminated with:
> 
> (add-hook 'w3-parse-hooks
> 	  (lambda ()
> 	    (while (search-forward "/>" nil t) 
> 	      (replace-match ">"))))
> 
> 
> Hoping this helps,
> 
> Thierry
> -- 
> thierry |point| emery |chez| free |point| fr

Thierry,

You blow me away with all of this! I've used defadvice a bit myself to
get around problems etc, but it would take me ages to work out
everything that you have so quickly. You have reinspired my faith in
newsgroups!

thanks,

Tim

-- 
Tim Cross
The e-mail address on this message is FALSE (obviously!). My real e-mail is
to a company in Australia called rapttech and my login is tcross - if you 
really need to send mail, you should be able to work it out!

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Maybe someone should suggest to Debian that they work with Emacs
> development instead of doubling the maintenance effort.

So far, Debian GNU/Linux "sid" includes GNU Emacs 21.4, which does not
contain the URL package.

I suppose that when Debian upgrades to GNU Emacs 22 (either released
or from CVS) there won't be such a double maintenance anymore ?

> Does Debian maintain (officially or not) the W3 package?

Yes, based on a 2001 snapshot of its CVS version.  It seems to me that
there are only minor differences with the current state of the CVS version.

> This package is not (yet?) in the Emacs development tree.

You are right.  It is still a GNU project though:
http://savannah.gnu.org/projects/w3/

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Wouldn't it be more correct to replace the first argument of
> url-callback-arguments with the redirect-uri if there are any
> such callback arguments present? I have experienced problems
> with your solution that went away with this improvement.

Through looking at the code, i thought that i had chosen the most
cautious alternative, but it seems that i was wrong ...

Is it ok with you if i request Emacs and Debian maintainers to
replace:
(list redirect-uri)
with:
(cons redirect-uri (cdr url-callback-arguments))
?

Thanks again for your careful testing,

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Kevin Rodgers]

Klaus Straubinger wrote:
 > Wouldn't it be more correct to replace the first argument of
 > url-callback-arguments with the redirect-uri if there are any
 > such callback arguments present? I have experienced problems
 > with your solution that went away with this improvement.

How exactly is that improvement implemented?

(cons redirect-uri url-callback-arguments)

(if url-callback-arguments
     (cons redirect-uri (cdr url-callback-arguments))
   (list redirect-uri))

-- 
Kevin Rodgers

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

I wrote:

> Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:
>
>> Wouldn't it be more correct to replace the first argument of
>> url-callback-arguments with the redirect-uri if there are any
>> such callback arguments present? I have experienced problems
>> with your solution that went away with this improvement.
>
> Through looking at the code, i thought that i had chosen the most
> cautious alternative, but it seems that i was wrong ...

Ok, i have taken a much better look at the different places in Debian
`sid' GNU Emacs 21.4 in which there are calls to `url-retrieve' and
found out that my bogus patch caused problems in
`w3-maybe-start-image-download' and
`w3-maybe-start-background-image-download',
but also in the Gnus library "nnweb.el".

I have submitted two patches for these problems to Debian:
 - the first one replaces in "url-http.el" my erroneous:
			  (list redirect-uri)
with:
			  (cons redirect-uri
				(and url-callback-arguments
				     (cdr url-callback-arguments)))
 - the second patch adds in "nnweb.el" `url' as first argument
   for `nnweb-callback' and as third argument in the call to
   `nnweb-url-retrieve-asynch' in `nnweb-fetch-url' (arguments used
   when calling `nnweb-callback' start with this third argument).

I have also submitted a modified patch to "url-http.el" as a follow-up
to my M-x report-emacs-bug.
The patch on "nnweb.el" is not applicable to CVS Emacs: library
"nnweb.el" does not call `url-retrieve' anymore, but uses
`mm-url-insert' instead, which calls `url-insert-file-contents'.

Finally, i have found that backporting to Emacs 21.4 the new
`url-retrieve-synchronously' from "url.el" in CVS Emacs avoids some
cases of Emacs 21.4 locking up (e.g. when a site has lots of images),
and have also submitted a patch for this to Debian.

Thanks again for pointing out my mistake,

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Thierry Emery <see.sig@spamfoil.invalid> wrote:

> Finally, i have found that backporting to Emacs 21.4 the new
> `url-retrieve-synchronously' from "url.el" in CVS Emacs avoids some
> cases of Emacs 21.4 locking up (e.g. when a site has lots of images)

Indeed. Did you also experience locking-up in redirecting? I noticed
improvement after inserting another (accept-process-output url-http-process)
there.

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Kevin Rodgers <ihs_4664@yahoo.com> wrote:

> How exactly is that improvement implemented?
>
> (cons redirect-uri url-callback-arguments)
>
> (if url-callback-arguments
>      (cons redirect-uri (cdr url-callback-arguments))
>    (list redirect-uri))

I chose something like the second alternative:

  (if url-callback-arguments
      (setcar url-callback-arguments redirect-uri)
    (setq url-callback-arguments (list redirect-uri)))

The first would not be correct because the URL sometimes already
present as first argument in the url-callback-arguments list must be
replaced.

Would it work then to simply write

  (setq url-callback-arguments
          (append (list redirect-uri) (cdr url-callback-arguments)))

independent of the value of url-callback-arguments?

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Kevin Rodgers]

Klaus Straubinger wrote:
 > Kevin Rodgers <ihs_4664@yahoo.com> wrote:
 >>How exactly is that improvement implemented?
 >>
 >>(cons redirect-uri url-callback-arguments)
 >>
 >>(if url-callback-arguments
 >>     (cons redirect-uri (cdr url-callback-arguments))
 >>   (list redirect-uri))
 >
 > I chose something like the second alternative:
 >
 >   (if url-callback-arguments
 >       (setcar url-callback-arguments redirect-uri)
 >     (setq url-callback-arguments (list redirect-uri)))

Is it necessary to destructively modify url-callback-arguments?  I.e.
is it used elsewhere that must reflect the redirection as well?

Note that setcar does not return the modified cons, so I think that
should be (asssuming setcar is indeed necessary):

   (if url-callback-arguments
       (progn
         (setcar url-callback-arguments redirect-uri)
         url-callback-arguments)
     (setq url-callback-arguments (list redirect-uri)))

 > The first would not be correct because the URL sometimes already
 > present as first argument in the url-callback-arguments list must be
 > replaced.

Again: destructively, or just in this particular call to url-retrieve?

 > Would it work then to simply write
 >
 >   (setq url-callback-arguments
 >           (append (list redirect-uri) (cdr url-callback-arguments)))
 >
 > independent of the value of url-callback-arguments?

Yes, it's not necessary to test url-callback-arguments and it's better
to avoid destructive operations on lists unless you're sure of what
you're doing.

But (append (list x) ...) is better expressed as (cons x ...):

   (cons redirect-uri (cdr url-callback-arguments))

Again, why setq?  Do you really need to change its global value?

-- 
Kevin Rodgers

Re: Emacs 21 and w3 on Debian

[Thierry Emery]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Indeed. Did you also experience locking-up in redirecting?

No ; did you ? If so, do you have an accessible example ?

> I noticed
> improvement after inserting another (accept-process-output url-http-process)

"another" ?
(i couldn't find any occurrence of this in the Debian package source)

> there.

and i don't see where this is ...

Could you give more information ?

Thierry
-- 
thierry |dot| emery |at| free |dot| fr

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Kevin Rodgers <ihs_4664@yahoo.com> wrote:

> Is it necessary to destructively modify url-callback-arguments?

I am not sure because I have no complete overview over the URL library.
But in many places dynamic binding is used where one indeed must modify
the globally visible value of variables. See for example the definition
of the function "url-http-activate-callback". By the way: The code uses
"(declare (special ...))" to mark these cases and to pacify the
byte-compiler. Someone with CVS write access could replace these with an
equivalent non-CL statement, i.e., append the named variables to
"byte-compile-bound-variables".

> But (append (list x) ...) is better expressed as (cons x ...):
>
>    (cons redirect-uri (cdr url-callback-arguments))

Isn't that a matter of taste? I find it more intuitive to use "append"
and "list" for list operations and "cons" for simple cells. But I am no
lisp expert.

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Thierry Emery <see.sig@spamfoil.invalid> wrote:

>> I noticed
>> improvement after inserting another (accept-process-output url-http-process)
>
> "another" ?
> (i couldn't find any occurrence of this in the Debian package source)

It is in "url-retrieve-synchronously" as provided by Emacs CVS.

>> there.
>
> and i don't see where this is ...

I did it in "url-http-parse-headers" where responses of class 3
(redirection) are handled and another "url-retrieve" is called.
Before this could succeed, one has to make sure that the previous
process has a chance to finish, hence
(accept-process-output url-http-process).

> Could you give more information ?

For me the problem only happened with sites that provide no
Content-Length header. Unfortunately, I cannot remember any more with
which site I tested.

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Kevin Rodgers]

Klaus Straubinger wrote:
 > Kevin Rodgers <ihs_4664@yahoo.com> wrote:
 >>Is it necessary to destructively modify url-callback-arguments?
 >
 > I am not sure because I have no complete overview over the URL library.
 > But in many places dynamic binding is used where one indeed must modify
 > the globally visible value of variables.

That is not what I asked.  One can change the globally visible value of
a variable without destructively modifying the original list structure.
If you intend to change the variable permanently, you use setq; if you
intend to change it temporarily (e.g. for the duration of a function
call), you use let.  But you should destructively modify the original
value only when changing the variable permanently, and only if you
understand the implications (namely, that you are modifying by
side-effect any other variable that happens to share that data
structure).

 > See for example the definition
 > of the function "url-http-activate-callback".

I don't see any modification of a global variable (permanent or
temporary, destructive or not) in the CVS implementation at
http://savannah.gnu.org/cgi-bin/viewcvs/emacs/emacs/lisp/url/url-http.el?rev=HEAD&content-type=text/vnd.viewcvs-markup

 > By the way: The code uses
 > "(declare (special ...))" to mark these cases and to pacify the
 > byte-compiler. Someone with CVS write access could replace these with an
 > equivalent non-CL statement, i.e., append the named variables to
 > "byte-compile-bound-variables".

No, those declarations are there to pacify the Common Lisp cravings of
the author.  The right thing to do is to defvar the global variables at
the top level (either in url-http.el itself or in one of the libraries
it requires).

 >>But (append (list x) ...) is better expressed as (cons x ...):
 >>
 >>   (cons redirect-uri (cdr url-callback-arguments))
 >
 > Isn't that a matter of taste? I find it more intuitive to use "append"
 > and "list" for list operations and "cons" for simple cells. But I am no
 > lisp expert.

Yes, I suppose it is a matter of style.  But the fact is that cons is
the primitive function for constructing a new list from an element and
an old list.  And 2 function calls are seldom more readable than 1.

-- 
Kevin Rodgers

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Kevin Rodgers <ihs_4664@yahoo.com> wrote:

[(declare (special ...))]
> those declarations are there to pacify the Common Lisp cravings of
> the author.  The right thing to do is to defvar the global variables at
> the top level (either in url-http.el itself or in one of the libraries
> it requires).

Could you do that? It is (or has been, last time I looked) on the
release-critical list to get rid of these declarations.

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Stefan Monnier]

>> By the way: The code uses
>> "(declare (special ...))" to mark these cases and to pacify the
>> byte-compiler. Someone with CVS write access could replace these with an
>> equivalent non-CL statement, i.e., append the named variables to
>> "byte-compile-bound-variables".

> No, those declarations are there to pacify the Common Lisp cravings of
> the author.  The right thing to do is to defvar the global variables at
> the top level (either in url-http.el itself or in one of the libraries
> it requires).

There's nothing wrong with (declare (special ...) ...).

>>> But (append (list x) ...) is better expressed as (cons x ...):
>>> 
>>> (cons redirect-uri (cdr url-callback-arguments))
>> 
>> Isn't that a matter of taste? I find it more intuitive to use "append"
>> and "list" for list operations and "cons" for simple cells.  But I am no
>> lisp expert.

It's just a lot less efficient.


        Stefan

Re: Emacs 21 and w3 on Debian

[Stefan Monnier]

> [(declare (special ...))]
>> those declarations are there to pacify the Common Lisp cravings of
>> the author.  The right thing to do is to defvar the global variables at
>> the top level (either in url-http.el itself or in one of the libraries
>> it requires).
> Could you do that?

Please don't unless there's a good reason other than personal preference.

> It is (or has been, last time I looked) on the
> release-critical list to get rid of these declarations.

Which list?  Which release?  It's not in admin/FOR-RELEASE and it's not
known to create any particular problem.


        Stefan

Re: Emacs 21 and w3 on Debian

[Klaus Straubinger]

Stefan Monnier <monnier@iro.umontreal.ca> wrote:

[remove (declare (special ...))]
> Please don't unless there's a good reason other than personal preference.

I thought it is recommended to get rid of CL constructs.
If that is (still?) correct, it is not only personal preference.

> It's not in admin/FOR-RELEASE

It had been until February 28.

> and it's not known to create any particular problem.

Indeed. I never said anything else.

-- 
Klaus Straubinger

Re: Emacs 21 and w3 on Debian

[Stefan Monnier]

>> Please don't unless there's a good reason other than personal preference.
> I thought it is recommended to get rid of CL constructs.
> If that is (still?) correct, it is not only personal preference.

The only convention I know is that packages bundled with Emacs shouldn't use
CL at run-time, tho they are free to use CL at compile-time.  I.e. CL macros
are perfectly fine and are used pretty frequently.


        Stefan