%-escapes in file URL

%-escapes in file URL

[YAMAMOTO Mitsuharu]

In dnd-handle-one-url, %-escapes in the given file URL is decoded
immediately.  But I think that should be done when extracting the file
name from it (i.e., in dnd-get-local-file-name) in order to cope with
applications outside Emacs via browse-url.  On the other hand,
%-escapes are needed when constructing a file URL from a file name.

Below is a patch for such changes.  The w32-win.el part is by M.Fujii.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Index: lisp/dnd.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/dnd.el,v
retrieving revision 1.8
diff -c -r1.8 dnd.el
*** lisp/dnd.el	6 Feb 2006 14:33:32 -0000	1.8
--- lisp/dnd.el	15 May 2006 07:48:02 -0000
***************
*** 69,107 ****
  
  ;; Functions
  
! (defun dnd-handle-one-url (window action arg)
    "Handle one dropped url by calling the appropriate handler.
  The handler is first located by looking at `dnd-protocol-alist'.
  If no match is found here, and the value of `browse-url-browser-function'
  is a pair of (REGEXP . FUNCTION), those regexps are tried for a match.
  If no match is found, just call `dnd-insert-text'.
  WINDOW is where the drop happend, ACTION is the action for the drop,
! ARG is the URL that has been dropped.
  Returns ACTION."
    (require 'browse-url)
!   (let* ((uri (replace-regexp-in-string
! 	       "%[A-Z0-9][A-Z0-9]"
! 	       (lambda (arg)
! 		 (format "%c" (string-to-number (substring arg 1) 16)))
! 	       arg))
! 	 ret)
      (or
       (catch 'done
         (dolist (bf dnd-protocol-alist)
! 	 (when (string-match (car bf) uri)
! 	   (setq ret (funcall (cdr bf) uri action))
  	   (throw 'done t)))
         nil)
       (when (not (functionp browse-url-browser-function))
         (catch 'done
  	 (dolist (bf browse-url-browser-function)
! 	   (when (string-match (car bf) uri)
  	     (setq ret 'private)
! 	     (funcall (cdr bf) uri action)
  	     (throw 'done t)))
  	 nil))
       (progn
!        (dnd-insert-text window action uri)
         (setq ret 'private)))
      ret))
  
--- 69,102 ----
  
  ;; Functions
  
! (defun dnd-handle-one-url (window action url)
    "Handle one dropped url by calling the appropriate handler.
  The handler is first located by looking at `dnd-protocol-alist'.
  If no match is found here, and the value of `browse-url-browser-function'
  is a pair of (REGEXP . FUNCTION), those regexps are tried for a match.
  If no match is found, just call `dnd-insert-text'.
  WINDOW is where the drop happend, ACTION is the action for the drop,
! URL is what has been dropped.
  Returns ACTION."
    (require 'browse-url)
!   (let (ret)
      (or
       (catch 'done
         (dolist (bf dnd-protocol-alist)
! 	 (when (string-match (car bf) url)
! 	   (setq ret (funcall (cdr bf) url action))
  	   (throw 'done t)))
         nil)
       (when (not (functionp browse-url-browser-function))
         (catch 'done
  	 (dolist (bf browse-url-browser-function)
! 	   (when (string-match (car bf) url)
  	     (setq ret 'private)
! 	     (funcall (cdr bf) url action)
  	     (throw 'done t)))
  	 nil))
       (progn
!        (dnd-insert-text window action url)
         (setq ret 'private)))
      ret))
  
***************
*** 134,139 ****
--- 129,139 ----
  		 ((string-match "^file:" uri)		; Old KDE, Motif, Sun
  		  (substring uri (match-end 0))))))
      (when (and f must-exist)
+       (setq f (replace-regexp-in-string
+ 	       "%[A-Z0-9][A-Z0-9]"
+ 	       (lambda (arg)
+ 		 (format "%c" (string-to-number (substring arg 1) 16)))
+ 	       f))
        (let* ((decoded-f (decode-coding-string
  			 f
  			 (or file-name-coding-system
Index: lisp/x-dnd.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/x-dnd.el,v
retrieving revision 1.19
diff -c -r1.19 x-dnd.el
*** lisp/x-dnd.el	6 Feb 2006 14:33:35 -0000	1.19
--- lisp/x-dnd.el	15 May 2006 07:48:02 -0000
***************
*** 263,269 ****
  	retval)
      (dolist (bf uri-list)
        ;; If one URL is handeled, treat as if the whole drop succeeded.
!       (let* ((file-uri (concat "file://" bf))
  	     (did-action (dnd-handle-one-url window action file-uri)))
  	(when did-action (setq retval did-action))))
      retval))
--- 263,274 ----
  	retval)
      (dolist (bf uri-list)
        ;; If one URL is handeled, treat as if the whole drop succeeded.
!       (setq bf
! 	    (encode-coding-string bf (or file-name-coding-system
! 					 default-file-name-coding-system)))
!       (let* ((file-uri (concat "file://"
! 			       (mapconcat 'url-hexify-string
! 					  (split-string bf "/") "/")))
  	     (did-action (dnd-handle-one-url window action file-uri)))
  	(when did-action (setq retval did-action))))
      retval))
Index: lisp/term/mac-win.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/term/mac-win.el,v
retrieving revision 1.70
diff -c -r1.70 mac-win.el
*** lisp/term/mac-win.el	10 May 2006 08:22:39 -0000	1.70
--- lisp/term/mac-win.el	15 May 2006 07:48:02 -0000
***************
*** 1627,1633 ****
    (let ((ae (mac-event-ae event)))
      (dolist (file-name (mac-ae-list ae nil 'undecoded-file-name))
        (if file-name
! 	  (dnd-open-local-file (concat "file:" file-name) nil)))
      (let ((selection-range (mac-ae-selection-range ae))
  	  (search-text (mac-ae-text-for-search ae)))
        (cond (selection-range
--- 1616,1625 ----
    (let ((ae (mac-event-ae event)))
      (dolist (file-name (mac-ae-list ae nil 'undecoded-file-name))
        (if file-name
! 	  (dnd-open-local-file
! 	   (concat "file://"
! 		   (mapconcat 'url-hexify-string
! 			      (split-string file-name "/") "/")) nil)))
      (let ((selection-range (mac-ae-selection-range ae))
  	  (search-text (mac-ae-text-for-search ae)))
        (cond (selection-range
Index: lisp/term/w32-win.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/term/w32-win.el,v
retrieving revision 1.76
diff -c -r1.76 w32-win.el
*** lisp/term/w32-win.el	18 Mar 2006 13:13:30 -0000	1.76
--- lisp/term/w32-win.el	15 May 2006 07:48:02 -0000
***************
*** 112,117 ****
--- 112,125 ----
        (if (and (> x 0) (> y 0))
  	  (set-frame-selected-window nil window))
        (mapcar (lambda (file-name)
+ 		(let ((f (subst-char-in-string ?\\ ?/ file-name))
+ 		      (coding (or file-name-coding-system
+ 				  default-file-name-coding-system)))
+ 		  (setq file-name
+ 			(mapconcat 'url-hexify-string
+ 				   (split-string (encode-coding-string f coding)
+ 						 "/")
+ 				   "/")))
  		(dnd-handle-one-url window 'private
  				    (concat "file:" file-name)))
  		(car (cdr (cdr event)))))

Re: %-escapes in file URL

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
> In dnd-handle-one-url, %-escapes in the given file URL is decoded
> immediately.  But I think that should be done when extracting the file
> name from it (i.e., in dnd-get-local-file-name) in order to cope with
> applications outside Emacs via browse-url.  On the other hand,
> %-escapes are needed when constructing a file URL from a file name.

I see that the functions in browse-url does some %-escaping where so is 
needed.  Would it not be better to keep it consistent, i.e. always give 
browse-url functions URL:s without %-escapes?

	Jan D.

Re: %-escapes in file URL

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 15 May 2006 11:59:52 +0200, Jan Djärv <jan.h.d@swipnet.se> said:

> I see that the functions in browse-url does some %-escaping where so
> is needed.  Would it not be better to keep it consistent,
> i.e. always give browse-url functions URL:s without %-escapes?

Which one?  `browse-url-file-url' takes a file name, not a URL.
(BTW, I think (setq file (encode-coding-system file (or
default-file-name-coding-system file-name-coding-system))) is needed
before %-escaping.)
`browse-url-{netscape,mozilla,firefox}' take a URL, but they only
%-excape a few characters such as `)' so as to avoid confusion with
their remote control syntax.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: %-escapes in file URL

[Jan Djärv]

YAMAMOTO Mitsuharu skrev:
>>>>>> On Mon, 15 May 2006 11:59:52 +0200, Jan Djärv <jan.h.d@swipnet.se> said:
> 
>> I see that the functions in browse-url does some %-escaping where so
>> is needed.  Would it not be better to keep it consistent,
>> i.e. always give browse-url functions URL:s without %-escapes?
> 
> Which one?  `browse-url-file-url' takes a file name, not a URL.
> (BTW, I think (setq file (encode-coding-system file (or
> default-file-name-coding-system file-name-coding-system))) is needed
> before %-escaping.)
> `browse-url-{netscape,mozilla,firefox}' take a URL, but they only
> %-excape a few characters such as `)' so as to avoid confusion with
> their remote control syntax.

I was thinking of browse-url-{netscape,mozilla,firefox}.

	Jan D.

Re: %-escapes in file URL

[Richard Stallman]

    In dnd-handle-one-url, %-escapes in the given file URL is decoded
    immediately.  But I think that should be done when extracting the file
    name from it (i.e., in dnd-get-local-file-name) in order to cope with
    applications outside Emacs via browse-url.

Could you explain the connection, please?

Re: %-escapes in file URL

[YAMAMOTO Mitsuharu]

Richard Stallman <rms <at> gnu.org> writes:

> 
>     In dnd-handle-one-url, %-escapes in the given file URL is decoded
>     immediately.  But I think that should be done when extracting the file
>     name from it (i.e., in dnd-get-local-file-name) in order to cope with
>     applications outside Emacs via browse-url.
> 
> Could you explain the connection, please?
> 

(I'm resending this from Gmane, because my emails seem to be rejected by
mx10.gnu.org with "Unrouteable address" these days.) 

Sure.  RFC 2396 says:

2.4.2. When to Escape and Unescape

   A URI is always in an "escaped" form, since escaping or unescaping a
   completed URI might change its semantics.  Normally, the only time
   escape encodings can safely be made is when the URI is being created
   from its component parts; each component may have its own set of
   characters that are reserved, so only the mechanism responsible for
   generating or interpreting that component can determine whether or
   not escaping a character will change its semantics. Likewise, a URI
   must be separated into its components before the escaped characters
   within those components can be safely decoded.

So, URLs coming from other applications (e.g, via drag-and-drop)
should be assumed to be in escaped forms.  Likewise, URLs going to
other applications (e.g., via browse-url) should be in escaped forms.
Some functions in browse-url such as `browse-url-netscape' does
%-escaping, but that is not re-escaping but just increasing the kind
of characters that should be escaped (notably `,' and `)').

A simple and natural policy is to always keep URLs in escaped forms,
whether or not they are used inside Emacs only.  So, my proposal is as
follows:

  * Always treat URLs as those in escaped form.  Don't re-escape or
    unescape them as long as they are URLs (i.e, have a prefix such as
    "file://").  Increasing the kind of escaped characters is not
    re-escaping.

  * When creating a URL from a file name, encode the file name with
    the file name coding system and then %-escape it.

  * When creating a file name from a URL, %-unescape the URL and then
    decode it with the file name coding system.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: %-escapes in file URL

[Jan Djärv]

YAMAMOTO Mitsuharu wrote:

> A simple and natural policy is to always keep URLs in escaped forms,
> whether or not they are used inside Emacs only.  So, my proposal is as
> follows:
> 
>   * Always treat URLs as those in escaped form.  Don't re-escape or
>     unescape them as long as they are URLs (i.e, have a prefix such as
>     "file://").  Increasing the kind of escaped characters is not
>     re-escaping.
> 
>   * When creating a URL from a file name, encode the file name with
>     the file name coding system and then %-escape it.
> 
>   * When creating a file name from a URL, %-unescape the URL and then
>     decode it with the file name coding system.

I'm fine with that policy.  Can you checkin your patch?

	Jan D.

Re: %-escapes in file URL

[YAMAMOTO Mitsuharu]

Jan Djärv <jan.h.d <at> swipnet.se> writes:

> I'm fine with that policy.  Can you checkin your patch?

Done.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: %-escapes in file URL

[Richard Stallman]

    (I'm resending this from Gmane, because my emails seem to be rejected by
    mx10.gnu.org with "Unrouteable address" these days.) 

Can you please show an example of a failure message?
If there is a problem, we need to FIX it!

Re: %-escapes in file URL

[Richard Stallman]

    A simple and natural policy is to always keep URLs in escaped forms,
    whether or not they are used inside Emacs only.  So, my proposal is as
    follows:

      * Always treat URLs as those in escaped form.  Don't re-escape or
	unescape them as long as they are URLs (i.e, have a prefix such as
	"file://").  Increasing the kind of escaped characters is not
	re-escaping.

      * When creating a URL from a file name, encode the file name with
	the file name coding system and then %-escape it.

      * When creating a file name from a URL, %-unescape the URL and then
	decode it with the file name coding system.

Do people agree with that proposal?

What changes are needed to implement it?

Re: %-escapes in file URL

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 20 May 2006 01:04:27 -0400, Richard Stallman <rms@gnu.org> said:

>     (I'm resending this from Gmane, because my emails seem to be
>     rejected by mx10.gnu.org with "Unrouteable address" these days.)

> Can you please show an example of a failure message?  If there is a
> problem, we need to FIX it!

(I'm trying to send this from my ordinary mail account.  If you can
receive this message, then the failure was temporary.)

Here's a failure report from our SMTP server:

Reporting-MTA: dns; mathmail.math.s.chiba-u.ac.jp
X-Postfix-Queue-ID: 9A4792CCC
X-Postfix-Sender: rfc822; mituharu@math.s.chiba-u.ac.jp
Arrival-Date: Thu, 18 May 2006 10:31:59 +0900 (JST)

Final-Recipient: rfc822; emacs-devel@gnu.org
Action: failed
Status: 5.0.0
Diagnostic-Code: X-Postfix; host mx10.gnu.org[199.232.76.166] said:
    550-Verification failed for <mituharu@math.s.chiba-u.ac.jp> 550-Unrouteable
    address 550 Sender verify failed (in reply to RCPT TO command)

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: %-escapes in file URL

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 20 May 2006 01:04:28 -0400, Richard Stallman <rms@gnu.org> said:

>     A simple and natural policy is to always keep URLs in escaped forms,
>     whether or not they are used inside Emacs only.  So, my proposal is as
>     follows:

>       * Always treat URLs as those in escaped form.  Don't re-escape or
> 	unescape them as long as they are URLs (i.e, have a prefix such as
> 	"file://").  Increasing the kind of escaped characters is not
> 	re-escaping.

>       * When creating a URL from a file name, encode the file name with
> 	the file name coding system and then %-escape it.

>       * When creating a file name from a URL, %-unescape the URL and then
> 	decode it with the file name coding system.

> Do people agree with that proposal?

> What changes are needed to implement it?

Oops, I've already installed the change as Jan agreed it.

  http://lists.gnu.org/archive/html/emacs-devel/2006-05/msg00784.html
  http://lists.gnu.org/archive/html/emacs-devel/2006-05/msg01067.html

P.S.  The email failure turned out to be a temporary one.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

If mail bounces

[Richard Stallman]

Everyone: if your mail to a GNU mailing list bounces,
please report it immediately to sysadmin.

Please don't
 sit on the problem.
Please don't
 complain privately to other Emacs developers.
Please don't
 report it here (unless your mail to sysadmin doesn't
 get through and this is the last resort).
Please do
 report it to the people who can fix it!

Re: %-escapes in file URL

[Richard Stallman]

    > What changes are needed to implement it?

    Oops, I've already installed the change as Jan agreed it.

If Jan agrees with it, then this was probably the right thing to do.
Thanks.

    P.S.  The email failure turned out to be a temporary one.

Ok, I won't tell sysadmin then.