file://host/location URLs

file://host/location URLs

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 373 bytes --]

I found it extremely surprising that a file:// URL can try to FTP into a remote
machine. Can we please remove that behavior? If a user wants to use FTP, he
should be able to use an FTP URL.

In fact, I really don't see why file:// URLs should go through url-handlers.el
at all. A file URL should be a simple pass-through to the normal file handler
machinery, yes?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: file://host/location URLs

[Stefan Monnier]

> I found it extremely surprising that a file:// URL can try to FTP into
> a remote machine.  Can we please remove that behavior?

Please do.

> If a user wants to use FTP, he should be able to use an FTP URL.

Agreed.

> In fact, I really don't see why file:// URLs should go through
> url-handlers.el at all.  A file URL should be a simple pass-through to
> the normal file handler machinery, yes?

Someone has to do the "pass-through", and I think it's OK if that
someone is url-handlers.


        Stefan

Re: file://host/location URLs

[James Cloos]

>>>>> "DC" == Daniel Colascione <dancol@dancol.org> writes:

DC> In fact, I really don't see why file:// URLs should go through
DC> url-handlers.el at all. A file URL should be a simple pass-through
DC> to the normal file handler machinery, yes?

Only file:/// or file://localhost/ should point to the local
filesystem.  file://example.org/foo.bar is a valid uri to some
resource on example.org using some (unspecified) access method.

Whether emacs still should handle URIs of that form, and if so
how, are /interesting/ questions.

(Back in the day, ftp /was/ the correct answer.  Perhaps not any more?)

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: file://host/location URLs

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

On 11/16/2012 4:58 PM, James Cloos wrote:
>>>>>> "DC" == Daniel Colascione <dancol@dancol.org> writes:
> 
> DC> In fact, I really don't see why file:// URLs should go through
> DC> url-handlers.el at all. A file URL should be a simple pass-through
> DC> to the normal file handler machinery, yes?
> 
> Only file:/// or file://localhost/ should point to the local
> filesystem.  file://example.org/foo.bar is a valid uri to some
> resource on example.org using some (unspecified) access method.

It's elegant in theory to make URIs network-transparent, but in practice,
"file:" always refers to a local file. Nobody constructs a file URI intending to
refer to a remote resource not accessible through ordinary filesystem mounts,
and plenty of people accidentally construct file URIs with enough leading
slashes to convince Emacs that a resource is remote. Until I fixed it, our
drag-and-drop code did just that.

If anyone actually uses remote file URIs, please tell me. Otherwise, I intend to
make "file:" and "file://" strictly equivalent and always local.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 258 bytes --]

Re: file://host/location URLs

[Achim Gratz]

Daniel Colascione writes:
> If anyone actually uses remote file URIs, please tell me. Otherwise, I intend to
> make "file:" and "file://" strictly equivalent and always local.

Common bugs in applications and misconceptions of some users
notwithstanding, an URI should implement:

http://tools.ietf.org/html/rfc3986

You may find that your proposal (IIUIC) is violating that specification.
The only instances a file: URI refers to the local file system is if the
authority (host) part is missing or the local host is explicitly given
as the authority.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: file://host/location URLs

[James Cloos]

>>>>> "AG" == Achim Gratz <Stromeko@nexgo.de> writes:

AG> Common bugs in applications and misconceptions of some users
AG> notwithstanding, an URI should implement:

AG> http://tools.ietf.org/html/rfc3986

And rfc 1738 for file://’s definition and syntax.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: file://host/location URLs

[Chong Yidong]

James Cloos <cloos@jhcloos.com> writes:

>>>>>> "AG" == Achim Gratz <Stromeko@nexgo.de> writes:
>
> AG> Common bugs in applications and misconceptions of some users
> AG> notwithstanding, an URI should implement:
>
> AG> http://tools.ietf.org/html/rfc3986
>
> And rfc 1738 for file://’s definition and syntax.

To expand on these points: "file:" and "file://" not equivalent under
RFC 3986, so don't do that.

It seems acceptable to signal an error if a file:// URI specifies a
remote host, instead of using FTP.  But if the hostname is that of the
local computer, it ought to work fine.  For that reason, I think
url-handlers.el is still needed, and that's where the fix needs to be.
Currently, the url library treats file:// and ftp:// urls as synonymous,
but that can be changed.

Re: file://host/location URLs

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

On 11/17/12 12:25 AM, Achim Gratz wrote:
> Daniel Colascione writes:
>> If anyone actually uses remote file URIs, please tell me. Otherwise, I intend to
>> make "file:" and "file://" strictly equivalent and always local.
> 
> Common bugs in applications and misconceptions of some users
> notwithstanding, an URI should implement:
> 
> http://tools.ietf.org/html/rfc3986
> 
> You may find that your proposal (IIUIC) is violating that specification.
> The only instances a file: URI refers to the local file system is if the
> authority (host) part is missing or the local host is explicitly given
> as the authority.

Yes, my proposal violates the RFC. I maintain that nobody deliberately
constructs file URLs pointing to remote hosts, and that the behavior
that best matches user intent is to always interpret file URIs as
local, RFC be damned.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: file://host/location URLs

[Stephen J. Turnbull]

Daniel Colascione writes:

 > Yes, my proposal violates the RFC. I maintain that nobody deliberately
 > constructs file URLs

Shouldn't you put a period here? ;-)  Seriously, I expect that people
who deliberately write file URIs probably know what an RFC is, and may
even have read the relevant ones.  It's not like an http URI where
any Facebook user has at least seen them.

On the contrary, I had an experimental implementation of an URL
handler that used TRAMP to deal with file URLs, and tested it with
deliberately constructed file URLs.  So "nobody" is wrong.  I've seen
such URLs in the wild, used properly though not in a programmatic
context (in a trust group with access to several hosts, indicating use
with SSH or scp, written on paper napkins and the like).

 > pointing to remote hosts,

I disagree.  To me, it's the obvious RFC-standardized way to write an
ssh/scp/sftp URL[1] for fetching a file from a specific host, where
the user of the URL is expected to be able to access the same file
system on that host as the writer, but without constraining the user
to a possibly inconvenient transport protocol.

If you're saying that users sometimes write file://usr/bin/emacs when
they mean either file:///usr/bin/emacs, file:/usr/bin/emacs, or maybe
file:usr/bin/emacs, too bad for those users.  Even on Windows, with a
leading doubled slash the following element indicates the host
providing a service (ie, a namespace authority in URI-speak).

As for the bug in Emacs (and other programs), what else is new?  Emacs
is buggy.  Fix the bug.

 > and that the behavior that best matches user intent is to always
 > interpret file URIs as local, RFC be damned.

And what are you proposing?  Should file://bogus/foo be interpreted as
file:///bogus/foo or as file:bogus/foo?  Or perhaps as file:///foo?

If you want to know what users think, make file://bogus/foo...  error
at parse time if bogus doesn't resolve to the local host.  They'll
tell you.  But it's a bad idea to guess, and a worse idea to take a
precise syntax and make it fuzzy.  Next users will request that such
fuzziness should be allowed for http and other URLs.  After all, it's
obvious what they meant when they wrote http:/www.gnu.org/, right?

By the way, none of the above means that you shouldn't write a
defuzzer program to guess what possibly broken URIs were intended to
be.  Users who can't or won't learn how to write conforming URIs can
install it themselves, and I see nothing wrong with that.

But Emacs itself should follow the RFC here.  The RFC is not broken.


Footnotes: 
[1]  Yes, I'm aware of the *long*-expired IETF draft, though not why
it expired without being approved for RFC status.  Possibly because of
the overlap with the existing file URI?

Re: file://host/location URLs

[Achim Gratz]

Stephen J. Turnbull writes:
> I disagree.  To me, it's the obvious RFC-standardized way to write an
> ssh/scp/sftp URL[1] for fetching a file from a specific host, where
> the user of the URL is expected to be able to access the same file
> system on that host as the writer, but without constraining the user
> to a possibly inconvenient transport protocol.

Yes, and that is what I use them for: if I need to point to a file that
resides on a specific host that I may or may not have direct access to
depending on where I am, then specifying that resource as a non-local
file URI is mightily convenient.  If it resolves to localhost then it
should be a normal file access, if the target file system is already
mounted or reachable via a network file system, then I'd like to use
that access path and if all else fails, then try scp, ssh or ftp or even
avian carrier (see RFC1149/2549).

If there's anything to fix in Emacs then it is its insistence to only
try FTP for those (if I always wanted to use FTP then I'd use an FTP
URI).  If it warned users about obviously malformed URIs and offered
sensible corrections that would also be nice.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: file://host/location URLs

[Jason Rumney]

Daniel Colascione <dancol@dancol.org> writes:

> Yes, my proposal violates the RFC. I maintain that nobody deliberately
> constructs file URLs pointing to remote hosts

I do it all the time.  Windows has a well defined default protocol for
accessing remote files on named hosts, and such URLs mostly work (they
even used to work on Emacs IIRC until url-handler came along and started
forcing them over to FTP).

GNOME, KDE and others may also have mechanisms for dealing with remote
file URLs that we could use.

Re: file://host/location URLs

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1385 bytes --]

On 11/20/2012 4:54 AM, Jason Rumney wrote:
> Daniel Colascione <dancol@dancol.org> writes:
> 
>> Yes, my proposal violates the RFC. I maintain that nobody deliberately
>> constructs file URLs pointing to remote hosts
> 
> I do it all the time.  Windows has a well defined default protocol for
> accessing remote files on named hosts, and such URLs mostly work (they
> even used to work on Emacs IIRC until url-handler came along and started
> forcing them over to FTP).

Yes, the Windows remote access mechanism is good, party because applications
don't need to be aware of it to use it. They can just use normal filesystem
access APIs. If we treat file URIs as simple filenames, remote access under
Windows will continue to work.

> GNOME, KDE and others may also have mechanisms for dealing with remote
> file URLs that we could use.

All right, let me make a different proposal: by default, we'll treat file://
URIs with non-localhost host parts as errors, since FTP is seldom appropriate.
On Windows and Cygwin systems, we'll treat file://foo/bar/qux as
file:///foo/bar/qux since the underlying OS understands how to access host foo.
If someone wants to integrate Emacs with gnome-vfs, he can update the file-URI
code at that time.

File URI handling seems to be generally broken for local files anyway. Try
(find-file "file://localhost/etc/passwd").


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 258 bytes --]

Re: file://host/location URLs

[Achim Gratz]

Daniel Colascione writes:
> All right, let me make a different proposal: by default, we'll treat file://
> URIs with non-localhost host parts as errors, since FTP is seldom
> appropriate.

It should go to a handler that determines the best protocol to use based
on what that authority part is.  If it doesn't find anything appropriate
it can still throw up its hands in despair.  And conditioning the whole
thing on whether the authority resolves to local or not may not be quite
appropriate either; I can be easily arranged so that one can't access a
file via the file system, but still get it via some other protocol.

> On Windows and Cygwin systems, we'll treat file://foo/bar/qux as
> file:///foo/bar/qux since the underlying OS understands how to access
> host foo.

The latter URI is malformed for many clients.  Last I looked you needed
five slashes for this to work consistently (two for the empty authority,
one for root, two for starting a UNC path).  But again, depending on
what the host part is, this may not be the correct thing to do (there
are files I can access over the network, but not via UNC path).

> File URI handling seems to be generally broken for local files anyway. Try
> (find-file "file://localhost/etc/passwd").

Yes, that needs fixing, too.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: file://host/location URLs

[joakim]

Daniel Colascione <dancol@dancol.org> writes:

> On 11/17/12 12:25 AM, Achim Gratz wrote:
>> Daniel Colascione writes:
>>> If anyone actually uses remote file URIs, please tell me. Otherwise, I intend to
>>> make "file:" and "file://" strictly equivalent and always local.
>> 
>> Common bugs in applications and misconceptions of some users
>> notwithstanding, an URI should implement:
>> 
>> http://tools.ietf.org/html/rfc3986> 
>> You may find that your proposal (IIUIC) is violating that specification.
>> The only instances a file: URI refers to the local file system is if the
>> authority (host) part is missing or the local host is explicitly given
>> as the authority.
>
> Yes, my proposal violates the RFC. I maintain that nobody deliberately
> constructs file URLs pointing to remote hosts, and that the behavior
> that best matches user intent is to always interpret file URIs as
> local, RFC be damned.
>
>

In the xwidget branch I use the attached patch to massage the url
entered by the user. It makes browsing more convenient.

Do you think this solution could be expanded to also handle file://
urls? (The url fixup should be opt-in but I havent done that yet)

=== modified file 'lisp/net/browse-url.el'
--- lisp/net/browse-url.el	2012-09-17 05:41:04 +0000
+++ lisp/net/browse-url.el	2012-11-13 01:55:06 +0000
@@ -666,7 +666,7 @@
 ;; functions allows them to be stand-alone commands, making it easier
 ;; to switch between browsers.
 
-(defun browse-url-interactive-arg (prompt)
+(defun browse-url-interactive-arg (prompt &optional default-url)
   "Read a URL from the minibuffer, prompting with PROMPT.
 If `transient-mark-mode' is non-nil and the mark is active,
 it defaults to the current region, else to the URL at or before
@@ -683,7 +683,8 @@
 				      "[\t\r\f\n ]+" ""
 				      (buffer-substring-no-properties
 				       (region-beginning) (region-end))))
-				(browse-url-url-at-point)))
+				(browse-url-url-at-point)
+                                default-url))
 	(not (eq (null browse-url-new-window-flag)
 		 (null current-prefix-arg)))))
 
@@ -790,6 +791,13 @@
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; Browser-independent commands
 
+(defun url-tidy (url)
+  "Tidy up URL as much as possible."
+  (if (equal 0 (string-match ".*://" url))
+      url
+    (concat "http://" url) ;;TODO guess more url forms, like mailto
+    ))
+
 ;; A generic command to call the current browse-url-browser-function
 
 ;;;###autoload
@@ -802,6 +810,7 @@
   (interactive (browse-url-interactive-arg "URL: "))
   (unless (called-interactively-p 'interactive)
     (setq args (or args (list browse-url-new-window-flag))))
+  (setq url (url-tidy url))
   (let ((process-environment (copy-sequence process-environment))
 	(function (or (and (string-match "\\`mailto:" url)
 			   browse-url-mailto-function)

-- 
Joakim Verona

Re: file://host/location URLs

[Andreas Schwab]

joakim@verona.se writes:

> +(defun url-tidy (url)
> +  "Tidy up URL as much as possible."
> +  (if (equal 0 (string-match ".*://" url))

     (if (string-match "\\`[^:]*://" url)

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: file://host/location URLs

[joakim]

Andreas Schwab <schwab@linux-m68k.org> writes:

> joakim@verona.se writes:
>
>> +(defun url-tidy (url)
>> +  "Tidy up URL as much as possible."
>> +  (if (equal 0 (string-match ".*://" url))
>
>      (if (string-match "\\`[^:]*://" url)

Thanks Andreas! (you are of course free to commit to the xwidget branch
if you'd like)

>
> Andreas.

-- 
Joakim Verona

Re: file://host/location URLs

[Stephen J. Turnbull]

James Cloos writes:

 > Only file:/// or file://localhost/ should point to the local
 > filesystem.  file://example.org/foo.bar is a valid uri to some
 > resource on example.org using some (unspecified) access method.
 > 
 > Whether emacs still should handle URIs of that form, and if so
 > how, are /interesting/ questions.

The question is whether file://remote-host/ URIs are out there in the
wild.  (Stefan and Daniel, you guys should be ashamed of yourselves,
users are unlikely to use file: themselves -- so especially in that
situation Emacs should DTRT *for* them.)

 > (Back in the day, ftp /was/ the correct answer.  Perhaps not any more?)

I would hope that "emacs file://host/foo" is equivalent to "ssh host
emacs file:///foo".  For ftpds running in a chroot, that won't be the
case.  So I would prefer scp or ssh to FTP here.  (Why, yes, We Are
Not Human, We Are TRAMPs!)