testing for a remote file to include file on a Windows mapped drive

testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

I have some code that needs to avoid some particular processing when the
current input might name a remote file. I don't want the code to try to
contact anything remote in this case.

In Windows, you can map a remote directory to a virtual drive. So, for
instance, "h:foo" or "h:/foo" might represent a remote file. But the same
name would represent a local file if drive `h' is local, not mapped.

How can I test in Emacs, for Windows, whether a drive is local or mapped, in
order to determine whether "h:foo" might be local or remote?

AFAICT, the various ways Emacs uses to test file-name syntax to determine
remoteness do not take Windows drive mapping into account. I don't know if
that's a problem for Emacs in general (why wouldn't it be?), but it is a
problem for my use case.

I've looked at these. None does what I need.

. file-remote-p
. ffap-file-remote-p
. dired-sort-menu-remote-p
  "\\`/\\([^@:/]+@\\)?\\([^:/]+\\):"
. ange-ftp-name-format
  "\\`/\\(\\([^/:]*\\)@\\)?\\([^@/:]*[^@/:.]\\):\\(.*\\)"
. remote (ange-ftp) filenames (see make-auto-save-file-name)
  "^/\\w+@[-A-Za-z0-9._]+:"
. ffap-ftp-regexp
  "\\`/[^/:]+:"

Suggestions?

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

I should have mentioned that all I've come up with for this so far is this:

(and (memq system-type '(ms-dos windows-nt))
     (string-match "[^/]?:" input))

But of course a file name could contain a colon (:), which would give false
positives. Still, this is better than nothing.

Is there some way to ask Windows (from Emacs) whether a drive letter is
mapped or local?

RE: testing for a remote file to include file on a Windows mappeddrive

[Drew Adams]

> Is there some way to ask Windows (from Emacs) whether a drive letter is
> mapped or local?

In the absence of any suggestion, I'm thinking of providing a user option in
my code that specifies the local drive letters, defaulting to, say, A
through I.

Googling showed that local drive vs mapped network drive can be tested, in
C# at least (using GetDriveType() or DriveInfo.DriveType). I didn't come
across a way to test it from C, however.

It would be great if Emacs were somehow to provide a predicate for this. In
spite of its name, `file-remote-p' does not consider files on mapped network
drives to be remote. They are perhaps not remote in the original Emacs sense
of the word (requiring remote protocol access, such as FTP), but they are
remote in terms of access time. And many of the uses of `file-remote-p' are
concerned about access time.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> I have some code that needs to avoid some particular processing when the
> current input might name a remote file. I don't want the code to try to
> contact anything remote in this case.

> In Windows, you can map a remote directory to a virtual drive. So, for
> instance, "h:foo" or "h:/foo" might represent a remote file. But the same
> name would represent a local file if drive `h' is local, not mapped.

The same problem exists under GNU/Linux and friends.
~/foo might be a mount point and all files underneath are access via NFS
or SSHFS (or whatever else).

Under GNU/Linux, I believe there are C-level primitives that can help
figure out whether a file is really local or not, so we could use those
to improve file-remote-p.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Jason Rumney]

Drew Adams wrote:
> Googling showed that local drive vs mapped network drive can be tested, in
> C# at least (using GetDriveType() or DriveInfo.DriveType). I didn't come
> across a way to test it from C, however.
>   

GetDriveType can be called from C. Emacs already uses it in w32.c, but 
only to determine which drives are fixed, thus their information can be 
cached indefinitely.

RE: testing for a remote file to include file on a Windows mappeddrive

[Drew Adams]

> > Googling showed that local drive vs mapped network drive can be
> > tested, in C# at least (using GetDriveType() or
> > DriveInfo.DriveType). I didn't come
> > across a way to test it from C, however.
>
> GetDriveType can be called from C. Emacs already uses it in w32.c, but
> only to determine which drives are fixed, thus their information can be
> cached indefinitely.

Good. Do you agree that it would be good for either `file-remote-p' or some
new function to take mapped network drives into account for remote files?

It might be that some uses of `file-remote-p' should not consider a file on
a network drive to be remote - I don't know.

Also, IIUC, `file-remote-p' isn't only boolean; the non-nil return value
means something. I don't know what should be returned for a file on a mapped
network drive.

And should we perhaps have a function that distinguishes remote files that
are not on a mapped drive from those that are? Is there any need to
distinguish these? (We currently do distinguish them, in that
`file-remote-p' returns non-nil for the former and nil for the latter.)

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> Also, IIUC, `file-remote-p' isn't only boolean; the non-nil return value
> means something. I don't know what should be returned for a file on a mapped
> network drive.

I'm not thrilled about the non-boolean return value of file-remote-p.
Its semantics seem to be custom-tailored for Tramp and doesn't make so
much sense in general :-(

> And should we perhaps have a function that distinguishes remote files that
> are not on a mapped drive from those that are? Is there any need to
> distinguish these? (We currently do distinguish them, in that
> `file-remote-p' returns non-nil for the former and nil for the latter.)

It all depends what you want to know.  In doc-view, we recently tried to
use file-remote-p in order to figure out whether we could pass the file
name to a local process, only to discover that it wasn't the right
predicate to use.  I replaced it later on with
(let ((file-name-handlers-alist nil)) (file-readable-p <FILE>)).
I think file-remote-p should really give a kind of performance and
reliability info: is the file available from local (i.e. fast and
reliable) storage, or is it only available via a network access which
may be slow and may fail.
Now whether access via NFS should be considered as remote or not is not
clear: it's ofen used for home directories, and treating all the user's
file as "remote" is probably not a great idea.


        Stefan

RE: testing for a remote file to include file on a Windows mappeddrive

[Drew Adams]

> > Also, IIUC, `file-remote-p' isn't only boolean; the non-nil return value
> > means something. I don't know what should be returned for a
> > file on a mapped network drive.
>
> I'm not thrilled about the non-boolean return value of file-remote-p.
> Its semantics seem to be custom-tailored for Tramp and doesn't make so
> much sense in general :-(

Perhaps that other info should be relegated to some other function. Or
perhaps it should be local to Tramp, if it is not useful elsewhere.

At least for the use of `file-remote-p' that is to avoid possibly expensive
remote access, any info that might take more time to dig up should be left
out.

> > And should we perhaps have a function that distinguishes remote
> > files that are not on a mapped drive from those that are? Is
> > there any need to distinguish these? (We currently do distinguish
> > them, in that `file-remote-p' returns non-nil for the former and
> > nil for the latter.)
>
> It all depends what you want to know.  In doc-view, we recently tried to
> use file-remote-p in order to figure out whether we could pass the file
> name to a local process, only to discover that it wasn't the right
> predicate to use.  I replaced it later on with
> (let ((file-name-handlers-alist nil)) (file-readable-p <FILE>)).
>
> I think file-remote-p should really give a kind of performance and
> reliability info: is the file available from local (i.e. fast and
> reliable) storage, or is it only available via a network access which
> may be slow and may fail.

Yes, I think that is the most common use of `file-remote-p', in any case.
Whether that is, or needs to be, a true test of remoteness is another
question. If not, the name could perhaps be changed.

Typically, I think what's wanted in that use case is to just get an idea
whether the file name is likely to represent a remote file - info as to
whether or not the file is in fact remote is often not needed. For this
common use case, the function is in fact a sort of
`file-might-take-a-while-to-access-p' - it is not the remoteness that is as
important as the potential access time.

For example, if it takes actually connecting remotely to find out for sure
whether some file exists remotely, that is not appropriate for many current
uses of `file-remote-p', IIUC. At least, that was the use case I had in
mind: I don't want the function to take any extra time to find out for sure;
I just want to know if it is likely to be remote, in which case I will skip
certain operations.

> Now whether access via NFS should be considered as remote or not is not
> clear: it's ofen used for home directories, and treating all the user's
> file as "remote" is probably not a great idea.

I agree that the different notions of "remote" should be clarified, and one
should be able to obtain each of them that represents an important use case.

It is definitely useful to be able to distinguish "possibly remote" from
"local" in terms of potential performance impact - that is, regardless of
whether the meaning is truly remote vs local. I imagine that some of the
other distinctions might also be useful. Your use of `file-readable-p' for
doc-view would presumably be one of those.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 25 Jan 2008 17:11:45 -0500
> Cc: Emacs-Devel <emacs-devel@gnu.org>
> 
> Under GNU/Linux, I believe there are C-level primitives that can help
> figure out whether a file is really local or not, so we could use those
> to improve file-remote-p.

Similar primitives exist on MS-Windows.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Fri, 25 Jan 2008 10:54:36 -0800
> 
> I should have mentioned that all I've come up with for this so far is this:
> 
> (and (memq system-type '(ms-dos windows-nt))
>      (string-match "[^/]?:" input))

The MS-DOS port does not support features where such remote files come
into existence (ange-ftp, tramp, etc.), so it's useless to include it
here.

Re: testing for a remote file to include file on a Windows mappeddrive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Fri, 25 Jan 2008 23:30:54 -0800
> Cc: Emacs-Devel <emacs-devel@gnu.org>, Jason Rumney <jasonr@gnu.org>
> 
> > I think file-remote-p should really give a kind of performance and
> > reliability info: is the file available from local (i.e. fast and
> > reliable) storage, or is it only available via a network access which
> > may be slow and may fail.
> 
> Yes, I think that is the most common use of `file-remote-p', in any case.
> Whether that is, or needs to be, a true test of remoteness is another
> question. If not, the name could perhaps be changed.

No need to change the name: we could add an optional argument to
file-remote-p.

> I just want to know if it is likely to be remote, in which case I will skip
> certain operations.

What operations are those?  You never actually described the specific
problem you are trying to solve, did you?

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Fri, 25 Jan 2008 10:47:51 -0800
> 
> I have some code that needs to avoid some particular processing when the
> current input might name a remote file. I don't want the code to try to
> contact anything remote in this case.
> 
> In Windows, you can map a remote directory to a virtual drive. So, for
> instance, "h:foo" or "h:/foo" might represent a remote file. But the same
> name would represent a local file if drive `h' is local, not mapped.
> 
> How can I test in Emacs, for Windows, whether a drive is local or mapped, in
> order to determine whether "h:foo" might be local or remote?

Does it help to run "net use" and parse its output?

Note that "net use" itself might be somewhat slow, but maybe your
undisclosed processing would be much slower than that, in which case
"net use" is justified.

> I've looked at these. None does what I need.
> 
> . file-remote-p
> . ffap-file-remote-p
> . dired-sort-menu-remote-p
>   "\\`/\\([^@:/]+@\\)?\\([^:/]+\\):"
> . ange-ftp-name-format
>   "\\`/\\(\\([^/:]*\\)@\\)?\\([^@/:]*[^@/:.]\\):\\(.*\\)"
> . remote (ange-ftp) filenames (see make-auto-save-file-name)
>   "^/\\w+@[-A-Za-z0-9._]+:"
> . ffap-ftp-regexp
>   "\\`/[^/:]+:"

These all make sense only in the context of the features that define
them.  In general, Emacs does not consider files on a mounted volume
to be remote.  The notion of remote files is in conjunction with Emacs
features that access files in ways that are not the normal filesystem
APIs.

It sounds like you want something entirely different, but it's hard to
say what that is without knowing what are you trying to accomplish,
and why applying that to remote files would be wrong.

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> (let ((file-name-handlers-alist nil)) (file-readable-p <FILE>)).

(and (not (file-remote-p file)) (file-readable-p file))

There could be file name handlers, which have an own implementation of
file-readable-p for local files.

> I think file-remote-p should really give a kind of performance and
> reliability info: is the file available from local (i.e. fast and
> reliable) storage, or is it only available via a network access which
> may be slow and may fail.

file-remote-p returns information about the *file name*, and not
necessarily about the file: "Can the file name be used literally on the
local host?" I believe this meaning shall be kept, for example, it is
needed to determine whether a file name must be trimmed when used as
argument of process-file and alike.

The new function you are looking for is file-mounted-p or so.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Jason Rumney]

Stefan Monnier wrote:
> It all depends what you want to know.  In doc-view, we recently tried to
> use file-remote-p in order to figure out whether we could pass the file
> name to a local process, only to discover that it wasn't the right
> predicate to use.
It seems there are two uses for such a predicate. One is to find out if 
local processes can be used on the file, the other to find out if the 
connection may be slow or unavailable. The current version of the 
function doesn't really cater to either use properly.

Re: testing for a remote file to include file on a Windows mappeddrive

[Jason Rumney]

Michael Albinus wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>   
>> (let ((file-name-handlers-alist nil)) (file-readable-p <FILE>)).
>>     
>
> (and (not (file-remote-p file)) (file-readable-p file))
>
> There could be file name handlers, which have an own implementation of
> file-readable-p for local files.
>   

If you are running an external process which needs to read the file, 
then it needs to be readable without any file-handler specifics.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > . file-remote-p
> > . ffap-file-remote-p
> > . dired-sort-menu-remote-p
> >   "\\`/\\([^@:/]+@\\)?\\([^:/]+\\):"
> > . ange-ftp-name-format
> >   "\\`/\\(\\([^/:]*\\)@\\)?\\([^@/:]*[^@/:.]\\):\\(.*\\)"
> > . remote (ange-ftp) filenames (see make-auto-save-file-name)
> >   "^/\\w+@[-A-Za-z0-9._]+:"
> > . ffap-ftp-regexp
> >   "\\`/[^/:]+:"
>
> These all make sense only in the context of the features that define
> them.  In general, Emacs does not consider files on a mounted volume
> to be remote.  The notion of remote files is in conjunction with Emacs
> features that access files in ways that are not the normal filesystem
> APIs.
>
> It sounds like you want something entirely different

As I said, I want to know if a file name is likely to represent a remote
file (in the Emacs sense you cited) or a file on a mapped network drive.

The reason is to avoid the time needed to access the remote machine. That's
the same reason that most of the above were defined, but they don't handle
the network-drive case.

RE: testing for a remote file to include file on a Windows mappeddrive

[Drew Adams]

> file-remote-p returns information about the *file name*, and not
> necessarily about the file: "Can the file name be used literally on the
> local host?" I believe this meaning shall be kept, for example, it is
> needed to determine whether a file name must be trimmed when used as
> argument of process-file and alike.
>
> The new function you are looking for is file-mounted-p or so.

In my case, and I think in many (most?) cases where `file-remote-p' is used
today, it is, as you say, not necessary to know much about an actual file;
it is enough to know that a given file name is *likely* to represent a
remote file.

Note that it is possible that a local file have a name that matches any of
the various "remote-file" syntaxes. The syntax is not proof that a file is
remote.

In my case, I am not really looking for `file-mounted-p'. Again, I don't
really care if such and such a file is really local or remote or on a
mounted (mapped) drive or directory. I only want to know if it is *likely*
that it is. That means that I don't need a (possibly more costly) function
to check things out remotely to determine whether such a remote/mounted file
exists.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <emacs-devel@gnu.org>
> Date: Sat, 26 Jan 2008 08:55:48 -0800
> 
> > These all make sense only in the context of the features that define
> > them.  In general, Emacs does not consider files on a mounted volume
> > to be remote.  The notion of remote files is in conjunction with Emacs
> > features that access files in ways that are not the normal filesystem
> > APIs.
> >
> > It sounds like you want something entirely different
> 
> As I said, I want to know if a file name is likely to represent a remote
> file (in the Emacs sense you cited) or a file on a mapped network drive.
> 
> The reason is to avoid the time needed to access the remote machine.

What kind of access is that, and how much time does it take, that you
want to avoid it?  Do you have any numbers, even approximate ones,
that compare access to local vs remote files on mounted volumes?

> That's the same reason that most of the above were defined, but they
> don't handle the network-drive case.

Accessing a remote file via the normal filesystem API (which is what
happens with remote drives and NFS-mounted volumes) is several orders
of magnitude faster than Tramp or ange-ftp.  So I find it hard to
believe that the same primitive would fulfill both of these needs.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > As I said, I want to know if a file name is likely to represent a remote
> > file (in the Emacs sense you cited) or a file on a mapped network drive.
> >
> > The reason is to avoid the time needed to access the remote machine.
>
> What kind of access is that, and how much time does it take, that you
> want to avoid it?  Do you have any numbers, even approximate ones,
> that compare access to local vs remote files on mounted volumes?

No.

> > That's the same reason that most of the above were defined, but they
> > don't handle the network-drive case.
>
> Accessing a remote file via the normal filesystem API (which is what
> happens with remote drives and NFS-mounted volumes) is several orders
> of magnitude faster than Tramp or ange-ftp.  So I find it hard to
> believe that the same primitive would fulfill both of these needs.

I didn't say anything about using "the same primitive". I explicitly said
that a different function might be provided. I'm indifferent as to whether
`file-remote-p' should be tweaked to take this into account or a new
function should be added. I and others have pointed out that `file-remote-p'
is already ambiguous in terms of its aim or use. It would be good to clarify
things and sort this out.

In the case of a file on a mapped network drive, it takes just as long, for
my setup at least, in terms of perception. I have no numbers, and I don't
need any. It is too long for me (and for people who reported the problem to
me). Today, AFAIK, there is no test for this, and `file-remote-p' returns
nil for such a file. Code that uses `file-remote-p' (or any of the regexps I
cited) does not avoid the performance hit from trying to access a file on a
network drive.

Perhaps the performance depends also on the particular remote machine and
its connectivity, VPN, firewall, etc. I don't know, and I don't care. For
me, it does no good to argue that accessing a mapped network drive is
"several orders of magnitude" faster than using Tramp or FTP. It is not
*noticeably* faster in my case, and, more importantly, it is much, much
slower for me than accessing a local file. No numbers; only perception. It
might be orders of magnitude faster, but it is too slow for me.

Trust me - If a file name is likely to represent a file that is on a network
drive, then I usually want to treat it as the name of a remote file in terms
of avoiding unnecessary access. That's all.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <emacs-devel@gnu.org>
> Date: Sat, 26 Jan 2008 09:57:17 -0800
> 
> > Accessing a remote file via the normal filesystem API (which is what
> > happens with remote drives and NFS-mounted volumes) is several orders
> > of magnitude faster than Tramp or ange-ftp.  So I find it hard to
> > believe that the same primitive would fulfill both of these needs.
> 
> I didn't say anything about using "the same primitive".

Well, you repeatedly talked about file-remote-p, so it's easy to err
like I did.  Sorry.

> Perhaps the performance depends also on the particular remote machine and
> its connectivity, VPN, firewall, etc. I don't know, and I don't care. For
> me, it does no good to argue that accessing a mapped network drive is
> "several orders of magnitude" faster than using Tramp or FTP. It is not
> *noticeably* faster in my case, and, more importantly, it is much, much
> slower for me than accessing a local file. No numbers; only perception. It
> might be orders of magnitude faster, but it is too slow for me.
> 
> Trust me - If a file name is likely to represent a file that is on a network
> drive, then I usually want to treat it as the name of a remote file in terms
> of avoiding unnecessary access. That's all.

You seem to perceive my questions as some kind of questioning the
validity of your reports.  That is a far cry from what I meant.

If this problem is something specific to your setup, you could devise
your own private solution: after all, you likely know what drives are
connected to remote machines in your case.  (Btw, you never responded
to my suggestion to try the output of "net use".)

But if you are asking for an Emacs primitive to solve this, then
please do at least some research and provide the details and
quantitative results about the problem.  At the very least, those
numbers will be used when various possible solutions are evaluated by
those who might work on this problem, as the baseline to measure the
speedup of those solutions.  It is also quite probable that we will
want to compare your results with ours, as we try to find the
reason(s) for the slowdown.  "Trust me" will not help us do that.

While you might not care about the reasons, we who maintain the
Windows port surely do care, because without understanding those
reasons, we cannot devise the right solution.  For example, while you
think that the reason is the fact that these files are on a remote
machine, the real reason might be something entirely different, in
which case the GetDriveType call is not going to solve the problem for
you.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>> (let ((file-name-handlers-alist nil)) (file-readable-p <FILE>)).

> (and (not (file-remote-p file)) (file-readable-p file))

> There could be file name handlers, which have an own implementation of
> file-readable-p for local files.

>> I think file-remote-p should really give a kind of performance and
>> reliability info: is the file available from local (i.e. fast and
>> reliable) storage, or is it only available via a network access which
>> may be slow and may fail.

> file-remote-p returns information about the *file name*, and not
> necessarily about the file: "Can the file name be used literally on the
> local host?" I believe this meaning shall be kept, for example, it is
> needed to determine whether a file name must be trimmed when used as
> argument of process-file and alike.

> The new function you are looking for is file-mounted-p or so.

I believe you're wrong and I'm right ;-)

What you consider as the meaning of `file-remote-p' is pretty close to
the info returned by (let ((file-name-handlers-alist nil)) (not
(file-readable-p <FILE>))), whereas what you want to call
`file-mounted-p' is much closer to what `file-remote-p' currently does.

E.g. (file-remote-p "/:/") will return nil, even though passing "/:/" as
an argument to a subprocess will likely fail to do what was intended.

After all, whether the filename can be passed to a subprocess has
nothing to do with whether or not it's "remote".

I suggest we clarify the meaning of file-remote-p to explain that it
returns info that relates to the time and reliability of the access to
this file.  And we introduce a new one `unhandled-file-name' which
returns either "an equivalent name that can passed to subprocesses"
(e.g. for the "/:/" file-name-handler, it just strips the leading "/:"
and similarly for the "file://" one) or nil if the file cannot be
directly accessed by a subprocess (either because it's only accessible
via ssh/ftp/tar/younameit it by a file-name-handler).


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> file-remote-p returns information about the *file name*, and not
>> necessarily about the file: "Can the file name be used literally on the
>> local host?" I believe this meaning shall be kept, for example, it is
>> needed to determine whether a file name must be trimmed when used as
>> argument of process-file and alike.
>
>> The new function you are looking for is file-mounted-p or so.
>
> I believe you're wrong and I'm right ;-)

For sure. At least it has been the case almost ever when we did discuss.

> What you consider as the meaning of `file-remote-p' is pretty close to
> the info returned by (let ((file-name-handlers-alist nil)) (not
> (file-readable-p <FILE>))), whereas what you want to call
> `file-mounted-p' is much closer to what `file-remote-p' currently does.
>
> E.g. (file-remote-p "/:/") will return nil, even though passing "/:/" as
> an argument to a subprocess will likely fail to do what was intended.
>
> After all, whether the filename can be passed to a subprocess has
> nothing to do with whether or not it's "remote".

Hmm. I haven't said that the result of `file-remote-p' is the only check
how to pass a file name as argument to a subprocess. But when
`file-remote-p' returns non-nil, it means that you can't take the
related file literally as part of ARGS in 'process-file' and friends.

> I suggest we clarify the meaning of file-remote-p to explain that it
> returns info that relates to the time and reliability of the access to
> this file.  And we introduce a new one `unhandled-file-name' which
> returns either "an equivalent name that can passed to subprocesses"
> (e.g. for the "/:/" file-name-handler, it just strips the leading "/:"
> and similarly for the "file://" one) or nil if the file cannot be
> directly accessed by a subprocess (either because it's only accessible
> via ssh/ftp/tar/younameit it by a file-name-handler).

At least for remote files I've introduced it silently ...
(file-remote-p "/sudo::/etc" 'localname) returns "/etc". But you are
right, `unhandled-file-name' would be more general but just for Tramp.

But we shall come back to the original request of Drew: he wants to know
whether a given file is "remote", independant of the syntax. Such
information cannot be provided by 'file-remote-p', because that function
has an important promise: "`file-remote-p' will never open a connection
on its own." (from the doc-string). It also doesn't touch a local file;
it works completely over the file name syntax. This promise is important
because of performance.

Rough idea: maybe one could use the device number, returned as last
attribute of `file-attributes', for a check whether a file has been
"mounted". I haven't seen code evaluating the device number.  And I
don't know whether it is set to a useful value on W32, for example.

Just for the records, ange-ftp sets it to -1, and Tramp returns (-1 x)
with x being a unique number for different remote connections.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Sat, 26 Jan 2008 23:15:08 +0100
> Cc: Jason Rumney <jasonr@gnu.org>, Drew Adams <drew.adams@oracle.com>,
> 	Emacs-Devel <emacs-devel@gnu.org>
> 
> Rough idea: maybe one could use the device number, returned as last
> attribute of `file-attributes', for a check whether a file has been
> "mounted". I haven't seen code evaluating the device number.  And I
> don't know whether it is set to a useful value on W32, for example.

Maybe I'm missing something, but how can the device number be used,
even on Posix platforms, to check if the file is on a mounted volume?
Can you show sample code?

RE: testing for a remote file to include file on a Windows mappeddrive

[Drew Adams]

> But we shall come back to the original request of Drew: he wants to know
> whether a given file is "remote", independant of the syntax. Such
> information cannot be provided by 'file-remote-p', because that function
> has an important promise: "`file-remote-p' will never open a connection
> on its own." (from the doc-string). It also doesn't touch a local file;
> it works completely over the file name syntax. This promise is important
> because of performance.

I'm not looking for something that needs to be "independent of the syntax".
It could use the syntax sometimes, but it might need to go beyond the syntax
sometimes. On Windows, it would need to be able to tell when "h:/foo" might
name a file on a network drive, and the syntax alone won't suffice for that.
I think Stefan suggested something similar for symbolic links on other
platforms.

I'm looking for something that will, as you say, "never open a connection on
its own". I'm interested in saving time. I want to distinguish a string that
is likely to name a remote file or a file on a mapped network drive from a
string that is likely to name a local file. That's all.

The problem I posed was that I couldn't find a function that makes that
distinction. `file-remote-p', in particular, returns nil for a file that is
on a mapped network drive.

I don't care what the function is called - whether (a) `file-remote-p' is
tweaked to return non-nil for a file on a network drive or (b) some other
function is provided; either would be fine by me.

FWIW, I am currently using Eli's "NET USE" suggestion (thanks). It
distinguishes network drives from local drives and non-existent drives,
which is sufficient for my use. (AFAICT, it does not distinguish a local
drive from a non-existent drive, but that's OK.) This is what I have now:

(defun my-file-remote-p (file)
  "Non-nil means FILE is likely to name a file on a remote system.
For MS Windows, this includes a file on a mapped network drive."
  (or (and (eq system-type 'windows-nt)
           (let ((case-fold-search t))
             (and (string-match "\\`\\([a-z]:\\)" file)
                  (eq 0 (condition-case nil
                            (call-process
                             shell-file-name nil nil nil
                             shell-command-switch
                             (concat "NET USE "
                                     (match-string 1 file)))
                          (error nil))))))
      (and (fboundp 'ffap-file-remote-p)
           (ffap-file-remote-p file))
      (file-remote-p file)))

I'm not necessarily suggesting this for Emacs; I'm just saying that it seems
to work for me.

BTW: Can `file-remote-p' ever return non-nil when `ffap-remote-p' returns
nil? If not, I'll remove the `ffap-remote-p' test. My guess is that neither
is a proper subset of the other, so I use both.

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> Rough idea: maybe one could use the device number, returned as last
>> attribute of `file-attributes', for a check whether a file has been
>> "mounted". I haven't seen code evaluating the device number.  And I
>> don't know whether it is set to a useful value on W32, for example.
>
> Maybe I'm missing something, but how can the device number be used,
> even on Posix platforms, to check if the file is on a mounted volume?
> Can you show sample code?

Not Elisp code, but see this from the shell (on my Ubuntu machine):

I get all filesystems per name and type by this:

arthur:~> cut -f2 -d" " /proc/mounts | xargs stat -f -c "-%n %T" 
-/ ext2/ext3
-/sys sysfs
-/proc proc
-/dev tmpfs
-/sys/fs/fuse/connections UNKNOWN (0x65735543)
-/ ext2/ext3
-/dev/.static/dev ext2/ext3
-/var/run tmpfs
-/var/lock tmpfs
-/lib/modules/2.6.22-14-generic/volatile tmpfs
-/dev/shm tmpfs
-/dev/pts devpts
-/var/run tmpfs
-/var/lock tmpfs
-/sys/kernel/security UNKNOWN (0x73636673)
-/proc/sys/fs/binfmt_misc UNKNOWN (0x42494e4d)
-/mnt/albinus UNKNOWN (0x65735546)
-/mnt/src UNKNOWN (0x65735546)

I get the device numbers of the filesystems by this:

arthur:~> cut -f2 -d" " /proc/mounts | xargs stat -c "%n %d"
/ 2049
/sys 0
/proc 3
/dev 14
/sys/fs/fuse/connections 16
/ 2049
/dev/.static/dev 2049
/var/run 18
/var/lock 19
/lib/modules/2.6.22-14-generic/volatile 20
/dev/shm 21
/dev/pts 11
/var/run 18
/var/lock 19
/sys/kernel/security 15
/proc/sys/fs/binfmt_misc 22
/mnt/albinus 23
/mnt/src 24

In Emacs, I evaluate

(nth 11 (file-attributes "~/src/emacs/lisp"))
 => 2049
(nth 11 (file-attributes "~/src/tramp/lisp"))
 => 24

"~/src/emacs" is a link to "/usr/local/src/emacs". The device number
2049 is related to device "/", which has the device type "ext2/ext3".

"~/src/tramp" is a link to "/mnt/src/tramp". The device number 24 is
related to device "/mnt/src", which has the device type "UNKNOWN".

These information (and other ones, if necessary) might be sufficient to
determine whether a given file is "mounted locally". Implementation
shall happen in Emacs C core,of course.

For remote files (in the sense of Tramp) it is obvious:

(nth 11 (file-attributes "/ssh::~/src/tramp/lisp"))
 => (-1 0)
(nth 11 (file-attributes "/ssh::~/src/emacs/lisp"))
 => (-1 0)

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: monnier@iro.umontreal.ca,  jasonr@gnu.org,  drew.adams@oracle.com,  emacs-devel@gnu.org
> Date: Sun, 27 Jan 2008 12:18:43 +0100
> 
> (nth 11 (file-attributes "~/src/emacs/lisp"))
>  => 2049
> (nth 11 (file-attributes "~/src/tramp/lisp"))
>  => 24
> 
> "~/src/emacs" is a link to "/usr/local/src/emacs". The device number
> 2049 is related to device "/", which has the device type "ext2/ext3".
> 
> "~/src/tramp" is a link to "/mnt/src/tramp". The device number 24 is
> related to device "/mnt/src", which has the device type "UNKNOWN".
> 
> These information (and other ones, if necessary) might be sufficient to
> determine whether a given file is "mounted locally". Implementation
> shall happen in Emacs C core,of course.

Are you saying that files with device number below a certain threshold
are guaranteed to be on a remote mounted volume?  If so, what is that
threshold, and is this universal among Posix platforms?

Also, wouldn't file-attributes (which calls `stat' internally) try to
access the mounted volume, and thus be as slow as the file access we
want to avoid?  For example, I thought that a call to `stat' would try
to mount automounted volumes, but perhaps I'm confused.

Re: testing for a remote file to include file on a Windows mappeddrive

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> Are you saying that files with device number below a certain threshold
> are guaranteed to be on a remote mounted volume?

That is definitely not true.  There is no inherent relationship between
device numbers and "remote mounted" (whatever that means) volumes.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> Are you saying that files with device number below a certain threshold
> are guaranteed to be on a remote mounted volume?  If so, what is that
> threshold, and is this universal among Posix platforms?

No. I'm just saying that a device number identifies a file system
uniquely. And it is good for retrieving further information.

> Also, wouldn't file-attributes (which calls `stat' internally) try to
> access the mounted volume, and thus be as slow as the file access we
> want to avoid?  For example, I thought that a call to `stat' would try
> to mount automounted volumes, but perhaps I'm confused.

That I don't know either. Shall be checked. I do it tomorrow.

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>> I suggest we clarify the meaning of file-remote-p to explain that it
>> returns info that relates to the time and reliability of the access to
>> this file.  And we introduce a new one `unhandled-file-name' which
>> returns either "an equivalent name that can passed to subprocesses"
>> (e.g. for the "/:/" file-name-handler, it just strips the leading "/:"
>> and similarly for the "file://" one) or nil if the file cannot be
>> directly accessed by a subprocess (either because it's only accessible
>> via ssh/ftp/tar/younameit it by a file-name-handler).

> At least for remote files I've introduced it silently ...
> (file-remote-p "/sudo::/etc" 'localname) returns "/etc". But you are
> right, `unhandled-file-name' would be more general but just for Tramp.

> But we shall come back to the original request of Drew: he wants to
> know whether a given file is "remote", independant of the syntax.
> Such information cannot be provided by 'file-remote-p', because that
> function has an important promise: "`file-remote-p' will never open
> a connection on its own." (from the doc-string).

That's indeed a good guaranteed, and I don't see it as an obstacle.
After all, if you'd need to contact a remote host to find out, then the
object can't be local.

> It also doesn't touch a local file; it works completely over the file
> name syntax. This promise is important because of performance.

It's not promised explicitly, tho I'm not sure what you mean by "touch".
In any case, looking at the file's contents can't be of much use.

> Rough idea: maybe one could use the device number, returned as last
> attribute of `file-attributes', for a check whether a file has been
> "mounted". I haven't seen code evaluating the device number.  And I
> don't know whether it is set to a useful value on W32, for example.

I don't think you can get any reliable useful info from the
device number.

Anyway, any objection to my proposal to clarify file-remote-p's intended
semantics, and to add unhandled-file-name (tho maybe it should be
called file-local-name, to make it closer to file-local-copy which is
kind of its complement)?


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> I think Stefan suggested something similar for symbolic links on other
> platforms.

No, not for symbolic links, for mount points.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Anyway, any objection to my proposal to clarify file-remote-p's intended
> semantics, and to add unhandled-file-name (tho maybe it should be
> called file-local-name, to make it closer to file-local-copy which is
> kind of its complement)?

Just for the records: I don't object. `file-local-name' sounds
good. And I'm also willing to provide the implementation for both
ange-ftp and Tramp, which is simple enough ...

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Richard Stallman]

`file-local-name' might not be a good name.

The operation `unhandled-file-name-directory' returns a directory that
isn't magic.  But remote names are not the only magic names, so
"local" isn't necessarily the right way to describe the relationship
between that directory and the argument.

I think the same goes for `unhandled-file-name'.

Perhaps `nonmagic-file-name' would be a better name.

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Richard Stallman <rms@gnu.org> writes:

> `file-local-name' might not be a good name.
>
> The operation `unhandled-file-name-directory' returns a directory that
> isn't magic.  But remote names are not the only magic names, so
> "local" isn't necessarily the right way to describe the relationship
> between that directory and the argument.
>
> I think the same goes for `unhandled-file-name'.
>
> Perhaps `nonmagic-file-name' would be a better name.

As usual, I don't care function names too much. But the mapping

  (nonmagic-file-name "/sudo::") => "/root"

doesn't seem to be obvious, although it would be the implementation
for Tramp.

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> As usual, I don't care function names too much. But the mapping

>   (nonmagic-file-name "/sudo::") => "/root"

> doesn't seem to be obvious, although it would be the implementation
> for Tramp.

Actually, this implementation would be incorrect for my purpose
(doc-view): the important aspect of unhandled-file-name is that
a subprocess will be able to use it.  So it should be accessible by the
normal Emacs user.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> `file-local-name' might not be a good name.
> The operation `unhandled-file-name-directory' returns a directory that
> isn't magic.  But remote names are not the only magic names, so
> "local" isn't necessarily the right way to describe the relationship
> between that directory and the argument.

The issue is not whether the returned file name is magic or not, but
whether it can be passed as-is to a subprocess.

> I think the same goes for `unhandled-file-name'.

I don't understand.  unhandled-file-name-directory is used to get
a directory that can be used as cwd for subprocesses.  And the new
operation is meant to be used to get a filename that can be passed
to subprocesses.  So AFAICT they share everything and should use the
same name.

> Perhaps `nonmagic-file-name' would be a better name.

The jka-compr implementation of this new operation should return the
file name unchanged, so it would return a filename that's still as magic
as before.  Hence nonmagic-file-name doesn't sound right.  But the main
reason why I do not like it is because it introduces a new word
("nonmagic") to describe the same concept as "unhandled" in
unhandled-file-name-directory.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>> I think the same goes for `unhandled-file-name'.

> I don't understand.  unhandled-file-name-directory is used to get
> a directory that can be used as cwd for subprocesses.  And the new
> operation is meant to be used to get a filename that can be passed
> to subprocesses.  So AFAICT they share everything and should use the
> same name.

And actually the default implementation of unhandled-file-name-directory
should be changed from

  /* If the file name has special constructs in it,
     call the corresponding file handler.  */
  handler = Ffind_file_name_handler (filename, Qunhandled_file_name_directory);
  if (!NILP (handler))
    return call2 (handler, Qunhandled_file_name_directory, filename);
  return Ffile_name_directory (filename);

to

  Lisp_Object ufn = Funhandled_file_name (Ffile_name_directory (filename));
  if (!NILP (ufn))
    return ufn;
  /* In most cases the unhandled-file-name handler already took care of it
     but there might be yet-unknown corner cases where it's not enough
     or maybe the file-name-handler hasn't been updated yet to define
     unhandled-file-name.  */
  handler = Ffind_file_name_handler (filename, Qunhandled_file_name_directory);
  if (!NILP (handler))
    return call2 (handler, Qunhandled_file_name_directory, filename);
  else
    return build_string ("/");

and my guess is that there won't be any file-name-handler that will ever
need to define unhandled-file-name-directory any more.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As usual, I don't care function names too much. But the mapping
>
>>   (nonmagic-file-name "/sudo::") => "/root"
>
>> doesn't seem to be obvious, although it would be the implementation
>> for Tramp.
>
> Actually, this implementation would be incorrect for my purpose
> (doc-view): the important aspect of unhandled-file-name is that
> a subprocess will be able to use it.  So it should be accessible by the
> normal Emacs user.

I'm lost. What do you expect as result of (nonmagic-file-name "/sudo::") ?
Something like `file-local-copy' returns?

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> I'm lost. What do you expect as result of (nonmagic-file-name "/sudo::") ?

nil

because this directory is (most likely) not accessible to a subprocess
run locally by Emacs.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I'm lost. What do you expect as result of (nonmagic-file-name "/sudo::") ?
>
> nil
>
> because this directory is (most likely) not accessible to a subprocess
> run locally by Emacs.

And in the unlikely case, the user can access "/root" nonmagically, it
shall return that string?

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As usual, I don't care function names too much. But the mapping
>
>>   (nonmagic-file-name "/sudo::") => "/root"
>
>> doesn't seem to be obvious, although it would be the implementation
>> for Tramp.
>
> Actually, this implementation would be incorrect for my purpose
> (doc-view): the important aspect of unhandled-file-name is that
> a subprocess will be able to use it.  So it should be accessible by the
> normal Emacs user.

I don't think that nonmagic-file-name is supposed to deal with
permissions.  If it were, it would be pretty pointless since most tramp
file names would differ in some way in the permissions.

-- 
David Kastrup

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>>> I'm lost. What do you expect as result of (nonmagic-file-name "/sudo::") ?
>> nil
>> because this directory is (most likely) not accessible to a subprocess
>> run locally by Emacs.

> And in the unlikely case, the user can access "/root" nonmagically, it
> shall return that string?

Since unhandled-file-name can return nil, the code that uses it has to
be able to deal with the case where the function returns nil (most
likely, it will use something like file-local-copy to make a copy that
can then be passed to the subprocess), so returning nil when the file is
actually accessible is suboptimal but correct and safe.

In other words, I'd rather just return nil even if /root happens to be
accessible because (who knows) the subprocess may actually need to write
to that directory.  And if /root is also available for write, then
I think this case is sufficiently unlikely or scary that returning nil
is good enough.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>>> As usual, I don't care function names too much. But the mapping
>> 
>>> (nonmagic-file-name "/sudo::") => "/root"
>> 
>>> doesn't seem to be obvious, although it would be the implementation
>>> for Tramp.
>> 
>> Actually, this implementation would be incorrect for my purpose
>> (doc-view): the important aspect of unhandled-file-name is that
>> a subprocess will be able to use it.  So it should be accessible by the
>> normal Emacs user.

> I don't think that nonmagic-file-name is supposed to deal with
> permissions.  If it were, it would be pretty pointless since most tramp
> file names would differ in some way in the permissions.

I don't see the problem: Tramp should simply always return nil.
That's fine.  The code (doc-view) will than fallback on the use of
file-local-copy to pass the file's content to the local process.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I don't see the problem: Tramp should simply always return nil.
> That's fine.  The code (doc-view) will than fallback on the use of
> file-local-copy to pass the file's content to the local process.

I'm really sorry, Stefan, but in this case I'm too stupid to see what's
the value of this function. In the remote case it shall always return
nil - this could be checked by (not (file-remote-p file)). What does the
function return the local case? Just the file name, expanded?

Maybe it helps to see your use case in doc-view.el?

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>> I don't see the problem: Tramp should simply always return nil.
>> That's fine.  The code (doc-view) will than fallback on the use of
>> file-local-copy to pass the file's content to the local process.

> I'm really sorry, Stefan, but in this case I'm too stupid to see what's
> the value of this function. In the remote case it shall always return
> nil - this could be checked by (not (file-remote-p file)). What does the
> function return the local case? Just the file name, expanded?

For the "/:" file-name handler, file-remote-p should return nil, and
unhandled-file-name should return the filename with the "/:" stripped.
For url-handler-mode, the "file://" should not be considered remote
either and unhandled-file-name should return the filename with the
"file://" stripped.

> Maybe it helps to see your use case in doc-view.el?

Given a filename (typically ending in ".pdf"), decide whether we can
pass it directly to a pdf2png program, or whether we first need to use
file-local-copy.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Richard Stallman]

    > `file-local-name' might not be a good name.
    > The operation `unhandled-file-name-directory' returns a directory that
    > isn't magic.  But remote names are not the only magic names, so
    > "local" isn't necessarily the right way to describe the relationship
    > between that directory and the argument.

    The issue is not whether the returned file name is magic or not, but
    whether it can be passed as-is to a subprocess.

I thought the two criteria were equivalent.  What am I missing?

    I don't understand.  unhandled-file-name-directory is used to get
    a directory that can be used as cwd for subprocesses.  And the new
    operation is meant to be used to get a filename that can be passed
    to subprocesses.  So AFAICT they share everything and should use the
    same name.

I don't understand.  Are you saying we only need one operation?
Or that the two operations should have similar names?

I agree they should have similar names.  What I'm suggesting
is that names that refer to `nonmagic' might be clearer
than names based on `unhandled'.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>> `file-local-name' might not be a good name.
>> The operation `unhandled-file-name-directory' returns a directory that
>> isn't magic.  But remote names are not the only magic names, so
>> "local" isn't necessarily the right way to describe the relationship
>> between that directory and the argument.

>     The issue is not whether the returned file name is magic or not, but
>     whether it can be passed as-is to a subprocess.

> I thought the two criteria were equivalent.  What am I missing?

/tmp/foo.gz is magic and can be passed as is to a subprocess.

>     I don't understand.  unhandled-file-name-directory is used to get
>     a directory that can be used as cwd for subprocesses.  And the new
>     operation is meant to be used to get a filename that can be passed
>     to subprocesses.  So AFAICT they share everything and should use the
>     same name.

> I don't understand.  Are you saying we only need one operation?
> Or that the two operations should have similar names?

> I agree they should have similar names.  What I'm suggesting
> is that names that refer to `nonmagic' might be clearer
> than names based on `unhandled'.

The "magic" file names are registered via file-name-handler-alis, not
via file-name-magic-alist, so "unhandled" makes at least as much sense,
doesn't it?


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For the "/:" file-name handler, file-remote-p should return nil, and
> unhandled-file-name should return the filename with the "/:" stripped.
> For url-handler-mode, the "file://" should not be considered remote
> either and unhandled-file-name should return the filename with the
> "file://" stripped.

I've got it: we were speaking about different functions. You need a
function which converts a file name to an argument which can be passed
literally to a *local* subprocess.

I was speaking about trimming a file name, which can be used then as
argument by *any* subprocess, either local, either remote
(`process-file' or `start-file-process').

Both functions might be useful.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Richard Stallman]

    >     The issue is not whether the returned file name is magic or not, but
    >     whether it can be passed as-is to a subprocess.

    > I thought the two criteria were equivalent.  What am I missing?

    /tmp/foo.gz is magic and can be passed as is to a subprocess.

I see.  So I guess `file-local-name' is right after all.
Sorry.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

>> For the "/:" file-name handler, file-remote-p should return nil, and
>> unhandled-file-name should return the filename with the "/:" stripped.
>> For url-handler-mode, the "file://" should not be considered remote
>> either and unhandled-file-name should return the filename with the
>> "file://" stripped.

> I've got it: we were speaking about different functions. You need a
> function which converts a file name to an argument which can be passed
> literally to a *local* subprocess.

> I was speaking about trimming a file name, which can be used then as
> argument by *any* subprocess, either local, either remote
> (`process-file' or `start-file-process').

> Both functions might be useful.

Hmm... I see your point and now I'm not so sure about the function
I propose.  I always thought that for process-file and
start-file-process, the best option was always to use
file-relative-name, but then the same argument should apply to my
file-local-name.  So the main missing feature is to figure out whether
that relative name will work, which depends on whether
unhandled-file-name-directory returns something that works or not.

I suggest to change unhandled-file-name-directory so that it can (and
should) return nil if the named file/directory cannot be
directly accessed.  And then let users of unhandled-file-name-directory
default to "~/" if it returned nil.  Then we can define file-local-name
as follows:

  (defun file-local-name/unhandled-file-name (file)
    (setq file (expand-file-name file))
    (let ((dir (unhandled-file-name-directory file)))
      (if file (expand-file-name (file-name-nondirectory file) dir))))

or alternatively, we can add file-local-name/unhandled-file-name and
drop unhandled-file-name-directory (which can be defined then as

  (defun unhandled-file-name-directory (file)
    (unhandled-file-name (file-name-directory (expand-file-name file))))


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Hmm... I see your point and now I'm not so sure about the function
> I propose.  I always thought that for process-file and
> start-file-process, the best option was always to use
> file-relative-name, but then the same argument should apply to my
> file-local-name.  So the main missing feature is to figure out whether
> that relative name will work, which depends on whether
> unhandled-file-name-directory returns something that works or not.

unhandled-file-name-directory is something that shall provide
`call-process' and `start-process' with a suitable default
directory. That's why Tramp always returns "~/", and ange-ftp does
"/tmp/". It doesn't look like it is usable in the `process-file' and
`start-file-process' cases.

file-relative-name might be sufficient in obvious cases:

  (file-relative-name "/sudo::/var/syslog/syslog.log" "/sudo::")
    => "../var/syslog/syslog.log"

But:

  (file-relative-name "/sudo::/var/syslog/syslog.log" "/ssh::")
    => "/sudo:root@arthur:/var/syslog/syslog.log"

This is correct, but inconvenient.

> I suggest to change unhandled-file-name-directory so that it can (and
> should) return nil if the named file/directory cannot be
> directly accessed.

... from a local subprocess. It shall not be used for default
directories of `process-file' and `start-file-process'.

> And then let users of unhandled-file-name-directory
> default to "~/" if it returned nil.  Then we can define file-local-name
> as follows:
>
>   (defun file-local-name/unhandled-file-name (file)
>     (setq file (expand-file-name file))
>     (let ((dir (unhandled-file-name-directory file)))
>       (if file (expand-file-name (file-name-nondirectory file) dir))))
>
> or alternatively, we can add file-local-name/unhandled-file-name and
> drop unhandled-file-name-directory (which can be defined then as
>
>   (defun unhandled-file-name-directory (file)
>     (unhandled-file-name (file-name-directory (expand-file-name
>     file))))

I agree. It doesn't make sense for Tramp and ange-ftp, guessing what
might be a good default directory for `call-process' and `start-process'.
They shall simply apply `ignore'; alternatively it shall not be called
per file-name-handler.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> unhandled-file-name-directory is something that shall provide
> `call-process' and `start-process' with a suitable default
> directory. That's why Tramp always returns "~/", and ange-ftp does
> "/tmp/". It doesn't look like it is usable in the `process-file' and
> `start-file-process' cases.

It's not needed for process-file and start-file-process.

> file-relative-name might be sufficient in obvious cases:

>   (file-relative-name "/sudo::/var/syslog/syslog.log" "/sudo::")
>     => "../var/syslog/syslog.log"

> But:

>   (file-relative-name "/sudo::/var/syslog/syslog.log" "/ssh::")
>     => "/sudo:root@arthur:/var/syslog/syslog.log"

> This is correct, but inconvenient.

No, it's very convenient: file-name-absolute-p will then tell you if the
process can reach those files.

>> I suggest to change unhandled-file-name-directory so that it can (and
>> should) return nil if the named file/directory cannot be
>> directly accessed.

> ... from a local subprocess. It shall not be used for default
> directories of `process-file' and `start-file-process'.

Yes, indeed, I'm only concerned about call-process and start-process.

>> And then let users of unhandled-file-name-directory
>> default to "~/" if it returned nil.  Then we can define file-local-name
>> as follows:
>> 
>> (defun file-local-name/unhandled-file-name (file)
>> (setq file (expand-file-name file))
>> (let ((dir (unhandled-file-name-directory file)))
>> (if file (expand-file-name (file-name-nondirectory file) dir))))
>> 
>> or alternatively, we can add file-local-name/unhandled-file-name and
>> drop unhandled-file-name-directory (which can be defined then as
>> 
>> (defun unhandled-file-name-directory (file)
>> (unhandled-file-name (file-name-directory (expand-file-name
>> file))))

> I agree. It doesn't make sense for Tramp and ange-ftp, guessing what
> might be a good default directory for `call-process' and `start-process'.
> They shall simply apply `ignore'; alternatively it shall not be called
> per file-name-handler.

Not all file handlers are like Tramp: jka-compr and several others only
modify the way files are accessed, rather than providing access to
more files.  So it still needs to be a file-name-handler.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> But:
>
>>   (file-relative-name "/sudo::/var/syslog/syslog.log" "/ssh::")
>>     => "/sudo:root@arthur:/var/syslog/syslog.log"
>
>> This is correct, but inconvenient.
>
> No, it's very convenient: file-name-absolute-p will then tell you if the
> process can reach those files.

There might be cases where this is still not sufficient. Think about a
Samba server, which offers several "shares" one cannot simply navigate
via `file-relative-name'. But given that the "smb" method of Tramp
does not support `process-file' and `start-file-process' (yet?), it
might be a good approximation to check for `file-name-absolute-p'.

>> I agree. It doesn't make sense for Tramp and ange-ftp, guessing what
>> might be a good default directory for `call-process' and `start-process'.
>> They shall simply apply `ignore'; alternatively it shall not be called
>> per file-name-handler.
>
> Not all file handlers are like Tramp: jka-compr and several others only
> modify the way files are accessed, rather than providing access to
> more files.  So it still needs to be a file-name-handler.

If I've checked correctly, only ange-ftp.el, tramp.el and
url-handlers.el offer an own implementation of
`unhandled-file-name-directory'. It might not be necessary to delegate
the implementation to those packages; the case of "remoteness" could
be handled directly in Funhandled_file_name_directory.

OTOH, it might be the best to keep the implementation as-it-is; "never
touch ..." etc pp.

Just being curious: url-handlers.el misses an own implementation of
`file-remote-p'; shouldn't it be there?

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> There might be cases where this is still not sufficient. Think about a
> Samba server, which offers several "shares" one cannot simply navigate
> via `file-relative-name'. But given that the "smb" method of Tramp
> does not support `process-file' and `start-file-process' (yet?), it
> might be a good approximation to check for `file-name-absolute-p'.

I don't understand what you're trying to say.

> If I've checked correctly, only ange-ftp.el, tramp.el and
> url-handlers.el offer an own implementation of
> `unhandled-file-name-directory'.  It might not be necessary to
> delegate the implementation to those packages; the case of
> "remoteness" could be handled directly in
> Funhandled_file_name_directory.

And I do not understand what implementation of
Funhandled_file_name_directory you have in mind.
Also note:

  (unhandled-file-name-directory "/:/toto/titi") -> "/toto"

and the intention is to add at some point:

  (unhandled-file-name-directory "file:///toto/titi") -> "/toto"

> Just being curious: url-handlers.el misses an own implementation of
> `file-remote-p'; shouldn't it be there?

Yes, it should.  Patches welcome ;-)


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> There might be cases where this is still not sufficient. Think about a
>> Samba server, which offers several "shares" one cannot simply navigate
>> via `file-relative-name'. But given that the "smb" method of Tramp
>> does not support `process-file' and `start-file-process' (yet?), it
>> might be a good approximation to check for `file-name-absolute-p'.
>
> I don't understand what you're trying to say.

Just that your proposal (use file-relative-name for trimming arguments
of remote subprocesses) works only, if the remote side has a UNIX-like
filesystem with a unique root "/". Given, that remote subprocesses are
implemented today just for such types of remote hosts, it is a
sufficient assumption.

>> If I've checked correctly, only ange-ftp.el, tramp.el and
>> url-handlers.el offer an own implementation of
>> `unhandled-file-name-directory'.  It might not be necessary to
>> delegate the implementation to those packages; the case of
>> "remoteness" could be handled directly in
>> Funhandled_file_name_directory.
>
> And I do not understand what implementation of
> Funhandled_file_name_directory you have in mind.

I wanted to say that unhandled-file-name-directory does not need a file
name handler. But ...

>   (unhandled-file-name-directory "file:///toto/titi") -> "/toto"

... convinces me that I'm wrong. Achhh.

>> Just being curious: url-handlers.el misses an own implementation of
>> `file-remote-p'; shouldn't it be there?
>
> Yes, it should.  Patches welcome ;-)

I'll do so. But it raises the question: Is "file:///toto/titi" remote? I
would say no, but it depends how file-remote-p is used. If it is applied
in tests like "Can I use the filename literally for local purposes",
then (file-remote-p" file:///toto/titi") must return a non-nil value.

>         Stefan

Best regards, Michael.

url-handler-file-remote-p (was: testing for a remote file to include file on a Windows mappeddrive)

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Just being curious: url-handlers.el misses an own implementation of
>> `file-remote-p'; shouldn't it be there?
>
> Yes, it should.  Patches welcome ;-)

What about this one:

(defun url-handler-file-remote-p (filename &optional identification connected)
  ;; Maybe we can find a suitable check for CONNECTED.
  ;; FIXME: if `filename' is a "file://" URL it shall be nil.  But this
  ;; must be introduced together with other primitive functions, like
  ;; `unhandled-file-name-directory'.
  (let ((url (url-generic-parse-url filename)))
    (when (url-type url)
      (cond
       ((eq identification 'method) (url-type url))
       ((eq identification 'user) (url-user url))
       ((eq identification 'host) (url-host url))
       ((eq identification 'localname) (url-filename url))
       (t (url-recreate-url
	   (url-parse-make-urlobj (url-type url) (url-user url) nil
				  (url-host url) (url-port url))))))))

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mappeddrive

[Stefan Monnier]

> Just that your proposal (use file-relative-name for trimming arguments
> of remote subprocesses) works only, if the remote side has a UNIX-like
> filesystem with a unique root "/". Given, that remote subprocesses are
> implemented today just for such types of remote hosts, it is a
> sufficient assumption.

It should still work on systems without a unique / root, as long as
file-remote-p returns different values for different roots.
It may work suboptimally, in that file-relative-name may return an
absolute file name, indicating that "it can't be reached" whereas it
actually can be reached, but I think this is OK for now.

>>> Just being curious: url-handlers.el misses an own implementation of
>>> `file-remote-p'; shouldn't it be there?
>> Yes, it should.  Patches welcome ;-)
> I'll do so. But it raises the question: Is "file:///toto/titi" remote? I
> would say no, but it depends how file-remote-p is used. If it is applied
> in tests like "Can I use the filename literally for local purposes",
> then (file-remote-p" file:///toto/titi") must return a non-nil value.

Welcome to this thread, which started around the discussion of what
file-remote-p should do and how that relates to
file-local-name/unhandled-file-name.  As has now been made more explicit
in the docstring, file-remote-p is intended to give an indication about
the speed and reliability of access to the file, not about whether the
filename can be passed to subprocesses.

The fact that file-remote-p is also used by file-relative-name is
probably bad and will sooner or later lead to situations where it's
unclear what file-remote-p should return.

At that point, we may want to introduce a new file name handler.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Welcome to this thread, which started around the discussion of what
> file-remote-p should do and how that relates to
> file-local-name/unhandled-file-name.  As has now been made more explicit
> in the docstring, file-remote-p is intended to give an indication about
> the speed and reliability of access to the file, not about whether the
> filename can be passed to subprocesses.

... and I'm still agnostic about :-)

>         Stefan

Best regards, Michael.

Re: url-handler-file-remote-p

[Stefan Monnier]

>>> Just being curious: url-handlers.el misses an own implementation of
>>> `file-remote-p'; shouldn't it be there?
>> 
>> Yes, it should.  Patches welcome ;-)

> What about this one:

> (defun url-handler-file-remote-p (filename &optional identification connected)
>   ;; Maybe we can find a suitable check for CONNECTED.
>   ;; FIXME: if `filename' is a "file://" URL it shall be nil.  But this
>   ;; must be introduced together with other primitive functions, like
>   ;; `unhandled-file-name-directory'.
>   (let ((url (url-generic-parse-url filename)))
>     (when (url-type url)
>       (cond
>        ((eq identification 'method) (url-type url))
>        ((eq identification 'user) (url-user url))
>        ((eq identification 'host) (url-host url))
>        ((eq identification 'localname) (url-filename url))
>        (t (url-recreate-url
> 	   (url-parse-make-urlobj (url-type url) (url-user url) nil
> 				  (url-host url) (url-port url))))))))

Looks good.


        Stefan

Re: url-handler-file-remote-p

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> What about this one:
>
> Looks good.

(Slightly different) installed.

>         Stefan

Best regards, Michael.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

I'm returning to this thread because I don't know what was finally done wrt what
I asked for in starting the thread: Treat a file name with a Windows mapped
network drive as remote but one with a local drive as not remote.

Could someone please summarize what was done about this? The thread seems to
have drifted away from its target, and I can't tell what the outcome was.

I'm still using the code I mentioned (see below), which tries NET USE on
Windows. Is that part perhaps unneeded now, because an equivalent is done by
Emacs itself? Note that while I find the NET USE code to be quick, a couple of
users have reported that it can seem to hang. Suggestions about that?

Also, could someone please reply specifically to my last message in the thread,
below (e.g. `file-remote-p' vs `ffap-remote-p')?

Thx.

> From: Drew Adams Sent: Saturday, January 26, 2008 5:36 PM
>
> I'm not looking for something that needs to be "independent 
> of the syntax". It could use the syntax sometimes, but it
> might need to go beyond the syntax sometimes. On Windows,
> it would need to be able to tell when "h:/foo" might
> name a file on a network drive, and the syntax alone won't 
> suffice for that. I think Stefan suggested something similar
> for symbolic links on other platforms.
> 
> I'm looking for something that will, as you say, "never open 
> a connection on its own". I'm interested in saving time. I
> want to distinguish a string that is likely to name a remote
> file or a file on a mapped network drive from a string that
> is likely to name a local file. That's all.
> 
> The problem I posed was that I couldn't find a function that 
> makes that distinction. `file-remote-p', in particular,
> returns nil for a file that is on a mapped network drive.
> 
> I don't care what the function is called - whether (a) 
> `file-remote-p' is tweaked to return non-nil for a file on a
> network drive or (b) some other function is provided; either
> would be fine by me.
> 
> FWIW, I am currently using Eli's "NET USE" suggestion
> (thanks). It distinguishes network drives from local drives
> and non-existent drives, which is sufficient for my use.
> (AFAICT, it does not distinguish a local drive from a
> non-existent drive, but that's OK.) This is what I have now:
> 
> (defun my-file-remote-p (file)
>   "Non-nil means FILE is likely to name a file on a remote system.
> For MS Windows, this includes a file on a mapped network drive."
>   (or (and (eq system-type 'windows-nt)
>            (let ((case-fold-search t))
>              (and (string-match "\\`\\([a-z]:\\)" file)
>                   (eq 0 (condition-case nil
>                             (call-process
>                              shell-file-name nil nil nil
>                              shell-command-switch
>                              (concat "NET USE "
>                                      (match-string 1 file)))
>                           (error nil))))))
>       (and (fboundp 'ffap-file-remote-p)
>            (ffap-file-remote-p file))
>       (file-remote-p file)))
> 
> I'm not necessarily suggesting this for Emacs; I'm just 
> saying that it seems to work for me.
> 
> BTW: Can `file-remote-p' ever return non-nil when 
> `ffap-remote-p' returns nil? If not, I'll remove the
> `ffap-remote-p' test. My guess is that neither
> is a proper subset of the other, so I use both.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> Could someone please summarize what was done about this?

Nothing.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Lennart Borgman (gmail)]

Stefan Monnier wrote:
>> Could someone please summarize what was done about this?
> 
> Nothing.


Is WNetGetConnection useful for this? (There is already code calling 
this function in Emacs.)

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > Could someone please summarize what was done about this?
> 
> Nothing.

OK, thanks for the prompt reply. Sometimes it's hard to see the closure in a
thread, and sometimes there is none. ;-)

Could you (someone) explain why nothing was done? Was it because it was decided
that: It was a silly problem and request (no need)? No idea how to fix it? No
resources to fix it? Someone is still working on it?

I realize that I didn't frame it as a bug report but as a discussion
(questions). If it were a bug report, there presumably would have been more
closure.

Assuming for the moment that there is no intention to do anything about it, can
someone at least weigh in on the code I sent (which I'm still using)?
Suggestions for improvement? What about the question wrt `file-remote-p' and
`ffap-remote-p'?

Thx.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>>> Could someone please summarize what was done about this?
>> Nothing.

Actually, this is not quite right: nothing came out of the part of the
discussion Drew referred to, but the other part (the one I cared about)
did result in a refined docstring for file-remote-p.

> Is WNetGetConnection useful for this? (There is already code calling this
> function in Emacs.)

I have no idea.  But note that you may have mapped drives that are quite
"local" in the file-remote-p sense.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> Date: Sun, 20 Apr 2008 22:03:53 +0200
> From: "Lennart Borgman (gmail)" <lennart.borgman@gmail.com>
> Cc: 'Jason Rumney' <jasonr@gnu.org>, 'Michael Albinus' <michael.albinus@gmx.de>,
> 	Drew Adams <drew.adams@oracle.com>, 'Emacs-Devel' <emacs-devel@gnu.org>
> 
> Stefan Monnier wrote:
> >> Could someone please summarize what was done about this?
> > 
> > Nothing.
> 
> 
> Is WNetGetConnection useful for this?

What for?  GetDriveType is perfectly capable of telling us whether the
drive is networked or not.  Or did you mean something else?

In any case, as long as the Unix code treats networked drives mounted
via NFS or similar as local ones, I will object doing otherwise on
Windows.  A primitive (separate from file-remote-p) that
differentiates between local and network-mounted volumes would be okay
with me, but only if it is supported on Unix as well.

Re: testing for a remote file to include file on a Windows mapped drive

[Lennart Borgman (gmail)]

Eli Zaretskii wrote:
>> Is WNetGetConnection useful for this?
> 
> What for?  GetDriveType is perfectly capable of telling us whether the
> drive is networked or not.  Or did you mean something else?

No, I just did not know how to test it.

> In any case, as long as the Unix code treats networked drives mounted
> via NFS or similar as local ones, I will object doing otherwise on
> Windows.  A primitive (separate from file-remote-p) that
> differentiates between local and network-mounted volumes would be okay
> with me, but only if it is supported on Unix as well.

Seems to be the right way.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Sun, 20 Apr 2008 13:24:52 -0700
> Cc: 'Emacs-Devel' <emacs-devel@gnu.org>,
> 	'Michael Albinus' <michael.albinus@gmx.de>, 'Jason Rumney' <jasonr@gnu.org>
> 
> Could you (someone) explain why nothing was done? Was it because it was decided
> that: It was a silly problem and request (no need)? No idea how to fix it? No
> resources to fix it? Someone is still working on it?

My impression was that the problem was not deemed important enough to
fix.  But that's me.

> Assuming for the moment that there is no intention to do anything about it, can
> someone at least weigh in on the code I sent (which I'm still using)?
> Suggestions for improvement? What about the question wrt `file-remote-p' and
> `ffap-remote-p'?

I don't think the current notion of remote files in Emacs is
consistent with what you want.  I suggest instead to lobby for a
separate primitive; I'd support that if it were available on Unix and
GNU systems as well.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> > Could someone please summarize what was done about this?
>> Nothing.

> Could you (someone) explain why nothing was done? Was it because it
> was decided that: It was a silly problem and request (no need)? No
> idea how to fix it? No resources to fix it? Someone is still working
> on it?

AFAIK, it's a difficult problem, and it's not very high up on the
priority.

> Assuming for the moment that there is no intention to do anything
> about it, can someone at least weigh in on the code I sent (which I'm
> still using)?  Suggestions for improvement?

> What about the question wrt `file-remote-p' and `ffap-remote-p'?

ffap-file-remote-p seems to be incompatible with file-remote-p because
it is also used to "normalize" file names and is applied to URLs even
when url-handler-mode is not loaded.

Maybe we should add to file-remote-p something equivalent to
ffap-rfs-regexp.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Jason Rumney]

Lennart Borgman (gmail) wrote:
> Is WNetGetConnection useful for this? (There is already code calling 
> this function in Emacs.)

Not really, it is coincidence that that code was added around the same 
timeframe as the thread Drew is talking about. That call just ensures we 
are logged on to the server/domain before attempting to use the drive.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> >> > Could someone please summarize what was done about this?
> >> Nothing.
> 
> > Could you (someone) explain why nothing was done? Was it because it
> > was decided that: It was a silly problem and request (no need)? No
> > idea how to fix it? No resources to fix it? Someone is still working
> > on it?
> 
> AFAIK, it's a difficult problem, and it's not very high up on the
> priority.

I see. Would it help for me to detail my use case more?

> > Assuming for the moment that there is no intention to do anything
> > about it, can someone at least weigh in on the code I sent 
> > (which I'm still using)?  Suggestions for improvement?

No?

> > What about the question wrt `file-remote-p' and `ffap-remote-p'?
> 
> ffap-file-remote-p seems to be incompatible with file-remote-p because
> it is also used to "normalize" file names and is applied to URLs even
> when url-handler-mode is not loaded.
> 
> Maybe we should add to file-remote-p something equivalent to
> ffap-rfs-regexp.

That was part of the suggestion/question: whether I should really need to call
two functions that way. I do so because it seemed to me that neither alone was
adequate to the task of telling me whether a file name might be remote. I try
ffap-file-remote-p first, because that seems less costly. 

I use ffap-file-remote-p only for matching the two regexps, not for its returned
string value. Why do you mention only ffap-rfs-regexp and not also
ffap-ftp-regexp?

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > Could you (someone) explain why nothing was done?
> 
> My impression was that the problem was not deemed important enough to
> fix.  But that's me.

Thanks, that makes it clearer.

If it helps understand why this can be important, I can explain my use case
more. I don't want to take up bandwidth, if not.

> > Assuming for the moment that there is no intention to do 
> > anything about it, can someone at least weigh in on the code
> > I sent (which I'm still using)? Suggestions for improvement?

Any ideas on this? Given that I'm using Lisp, not C, is what I'm doing a
reasonable way to handle the Windows mapped drives test?

> > What about the question wrt `file-remote-p' and `ffap-remote-p'?
> 
> I don't think the current notion of remote files in Emacs is
> consistent with what you want.  

Part of what I was saying was that we don't seem to have a single notion of
remote file-name test. The difference between what `file-remote-p' and
`ffap-remote-p' do is one aspect of that.

> I suggest instead to lobby for a separate primitive; I'd support
> that if it were available on Unix and GNU systems as well.

OK, consider this a lobby. ;-) But I don't know what kind of primitive you have
in mind. I explained what I thought I needed, but I'm not sure that is what you
have in mind.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> > Could you (someone) explain why nothing was done? Was it because it
>> > was decided that: It was a silly problem and request (no need)? No
>> > idea how to fix it? No resources to fix it? Someone is still working
>> > on it?
>> 
>> AFAIK, it's a difficult problem, and it's not very high up on the
>> priority.

> I see. Would it help for me to detail my use case more?

Maybe.  Try to phrase it in terms of the docstring of file-remote-p,
either showing how the function's implementation falls short of its spec,
or that the spec is not adequate for your use (maybe we need another
function for that, or maybe the spec can be adjusted).

>> > Assuming for the moment that there is no intention to do anything
>> > about it, can someone at least weigh in on the code I sent 
>> > (which I'm still using)?  Suggestions for improvement?
> No?

No idea what code you're talking about.

>> > What about the question wrt `file-remote-p' and `ffap-remote-p'?
>> 
>> ffap-file-remote-p seems to be incompatible with file-remote-p because
>> it is also used to "normalize" file names and is applied to URLs even
>> when url-handler-mode is not loaded.
>> 
>> Maybe we should add to file-remote-p something equivalent to
>> ffap-rfs-regexp.

> That was part of the suggestion/question: whether I should really need to call
> two functions that way.

I don't know.  Clearly ffap-file-remote-p is not considered as important
enough to be in the core.

> I use ffap-file-remote-p only for matching the two regexps, not for
> its returned string value. Why do you mention only ffap-rfs-regexp and
> not also ffap-ftp-regexp?

ffap-rfs-regexp seems to fall squarely in the domain of file-remote-p.
ffap-ftp-regexp should already be handled by ange-ftp's definition of
file-remote-p.


        Stefan

Re: testing for a remote file to include file on a Windows mappeddrive

[Eric Hanchrow]

>>>>> "Drew" == Drew Adams <drew.adams@oracle.com> writes:

    >> GetDriveType can be called from C.  Emacs already uses it in
    >> w32.c, but only to determine which drives are fixed, thus their
    >> information can be cached indefinitely.

    Drew> Good.  Do you agree that it would be good for either
    Drew> `file-remote-p' or some new function to take mapped network
    Drew> drives into account for remote files?

_I_ sure wish it did.  I like using global-auto-revert-mode, but that
can make Emacs hang when it checks a remote file.  And since
file-remote-p doesn't give the right answer ...

-- 
"La Vie en Rose," ... has an intricate structure, which is a
polite way of saying that it's a complete mess.
        A. O. Scott, in the New York Times

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> >> AFAIK, it's a difficult problem, and it's not very high up on the
> >> priority.
> > I see. Would it help for me to detail my use case more?
> 
> Maybe.  Try to phrase it in terms of the docstring of file-remote-p,
> either showing how the function's implementation falls short 
> of its spec, or that the spec is not adequate for your use (maybe we
> need another function for that, or maybe the spec can be adjusted).

As you said, the purpose of file-remote-p is to determine, without the cost of a
remote access, whether a file name represents a remote file. The aim in using it
is to be able to know that a file is remote, so you might then avoid the cost of
accessing it.

* A file on a Windows mapped network drive is remote, in the sense of incurring
a performance penalty for access. `file-remote-p' should return non-nil for such
a file, but nil for a file on a Windows local drive. That is not the case now.

* `file-remote-p' does not handle some file-name cases handled by
`ffap-file-remote-p', or it handles them less efficiently. It should incorporate
the syntactic file-name matching that `ffap-file-remote-p' does.

Again, I can detail my use case, to show why this is important to me, if it will
help.

> >> > Assuming for the moment that there is no intention to do anything
> >> > about it, can someone at least weigh in on the code I sent 
> >> > (which I'm still using)?  Suggestions for improvement?
> > No?
> 
> No idea what code you're talking about.

I sent it at least three times in this thread, including this morning. Again:

> (defun my-file-remote-p (file)
>   "Non-nil means FILE is likely to name a file on a remote system.
> For MS Windows, this includes a file on a mapped network drive."
>   (or (and (eq system-type 'windows-nt)
>            (let ((case-fold-search t))
>              (and (string-match "\\`\\([a-z]:\\)" file)
>                   (eq 0 (condition-case nil
>                             (call-process
>                              shell-file-name nil nil nil
>                              shell-command-switch
>                              (concat "NET USE "
>                                      (match-string 1 file)))
>                           (error nil))))))
>       (and (fboundp 'ffap-file-remote-p)
>            (ffap-file-remote-p file))
>       (file-remote-p file)))

FWIW, Michael Albinus tested that code against `file-remote-p' for one remote
file name. He came up with these times on a Ubuntu machine using Emacs 23.0.60
and Tramp 2.1.13 (so, without using the Windows NET USE part):

 (elp-instrument-function 'icicle-file-remote-p)
 (dotimes (i 1000)
   (icicle-file-remote-p "/ssh:albinus@fencepost.gnu.org"))
 
 (elp-results)
 icicle-file-remote-p  1000  0.0111750000  1.117...e-05
 
 (elp-instrument-function 'file-remote-p)
 (dotimes (i 1000)
   (file-remote-p "/ssh:albinus@fencepost.gnu.org"))
 
 (elp-results)
 file-remote-p  2004  1.4309400000  0.0007140419

So for at least some remote file names `ffap-file-remote-p' is quite a bit
faster than `file-remote-p' (`icicle-file-remote-p' here is what I called
`my-file-remote-p' above).

[Apologies, Michael, for quoting off-list mail; I assumed you wouldn't mind.]

> >> Maybe we should add to file-remote-p something equivalent to
> >> ffap-rfs-regexp.
> 
> > I use ffap-file-remote-p only for matching the two regexps, not for
> > its returned string value. Why do you mention only 
> > ffap-rfs-regexp and not also ffap-ftp-regexp?
> 
> ffap-rfs-regexp seems to fall squarely in the domain of file-remote-p.
> ffap-ftp-regexp should already be handled by ange-ftp's definition of
> file-remote-p.

So if ffap-ftp-regexp is subsumed by an ange-ftp test, the question becomes
whether that ange-ftp test should be included in `file-remote-p'.

Re: testing for a remote file to include file on a Windows mapped drive

[Jason Rumney]

Drew Adams wrote:

> As you said, the purpose of file-remote-p is to determine, without the cost of a
> remote access, whether a file name represents a remote file. The aim in using it
> is to be able to know that a file is remote, so you might then avoid the cost of
> accessing it.
>   

As mentioned in the earlier thread, remoteness is not a reliable 
indicator of speed. You can have fast remote drives and slow local 
drives. Maybe in the days of 10Mbps ethernet it was still reasonable to 
assume that networked drives were slow, but with gigabit ethernet you'd 
be hard pressed to notice a performance difference between a networked 
and local drive.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

"Drew Adams" <drew.adams@oracle.com> writes:

> As you said, the purpose of file-remote-p is to determine, without the cost of a
> remote access, whether a file name represents a remote file. The aim in using it
> is to be able to know that a file is remote, so you might then avoid the cost of
> accessing it.
>
> * A file on a Windows mapped network drive is remote, in the sense of incurring
> a performance penalty for access. `file-remote-p' should return non-nil for such
> a file, but nil for a file on a Windows local drive. That is not the case now.

I believe, we are speaking about 2 different functions:

* file-remote-p returns t, if a file is not directly accessible by
  underlying operating system's means. Such files always need some
  special file name handler functions in Emacs for proper
  handling. Such (absolute) file names cannot be used literally
  outside functions, which support file name handlers.

* file-mounted-p (as working name) returns t, if a file looks like an
  ordinary file from the operating system's point of view, but its
  physical location is on another machine. Examples are nfs, smbfs or
  sshfs mounts under GNU/Linux, or network shares under W32. This
  property cannot be detected by file name handler functions. The
  implementation shall be in the C core.

These functions are exclusive: file-remote-p and file-mounted-p shall
not return t for the same filename. Since one function uses the
filename only for checking the result, and the other function uses
operating system properties, I don't believe it makes sense to merge
them into one function.

A test for "slow access" is therefore

(or (file-mounted-p filename)
    (file-remote-p filename))

> [Apologies, Michael, for quoting off-list mail; I assumed you wouldn't mind.]

No problem, you can always quote me, unless I've stated otherwise.

> So if ffap-ftp-regexp is subsumed by an ange-ftp test, the question becomes
> whether that ange-ftp test should be included in `file-remote-p'.

???

The relevant test for ange-ftp is included in file-remote-p.

Best regards, Michael.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > As you said, the purpose of file-remote-p is to determine, 
> > without the cost of a remote access, whether a file name
> > represents a remote file. The aim in using it
> > is to be able to know that a file is remote, so you might 
> > then avoid the cost of accessing it.
> 
> As mentioned in the earlier thread, remoteness is not a reliable 
> indicator of speed. You can have fast remote drives and slow local 
> drives. Maybe in the days of 10Mbps ethernet it was still 
> reasonable to assume that networked drives were slow, but with gigabit 
> ethernet you'd be hard pressed to notice a performance difference
> between a networked and local drive.

That might be, but IIRC, the stated purpose of `file-remote-p' (from the earlier
thread) is to let you know that a file is remote _so that you can choose to
avoid accessing it_ if you want.

There was some discussion of whether `file-remote-p' should actually try to
access the named file, to know whether it in fact names a readable existing
remote file, and the conclusion was no: It should not itself incur a performance
penalty, but should try other means (e.g. syntactic) to determine whether the
name might represent a remote file. 

But the stated purpose was to be able to avoid then accessing the file if it was
thought to be remote, because that might incur a performance penalty.

IOW, `file-remote-p' might not always be an accurate gauge of access
performance, but it can be better than nothing in some common contexts.

Whether I access a local Windows drive (even a slow one) or a Windows mapped
network drive that happens to be in India, there is a world of difference. Maybe
that difference will diminish tomorrow, but today, at least, there is a big
difference. YMMV.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > As you said, the purpose of file-remote-p is to determine, 
> > without the cost of a remote access, whether a file name
> > represents a remote file. The aim in using it
> > is to be able to know that a file is remote, so you might 
> > then avoid the cost of accessing it.
> >
> > * A file on a Windows mapped network drive is remote, in 
> >   the sense of incurring a performance penalty for access.
> >   `file-remote-p' should return non-nil for such a file,
> >   but nil for a file on a Windows local drive. That is not
> >   the case now.
> 
> I believe, we are speaking about 2 different functions:
> 
> * file-remote-p returns t, if a file is not directly accessible by
>   underlying operating system's means. Such files always need some
>   special file name handler functions in Emacs for proper
>   handling. Such (absolute) file names cannot be used literally
>   outside functions, which support file name handlers.
> 
> * file-mounted-p (as working name) returns t, if a file looks like an
>   ordinary file from the operating system's point of view, but its
>   physical location is on another machine. Examples are nfs, smbfs or
>   sshfs mounts under GNU/Linux, or network shares under W32. This
>   property cannot be detected by file name handler functions. The
>   implementation shall be in the C core.
> 
> These functions are exclusive: file-remote-p and file-mounted-p shall
> not return t for the same filename. Since one function uses the
> filename only for checking the result, and the other function uses
> operating system properties, I don't believe it makes sense to merge
> them into one function.
> 
> A test for "slow access" is therefore
> 
> (or (file-mounted-p filename)
>     (file-remote-p filename))

I see. I wasn't aware of `file-mounted-p'.
I don't see it in my January build of Emacs 23.
 
> > So if ffap-ftp-regexp is subsumed by an ange-ftp test, the 
> > question becomes whether that ange-ftp test should be
> > included in `file-remote-p'.
> 
> The relevant test for ange-ftp is included in file-remote-p.

I guessed that. But in that case, perhaps it should be included at an earlier
stage, so that the performance (as shown by your tests) reflects that of
`ffap-ftp-regexp'.

My question was really about my code - whether I still need to use both tests. I
guess the answer is yes, at least for now; if not for pattern completeness
(coverage), at least for performance (try the ffap patterns first).

Re: testing for a remote file to include file on a Windows mappeddrive

[Andreas Schwab]

Eric Hanchrow <offby1@blarg.net> writes:

> _I_ sure wish it did.  I like using global-auto-revert-mode, but that
> can make Emacs hang when it checks a remote file.  And since
> file-remote-p doesn't give the right answer ...

Is a file on a CDROM device that needs to spin up on every access
file-remote-p?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

RE: testing for a remote file to include file on aWindows mappeddrive

[Drew Adams]

> > _I_ sure wish it did.  I like using global-auto-revert-mode,
> > but that can make Emacs hang when it checks a remote file.
> > And since file-remote-p doesn't give the right answer ...
> 
> Is a file on a CDROM device that needs to spin up on every access
> file-remote-p?

We agree that we might need other test functions, such as Michael suggested.

But as long as the agreed-upon purpose of `file-remote-p' is to let you avoid an
access performance cost, that can be the place to incorporate such info.

IOW, either we add other test functions, as Michael suggested with
`file-mounted-p', and you then test with (or ...) to see if there is a possible
slowdown, or we put the tests into `file-remote-p' and we recognize that the
name is not accurate. Or we roll all the tests into some other function (as I
tried to do with `my-remote-file'), say, `slow-access-file-p'.

The expressed need is to know whether accessing a file with a given name might
be significantly slower than accessing a local file on an ordinary drive etc.

Some of those terms might be vague, but the general idea is clear: We want to
get a performance assessment from the file name (perhaps also locally checking
known mounts, mapped drives etc.) - without incurring a significant performance
penalty for the test itself. 

A boolean estimate is probably enough: possible performance hit vs probably
fast. How that is done, and whether it indicates that the file is remote or on a
slow local drive or whatever, I really don't care (but we might care, in some
contexts).

For some uses, some people might want more than a boolean - a time estimate, so
they can judge "fast enough" for some particular use. But for myself, now,
classifying the file names into two groups (probably fast, possibly slow) is
enough.

Re: testing for a remote file to include file on aWindows mappeddrive

[Andreas Schwab]

"Drew Adams" <drew.adams@oracle.com> writes:

> The expressed need is to know whether accessing a file with a given name might
> be significantly slower than accessing a local file on an ordinary drive etc.

Is a file on a NAS volume connected via GbE slower than a file on a
locally mounted crappy USB stick?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: testing for a remote file to include file on a Windows mapped drive

[Andreas Schwab]

Michael Albinus <michael.albinus@gmx.de> writes:

> * file-mounted-p (as working name) returns t, if a file looks like an
>   ordinary file from the operating system's point of view, but its
>   physical location is on another machine. Examples are nfs, smbfs or
>   sshfs mounts under GNU/Linux, or network shares under W32.

What should file-mounted-p return on a diskless client?

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

"Drew Adams" <drew.adams@oracle.com> writes:

> I see. I wasn't aware of `file-mounted-p'.
> I don't see it in my January build of Emacs 23.

Of course, because it doesn't exist yet. It's a proposal for a new
function.

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Andreas Schwab <schwab@suse.de> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> * file-mounted-p (as working name) returns t, if a file looks like an
>>   ordinary file from the operating system's point of view, but its
>>   physical location is on another machine. Examples are nfs, smbfs or
>>   sshfs mounts under GNU/Linux, or network shares under W32.
>
> What should file-mounted-p return on a diskless client?

`t', as per definition. I'm not too happy with that; because there is
also the case of CDs or USB sticks, where the access might be slow,
though these devices are not mounted in the sense of file-mounted-p.

I would prefer a function which returns "fast" or "slow". Since we
cannot know it without testing (what shall be avoided), we have only
such "second hand indications".

One could consider additional regexps, which tell "when the mount
point matches this regexp, return `t' for file-mounted-p", and the
contrary case with the other regexp. But this would make the handling
more complex.

> Andreas.

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mapped drive

[Andreas Schwab]

Michael Albinus <michael.albinus@gmx.de> writes:

> One could consider additional regexps, which tell "when the mount
> point matches this regexp, return `t' for file-mounted-p", and the
> contrary case with the other regexp. But this would make the handling
> more complex.

That doesn't scale.  With hotplug names of mount points get arbitrary.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Andreas Schwab <schwab@suse.de> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> One could consider additional regexps, which tell "when the mount
>> point matches this regexp, return `t' for file-mounted-p", and the
>> contrary case with the other regexp. But this would make the handling
>> more complex.
>
> That doesn't scale.  With hotplug names of mount points get arbitrary.

One could offer to check for filesystem types, like udf or usbfs.

> Andreas.

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mapped drive

[Andreas Schwab]

Michael Albinus <michael.albinus@gmx.de> writes:

> One could offer to check for filesystem types, like udf or usbfs.

usbfs is not what you think it is.  The file system type has nothing to
do with the device type.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: testing for a remote file to include file on aWindows mappeddrive

[Stefan Monnier]

> `file-mounted-p', and you then test with (or ...) to see if there is
> a possible slowdown, or we put the tests into `file-remote-p' and we
> recognize that the name is not accurate.

Drew.  *Please* read the new docstring:

   A file is considered "remote" if accessing it is likely to be slower
   or less reliable than accessing local files.

Physical remoteness is irrelevant.  Even the transport protocol might
not be relevant sometimes (tho it can be good input to a default
heuristic).  The current implementation and behavior is not cast
in stone.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> * file-remote-p returns t, if a file is not directly accessible by
>   underlying operating system's means. Such files always need some
>   special file name handler functions in Emacs for proper
>   handling. Such (absolute) file names cannot be used literally
>   outside functions, which support file name handlers.

Please read the docstring of file-remote-p, it has "nothing" to do
with the above.  What you describe can misclassify both ways: remote
access can be provided by the OS (using NFS/AFS/SSHFS/...), and local
access can be provided by file-name-handlers (e.g. file:// URLs).

What you're describing is the unhandled-file-name function (which
currently doesn't exist: the closest is unhandled-file-name-directory).

BTW, along the same lines, we may want to provide an
unhandled-file-contents function, so as to detected files accessed via
auto-compression-mode or auto-encryption-mode.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> I would prefer a function which returns "fast" or "slow".

We have that.  It's called `file-remote-p':

   A file is considered "remote" if accessing it is likely to be slower or
   less reliable than accessing local files.

Admittedly, the current implementation is not always good enough.

> One could consider additional regexps, which tell "when the mount
> point matches this regexp, return `t' for file-mounted-p", and the
> contrary case with the other regexp.  But this would make the handling
> more complex.

That's what ffap-rfs-regexp tries to do.  And yes, I think it should be
added to file-remote-p.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

>> * file-remote-p returns t, if a file is not directly accessible by
>>   underlying operating system's means. Such files always need some
>>   special file name handler functions in Emacs for proper
>>   handling. Such (absolute) file names cannot be used literally
>>   outside functions, which support file name handlers.
>
> Please read the docstring of file-remote-p, it has "nothing" to do
> with the above.  What you describe can misclassify both ways: remote
> access can be provided by the OS (using NFS/AFS/SSHFS/...), and local
> access can be provided by file-name-handlers (e.g. file:// URLs).

We have had this discussion already weeks ago, and I couldn't convince
you. So I shouldn't try it again.

I just see this function from a practical point of view, providing an
implementation for a special case (Tramp syntax). file-remote-p
is based on file name handler functions, that means it works over the
file name syntax.

That's why I am not creative enough to see how to implement your
interpretation of that function.

I'm keeping quiet now.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> I just see this function from a practical point of view, providing an
> implementation for a special case (Tramp syntax). file-remote-p
> is based on file name handler functions, that means it works over the
> file name syntax.

`file-remote-p' is not "based on file-name-handler function".  Just like
`copy-file', it is a file operation which can be overridden by file-name
handlers.  But for files accessed directly via the OS, it can (and
should) provide any implementation it wants, according to its
intended behavior.

I know it started as a file-name-handler-only thingy, but its new
docstring and this discussion shows that it goes further than that.

> That's why I am not creative enough to see how to implement your
> interpretation of that function.

Maybe you're thinking of "how can I implement it in Tramp" whereas the
Tramp-side of that function is already implemented and this thread is
not only concerned about its behavior for non-file-name-handled files.


        Stefan

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > As you said, the purpose of file-remote-p is to determine, 
> > without the cost of a remote access, whether a file name
> > represents a remote file. The aim in using it is to be able
> > to know that a file is remote, so you might then avoid the
> > cost of accessing it.
> >   
> 
> As mentioned in the earlier thread, remoteness is not a reliable 
> indicator of speed. You can have fast remote drives and slow local 
> drives. Maybe in the days of 10Mbps ethernet it was still 
> reasonable to 
> assume that networked drives were slow, but with gigabit 
> ethernet you'd 
> be hard pressed to notice a performance difference between a 
> networked 
> and local drive.

See my reply to your other mail.

Regardless of where it is and what is called, we need some quick estimate of
whether accessing a file will be relatively fast or slow. 

And the purpose of file-remote-p was decided (in the earlier thread) to be as I
said above: to give a quick estimate with the aim of being able to avoid a
performance penalty. IOW, as long as we don't have other functions to do that,
`file-remote-p' is the place to do it.

I have no objection to renaming things or adding another function that takes
other things into account. My aim is the stated aim of file-remote-p (as decided
in the earlier thread): to avoid trying to access (e.g. match name against
existing files) files when that access might be slow. I need at least a binary
determination: fast vs slow.

RE: testing for a remote file to include file on aWindows mappeddrive

[Drew Adams]

> > The expressed need is to know whether accessing a file with 
> > a given name might be significantly slower than accessing a
> > local file on an ordinary drive etc.
> 
> Is a file on a NAS volume connected via GbE slower than a file on a
> locally mounted crappy USB stick?

Did you read my mail? The point is not where a file is or how it is accessed.
The point is to be able, as far as practicable, to determine quickly whether a
given file name represents a file whose access would likely be slow.

However that's done, and whatever limitations on the accuracy or completeness
might pertain, that's the need. 

No one is asking for 100% accuracy. We already have a function, `file-remote-p',
that helps to some extent in this regard. But it (or some other function) could
be better.

RE: testing for a remote file to include file on aWindows mappeddrive

[Drew Adams]

> > `file-mounted-p', and you then test with (or ...) to see if there is
> > a possible slowdown, or we put the tests into `file-remote-p' and we
> > recognize that the name is not accurate.
> 
> Drew.  *Please* read the new docstring:

Please reread what you just quoted - "or we put the tests into `file-remote-p'".
IOW, I'm OK with that approach. And I don't really care whether the function
name is accurate. It's what the function does that is the topic of discussion.

>    A file is considered "remote" if accessing it is likely to 
>    be slower or less reliable than accessing local files.
> 
> Physical remoteness is irrelevant.  Even the transport protocol might
> not be relevant sometimes (tho it can be good input to a default
> heuristic).  The current implementation and behavior is not cast
> in stone.

I have no problem with that. I specifically said it would be OK to do any of the
following:

1. Put the other tests into `file-remote-p', so that it is a general test of
likely fast/slow file access.

2. Add other functions to treat other kinds of fast/slow determination from
remoteness.

3. Do #1 and perhaps rename `file-remote-p'.

Again, I have no problem with continuing to call it `file-remote-p' and beefing
it up to take into account some of the other slowness considerations people have
mentioned. My only need for now is to have a function that does that.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > I would prefer a function which returns "fast" or "slow".
> 
> We have that.  It's called `file-remote-p':
> 
>    A file is considered "remote" if accessing it is likely to 
>    be slower or less reliable than accessing local files.
> 
> Admittedly, the current implementation is not always good enough.
> 
> > One could consider additional regexps, which tell "when the mount
> > point matches this regexp, return `t' for file-mounted-p", and the
> > contrary case with the other regexp.  But this would make 
> > the handling more complex.
> 
> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
> should be added to file-remote-p.

It sounds like we violently agree. Perhaps even all of us. 

At least I, for one, don't care where the test is or what we call it. I just
want a test like `file-remote-p' that is a little bit more savvy.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
>> should be added to file-remote-p.

> It sounds like we violently agree.

Then send a patch,


        Stefan


PS: As for the "NET USE" thingy, I'd want a more generic solution first,
but I don't know of an equivalent in the Unix world.

Re: testing for a remote file to include file on aWindows mappeddrive

[Stefan Monnier]

> 3. Do #1 and perhaps rename `file-remote-p'.

My quoting the docstring was specifically to explain that we already
solved this part of the problem by (re)defining "remote".

As for slowness, I can't remember anyone complaining about file-remote-p
being slow (you quoted numbers comparing its speed to other code, but
that doesn't make it slow).


        Stefan

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> >> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
> >> should be added to file-remote-p.
> 
> > It sounds like we violently agree.
> 
> Then send a patch,

It's not clear to me what should be in the patch. I was saying that we seem to
agree that file-remote-p should deal with more than it does now. I would like it
to handle the stuff that is in the code I sent, though it need not do it the
same way. I think it makes sense to add both ffap regexps, before the current
file-remote-p code.

I guess I'm saying that I think we generally agree, but it's not clear to me
what you want in the patch.

RE: testing for a remote file to include file on aWindows mappeddrive

[Drew Adams]

> > 3. Do #1 and perhaps rename `file-remote-p'.
> 
> My quoting the docstring was specifically to explain that we already
> solved this part of the problem by (re)defining "remote".

OK by me.

> As for slowness, I can't remember anyone complaining about 
> file-remote-p being slow (you quoted numbers comparing its
> speed to other code, but that doesn't make it slow).

Part of its mission is to give you a quick check, so speed is not irrelevant.
I'm not saying it needs to be optimal, but if the more expensive parts of the
test can be avoided by placing them last, so much the better, no?

In any case, please let me know what you have in mind for the patch.

I would like it to also handle Windows mapped network drives, since their access
is a potential slowdown. But you mentioned handling also Unix mounted drives,
and Eli mentioned that he didn't want us to include treatment for only Windows
in this regard. And Michael has apparently already written code for dealing with
mounts (his unpublished `file-mounted-p' code). Can we bring together what needs
to be done for this aspect?

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> >> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
>> >> should be added to file-remote-p.
>> 
>> > It sounds like we violently agree.
>> 
>> Then send a patch,

> It's not clear to me what should be in the patch. I was saying that we seem to
> agree that file-remote-p should deal with more than it does now. I would like it
> to handle the stuff that is in the code I sent, though it need not do it the
> same way. I think it makes sense to add both ffap regexps, before the current
> file-remote-p code.

> I guess I'm saying that I think we generally agree, but it's not clear to me
> what you want in the patch.

ffap-rfs-regexp is a good first step.  But try something, and I'll tell
you what I like and what I don't.


        Stefan

Re: testing for a remote file to include file on aWindows mappeddrive

[Stefan Monnier]

> In any case, please let me know what you have in mind for the patch.

The reason why I don't write the patch is to save the time it takes to
"figure out what I have in mind".   If I had something in mind, I'd have
committed it long ago.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Mon, 21 Apr 2008 09:46:48 +0200
> Cc: 'Emacs-Devel' <emacs-devel@gnu.org>,
> 	'Michael Albinus' <michael.albinus@gmx.de>,
> 	'Stefan Monnier' <monnier@iro.umontreal.ca>,
> 	'Jason Rumney' <jasonr@gnu.org>
> 
> I believe, we are speaking about 2 different functions:
> 
> * file-remote-p returns t, if a file is not directly accessible by
>   underlying operating system's means. Such files always need some
>   special file name handler functions in Emacs for proper
>   handling. Such (absolute) file names cannot be used literally
>   outside functions, which support file name handlers.
> 
> * file-mounted-p (as working name) returns t, if a file looks like an
>   ordinary file from the operating system's point of view, but its
>   physical location is on another machine. Examples are nfs, smbfs or
>   sshfs mounts under GNU/Linux, or network shares under W32. This
>   property cannot be detected by file name handler functions. The
>   implementation shall be in the C core.
> 
> These functions are exclusive: file-remote-p and file-mounted-p shall
> not return t for the same filename. Since one function uses the
> filename only for checking the result, and the other function uses
> operating system properties, I don't believe it makes sense to merge
> them into one function.

110% agreement.  And, given several good arguments posted by Andreas,
it sounds like the best implementation for determining whether
accessing a file is likely to be slow would be a simple alist, set by
the user, since only the user knows which filesystems on her machine
cause slowdown.  I don't see how Emacs can determine that by itself
without actually accessing the file.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 21 Apr 2008 10:33:57 -0400
> Cc: 'Emacs-Devel' <emacs-devel@gnu.org>, Drew Adams <drew.adams@oracle.com>,
> 	'Jason Rumney' <jasonr@gnu.org>
> 
> Please read the docstring of file-remote-p, it has "nothing" to do
> with the above.  What you describe can misclassify both ways: remote
> access can be provided by the OS (using NFS/AFS/SSHFS/...), and local
> access can be provided by file-name-handlers (e.g. file:// URLs).

That may be how you _want_ it to be, but the current implementation
does not live up to this intent, and it (the current implementation)
is sufficiently old to have everybody and their dog become accustomed
to the peculiar interpretation of the term ``remote'' in Emacs.
Here's the current code in Emacs 23:

    (defun file-remote-p (file &optional identification connected)
    "..."
    `file-remote-p' will never open a connection on its own."
      (let ((handler (find-file-name-handler file 'file-remote-p)))
	(if handler
	    (funcall handler 'file-remote-p file identification connected)
	  nil)))

So a file that does not have a handler is _never_ remote.

Unless and until we change that, this whole argument based on the doc
string is purely academic.  OTOH, those who are accustomed to the
current notion of ``remote'', meaning there's a file handler somewhere
in Emacs, will probably be surprised if it starts returning non-nil
for NFS-mounted volumes some day.

That is why I agree with Michael that a new primitive is in order.
Why you insist on changing the semantics of `file-remote-p' is a
mystery to me.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Mon, 21 Apr 2008 00:47:05 -0700
> Cc: 'Michael Albinus' <michael.albinus@gmx.de>,
> 	'Stefan Monnier' <monnier@iro.umontreal.ca>,
> 	'Emacs-Devel' <emacs-devel@gnu.org>
> 
> Whether I access a local Windows drive (even a slow one) or a Windows mapped
> network drive that happens to be in India, there is a world of difference.

No one in their right minds will mount a drive half the globe away via
NFS or similar networking filesystem.  They will always use something
like Tramp or ftp.  So this problem simply does not exist in practice.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I know it started as a file-name-handler-only thingy, but its new
> docstring and this discussion shows that it goes further than that.

Please don't let us decide because of the docstring. It was changed not
so long ago, without a corresponding implementation, as Eli has
shown. So it represents your intentions for file-remote-p - which I do
respect, but which I don't share (yet).

>> That's why I am not creative enough to see how to implement your
>> interpretation of that function.
>
> Maybe you're thinking of "how can I implement it in Tramp" whereas the
> Tramp-side of that function is already implemented and this thread is
> not only concerned about its behavior for non-file-name-handled files.

I wouldn't have so much problems with your proposal if file-remote-p
would be used just for the decision "Is the access fast or slow". But
that's not the only use case for file-remote-p. This function is used
also in cases for the decision "Do I need special handling for accessing
this file". Most cases, this question doesn't arise, because the file
name handler functions decide it silently. But there are cases, those
functions are not sufficient.

>         Stefan

Best regards, Michael.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > Whether I access a local Windows drive (even a slow one) or 
> > a Windows mapped network drive that happens to be in India,
> > there is a world of difference.
> 
> No one in their right minds will mount a drive half the globe away via
> NFS or similar networking filesystem.  They will always use something
> like Tramp or ftp.  So this problem simply does not exist in practice.

I didn't say anything about NFS or similar; I mentioned Windows mapped network
drives.

I don't know what your idea of "does not exist in practice" is, but I work for a
fairly large corporation with developers and servers all over the planet, and
that is how we work.

I have mapped drives to UNIX and GNU/Linux machines (running Samba), and those
machines can be anywhere. In some cases, it is faster to access a machine far
away than a machine next door - it all depends on the particular network
properties in play (protocols, hardware, whatever). However, accessing a mapped
network drive is typically much slower than accessing a local hard drive (please
- no comments about USB sticks).

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> >> >> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
> >> >> should be added to file-remote-p.
> >> > It sounds like we violently agree.
> >> 
> >> Then send a patch,
> 
> > It's not clear to me what should be in the patch. I was 
> > saying that we seem to agree that file-remote-p should
> > deal with more than it does now. I would like it
> > to handle the stuff that is in the code I sent, though it 
> > need not do it the same way. I think it makes sense to add
> > both ffap regexps, before the current file-remote-p code.
> 
> > I guess I'm saying that I think we generally agree, but 
> > it's not clear to me what you want in the patch.
> 
> ffap-rfs-regexp is a good first step.  But try something, and 
> I'll tell you what I like and what I don't.

I don't think so. It's not clear from the discussion (Eli & Michael vs you) that
any consensus has been reached about what should be in `file-remote-p' (vs
elsewhere). And you're not interested in adding the NET USE part to
`file-remote-p' in the meantime.

I've expressed a need for something (`file-remote-p' or something else; I don't
care) that quickly distinguishes, based on the file name and only local info,
whether the file named is likely to be slow to access or fast.

I'll let it go at that. I hope you will eventually come to agreement on where to
do that, and I hope that someone with knowledge of how to do that does it.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> So a file that does not have a handler is _never_ remote.

That's irrelevant: look at the uses, and you'll see that most of them
use file-remote-p in the sense described in the docstring, so all that's
needed is to provide an implementation for the unhandled files.

This may require some changes in file-relative-name, because this one
does use file-remote-p in a more specific way, but everything else I've
looked at uses file-remote-p as a way to test "fast&reliable or not".


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> that's not the only use case for file-remote-p. This function is used
> also in cases for the decision "Do I need special handling for accessing
> this file". Most cases, this question doesn't arise, because the file
> name handler functions decide it silently. But there are cases, those
> functions are not sufficient.

These places need to be fixed because they are wrong,


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <jasonr@gnu.org>, <michael.albinus@gmx.de>, <monnier@iro.umontreal.ca>,
>         <emacs-devel@gnu.org>
> Date: Mon, 21 Apr 2008 15:11:49 -0700
> 
> > > Whether I access a local Windows drive (even a slow one) or 
> > > a Windows mapped network drive that happens to be in India,
> > > there is a world of difference.
> > 
> > No one in their right minds will mount a drive half the globe away via
> > NFS or similar networking filesystem.  They will always use something
> > like Tramp or ftp.  So this problem simply does not exist in practice.
> 
> I didn't say anything about NFS or similar; I mentioned Windows mapped network
> drives.

Same same.  Note that I did say ``or similar networking filesystem''.

> I don't know what your idea of "does not exist in practice" is, but I work for a
> fairly large corporation with developers and servers all over the planet, and
> that is how we work.

Then you have very strange sysadmins, IMHO.

> However, accessing a mapped
> network drive is typically much slower than accessing a local hard drive (please
> - no comments about USB sticks).

That is simply not true in general, even before situations like USB
sticks are concerned.  Of course, if you are in the US and the drive
is in Asia, that could be true, but that is a very unusual situation.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: michael.albinus@gmx.de,  emacs-devel@gnu.org,  drew.adams@oracle.com,  jasonr@gnu.org
> Date: Mon, 21 Apr 2008 22:46:49 -0400
> 
> > So a file that does not have a handler is _never_ remote.
> 
> That's irrelevant

Since when is the code irrelevant?  "Use the code, Luc" has been
always one of the slogans of free software, because code tells you
about what the software does much more than anything else.

> look at the uses, and you'll see that most of them
> use file-remote-p in the sense described in the docstring, so all that's
> needed is to provide an implementation for the unhandled files.

You are reading too much into your own interpretation, IMO.  It's true
that file access via a handler is necessarily slower than via the
normal OS file APIs, but it doesn't mean the reverse is true: that any
slow file access is necessarily to a file for which file-remote-p
should return non-nil.

> This may require some changes in file-relative-name, because this one
> does use file-remote-p in a more specific way, but everything else I've
> looked at uses file-remote-p as a way to test "fast&reliable or not".

Like I said, it's a mystery to me why you insist on changing
file-remote-p for a different semantics.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > > > Whether I access a local Windows drive (even a slow one) or 
> > > > a Windows mapped network drive that happens to be in India,
> > > > there is a world of difference.
> > > 
> > > No one in their right minds will mount a drive half the 
> > > globe away via NFS or similar networking filesystem.
> > > They will always use something like Tramp or ftp.  So
> > > this problem simply does not exist in practice.
> >
> > I didn't say anything about NFS or similar; I mentioned 
> > Windows mapped network drives.
> 
> Same same.  Note that I did say ``or similar networking filesystem''.

So it makes no sense to map a Windows network drive to a remote UNIX file system
with SMB. I learn something everyday. I'll start letting the others know.

> > I don't know what your idea of "does not exist in practice" 
> > is, but I work for a fairly large corporation with developers
> > and servers all over the planet, and that is how we work.
> 
> Then you have very strange sysadmins, IMHO.

Oracle has very strange sysadmins and engineers. They are not in their right
minds. What they are doing simply does not exist in practice; they just don't
know it. Thanks for setting me straight, at least.

> > However, accessing a mapped network drive is typically
> > much slower than accessing a local hard drive (please -
> > no comments about USB sticks).
> 
> That is simply not true in general, even before situations like USB
> sticks are concerned.  Of course, if you are in the US and the drive
> is in Asia, that could be true, but that is a very unusual situation.

According to the network engineers I spoke with today, it is true in general.
It's about the network properties (software and hardware), not necessarily the
distance. It makes sense to me, but I'm no networking expert. You win.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> This may require some changes in file-relative-name, because this one
>> does use file-remote-p in a more specific way, but everything else I've
>> looked at uses file-remote-p as a way to test "fast&reliable or not".

> Like I said, it's a mystery to me why you insist on changing
> file-remote-p for a different semantics.

I'm saying it's not a different semantics.  It's nothing more than
a bug fix.  That `file-remote-p on an sshfs filesystem returns nil is
a bug.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <jasonr@gnu.org>, <michael.albinus@gmx.de>, <monnier@iro.umontreal.ca>,
>         <emacs-devel@gnu.org>
> Date: Mon, 21 Apr 2008 21:43:26 -0700
> 
> > > > > Whether I access a local Windows drive (even a slow one) or 
> > > > > a Windows mapped network drive that happens to be in India,
> > > > > there is a world of difference.
> > > > 
> > > > No one in their right minds will mount a drive half the 
> > > > globe away via NFS or similar networking filesystem.
> > > > They will always use something like Tramp or ftp.  So
> > > > this problem simply does not exist in practice.
> > >
> > > I didn't say anything about NFS or similar; I mentioned 
> > > Windows mapped network drives.
> > 
> > Same same.  Note that I did say ``or similar networking filesystem''.
> 
> So it makes no sense to map a Windows network drive to a remote UNIX file system
> with SMB.

Only if the drive is ``half the globe away''.

> > > However, accessing a mapped network drive is typically
> > > much slower than accessing a local hard drive (please -
> > > no comments about USB sticks).
> > 
> > That is simply not true in general, even before situations like USB
> > sticks are concerned.  Of course, if you are in the US and the drive
> > is in Asia, that could be true, but that is a very unusual situation.
> 
> According to the network engineers I spoke with today, it is true in general.
> It's about the network properties (software and hardware), not necessarily the
> distance. It makes sense to me, but I'm no networking expert. You win.

I hope this is not just about winning.  Network access is indeed
slower than local hard disk access, but only by small factors (again,
unless the network is extremely slow, which is not what one sees
normally).  By contrast, access to files for which file-remote-p
returns non-nil now is several orders of magnitude slower (10 seconds
is not unusual, vs fraction of a second for a networked file).

Here's a random example, from an XP SP2 machine where I'm typing this.
W: is a networked drive mounted via a corporate network, while D: is a
local drive.  `ls' is a Windows port of GNU ls from Coreutils 6.9:

  D:\usr>timep ls -l test
  total 0
  -rw-rw-rw- 1 P0009057 Domain Users 0 2008-04-22 08:49 foobar
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir1
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir10
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir2
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir3
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir4
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir5
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir6
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir7
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir8
  drwxrwxrwx 1 P0009057 Domain Users 0 2008-04-22 08:54 tdir9

  real    00h00m00.031s
  user    00h00m00.015s
  sys     00h00m00.015s

  D:\usr>timep ls -l w:/
  total 0
  drwxrwxrwx 1 P0009057       None         0 2007-02-14 20:13 Accounting
  drwxrwxrwx 1 P0009057       None         0 2008-04-16 08:25 CTI
  drwxrwxrwx 1 Administrators Domain Users 0 2004-07-21 12:54 Management
  drwxrwxrwx 1 Administrators Domain Users 0 2007-03-25 11:45 Marketing
  drwxrwxrwx 1 P0009057       None         0 2008-02-20 10:41 Nob
  drwxrwxrwx 1 Administrators Domain Users 0 2007-06-20 13:53 Projects
  drwxrwxrwx 1 Administrators Domain Users 0 2008-03-24 14:46 RealTime
  drwxrwxrwx 1 P0009057       None         0 2007-06-28 11:22 SIMTECH
  drwxrwxrwx 1 P0009057       None         0 2007-10-24 15:48 Telecom Solutions
  drwxrwxrwx 1 Administrators Domain Users 0 2003-10-26 16:19 TSG-QA
  drwxrwxrwx 1 Administrators Domain Users 0 2008-04-06 11:44 TSG_Marketing Material

  real    00h00m00.109s
  user    00h00m00.015s
  sys     00h00m00.031s

This is a 3-fold slowdown for a networked volume, and accessing 11
directories with a plethora of file I/O APIs still takes a fraction of
a second.

By contrast, just typing "plink fencepost.gnu.org 'ls -l test'" to
produce a listing of a directory on a drive ``half a globe away'' from
where I physically sit takes 4 seconds, and that's even without the
Tramp overhead that could well double or even triple that time.  So
access to what we now call ``remote'' files is 2 orders of magnitude
slower than accessing a local file.

Given these numbers, I fail to see how the same simple predicate can
satisfy the need for detecting both types of ``remote'' files.

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: michael.albinus@gmx.de,  emacs-devel@gnu.org,  drew.adams@oracle.com,  jasonr@gnu.org
> Date: Tue, 22 Apr 2008 01:46:10 -0400
> 
> >> This may require some changes in file-relative-name, because this one
> >> does use file-remote-p in a more specific way, but everything else I've
> >> looked at uses file-remote-p as a way to test "fast&reliable or not".
> 
> > Like I said, it's a mystery to me why you insist on changing
> > file-remote-p for a different semantics.
> 
> I'm saying it's not a different semantics.

I'm obviously failing to explain myself, because I was trying to say
that the current semantics is: ``remote'' files are files accessed via
a handler, as opposed to normal file I/O API (`open', `read', etc.).

RE: testing for a remote file to include file on a Windowsmapped drive

[Drew Adams]

> This is a 3-fold slowdown for a networked volume, and accessing 11
> directories with a plethora of file I/O APIs still takes a fraction of
> a second.
> 
> By contrast, just typing "plink fencepost.gnu.org 'ls -l test'" to
> produce a listing of a directory on a drive ``half a globe away'' from
> where I physically sit takes 4 seconds, and that's even without the
> Tramp overhead that could well double or even triple that time.  So
> access to what we now call ``remote'' files is 2 orders of magnitude
> slower than accessing a local file.
> 
> Given these numbers, I fail to see how the same simple predicate can
> satisfy the need for detecting both types of ``remote'' files.

You're arguing against a single predicate. I have no problem with multiple
predicates, as I stated clearly several times, including yesterday.

My understanding from the beginning has been that accessing files on a network
drive is generally faster than accessing the remote files that are reported
traditionally by `file-remote-p', but that both are generally slower than
accessing a typical local drive. You have confirmed that, so it seems we pretty
much agree now - at least about what happens, if not what to do about it.

This came up because a user of some of my code reported slow access in a
particular context, and that was for files on a Windows network drive. There was
no problem for local files, but, for this context, accessing the network drive
files was "too slow".

While remote files in the traditional sense might typically be slower still, the
point is that both are typically slower than local files, and, depending on the
context, both could be "too slow". Andromeda is orders of magnitude farther than
the sun, but both are too high for me to jump.

I want a way to distinguish the local from both kinds of "remote". (Again, I'm
interested only in time here, not whether the files are actually local or
remote.) If that's via two or three predicates instead of one, I don't mind.

With your help, I cobbled together something that works, for my context, better
than what `file-remote-p' alone gives. I thank you again for your NET USE
suggestion for Windows mapped drives - that helps a lot. I would like Emacs to
deal with this more generally (e.g. not just Windows). I also proposed that the
file-name patterns that `ffap-file-remote-p' tests be incorporated in
`file-remote-p'. That's all.

I described my use case a little bit. I can describe it more if that helps. But
the point is that I'm looking for a quick (local) way to test file names that
will give an idea whether accessing them might be slower than typical local
access.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> So a file that does not have a handler is _never_ remote.
>
> That's irrelevant: look at the uses, and you'll see that most of them
> use file-remote-p in the sense described in the docstring, so all that's
> needed is to provide an implementation for the unhandled files.
>
> This may require some changes in file-relative-name, because this one
> does use file-remote-p in a more specific way, but everything else I've
> looked at uses file-remote-p as a way to test "fast&reliable or not".

I have available a checkout from April 7th. Scanning lisp/*, there are
40 places file-remote-p is called. Let's not regard the 17 calls of
tramp*.el, they could be replaced simply by another, Tramp internal,
function.

From the other 23 calls, I would count 5 calls related to the question
"Is the access slow"; the other calls are due to the file name handler
thingies, which require another implementation of the respective
functionality.

And, if you say that the calls of file-remote-p shall be replaced by
something else, which is closer to the file name handler
functionality: which function do you have in mind? Back in February,
you've spoken about file-local-name/unhandled-file-name. These
functions don't exist (yet).

Stefan, honestly: I don't understand why file-remote-p needs such a
redefinition. I assume there is a plan behind, which we don't know.
It would really be helpful if we could understand your intention of
this change.

One explanation, also in February, was that you want to have

(file-remote-p "file:///toto/titi") => nil

But with the same reasoning, you could say

(file-remote-p "/sudo:$USER@localhost:/toto/titi") => nil

And that doesn't work, IMHO.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> * file-mounted-p (as working name) returns t, if a file looks like an
>>   ordinary file from the operating system's point of view, but its
>>   physical location is on another machine. Examples are nfs, smbfs or
>>   sshfs mounts under GNU/Linux, or network shares under W32. This
>>   property cannot be detected by file name handler functions. The
>>   implementation shall be in the C core.
>> 
>> These functions are exclusive: file-remote-p and file-mounted-p shall
>> not return t for the same filename. Since one function uses the
>> filename only for checking the result, and the other function uses
>> operating system properties, I don't believe it makes sense to merge
>> them into one function.
>
> 110% agreement.  And, given several good arguments posted by Andreas,
> it sounds like the best implementation for determining whether
> accessing a file is likely to be slow would be a simple alist, set by
> the user, since only the user knows which filesystems on her machine
> cause slowdown.  I don't see how Emacs can determine that by itself
> without actually accessing the file.

I like this idea. Let's forget file-mounted-p; Emacs cannot determine
by itself good enough whether something is fast or slow. And even
"fast" and "slow" depend on the user and her preferences.

Maybe we shall have something like `slow-accessible-files-list', a
list of regexps and predicates which denote file names with slow
access. (Please don't blame me for the name; it could be anything else).

If a file name matches one of the regexps, or a predicate with the
file name as argument returns non-nil, then the access to the file is
regarded as, hmm, slow. It could be something like this:

(defvar slow-accessible-files-list
  '(file-remote-p    ;; All files with file name handler functions.
    "^/media/disk")) ;; My USB stick.

And a function, let's call it `file-slow-accessible-p', might evaluate
that list.

Best regards, Michael.

Re: testing for a remote file to include file on a Windowsmapped drive

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <emacs-devel@gnu.org>, <michael.albinus@gmx.de>,
>         <monnier@iro.umontreal.ca>, <jasonr@gnu.org>
> Date: Tue, 22 Apr 2008 07:09:07 -0700
> 
> You're arguing against a single predicate.

That part was aimed at Stefan, not at you.

> My understanding from the beginning has been that accessing files on a network
> drive is generally faster than accessing the remote files that are reported
> traditionally by `file-remote-p', but that both are generally slower than
> accessing a typical local drive. You have confirmed that, so it seems we pretty
> much agree now - at least about what happens, if not what to do about it.

No, we do NOT agree.  I think 100msec is practically indistinguishable
from 30msec, but 4 or 10 seconds are quite another story.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> I have available a checkout from April 7th. Scanning lisp/*, there are
> 40 places file-remote-p is called. Let's not regard the 17 calls of
> tramp*.el, they could be replaced simply by another, Tramp internal,
> function.

> From the other 23 calls, I would count 5 calls related to the question
> "Is the access slow"; the other calls are due to the file name handler
> thingies, which require another implementation of the respective
> functionality.

Looking at it again, I see that it is a bit less uniform than
I remembered, indeed.

> And, if you say that the calls of file-remote-p shall be replaced by
> something else, which is closer to the file name handler
> functionality: which function do you have in mind? Back in February,
> you've spoken about file-local-name/unhandled-file-name. These
> functions don't exist (yet).

> Stefan, honestly: I don't understand why file-remote-p needs such a
> redefinition.  I assume there is a plan behind, which we don't know.
> It would really be helpful if we could understand your intention of
> this change.

I have no idea which change you're talking about.
I just have noticed several different concerns that tend to be
conflated:
- reliability and performance.
- ability to access the file via syscalls.
- whether relative file names can be used from one file name to another.
- whether a file's contents is directly accessible via syscalls.

The first is usually/always answered by calling `file-remote-p'.

The second is sometimes answered by calling file-remote-p, although it
does not give the right answer for /:/foo, and it clashes with the
previous use of file-remote-p if you want to say that "file://" is
not remote.  A closer match for this is unhandled-file-name-directory,
except that it historically didn't give useful information (it typically
returned /tmp rather than nil for handled file names) and it only
works for directories.

The third is currently answered by file-remote-p.  It's only used in
file-relative-name, AFAIK, so it doesn't matter much where the info
comes from, as long as file-relative-name is updated accordingly.
In any case abusing file-remote-p for that seems like a bad idea.

The last one is relevant for compressed files, encrypted files, and
files extracted from tar/zip/ar/rar/... archives.  This is usually
answered by a bunch of ad-hoc tests listing all the situations of which
the author was aware.  E.g. it'll need update for auto-encryption-mode.
We should introduce a new function for that.  Something like
unhandled-file-contents.  OTOH unhandled-file-contents can only return
non-nil if unhandled-file-name is non-nil, so maybe we should combine
the two, and use an argument `contents' to indicate whether we only care
about accessing the file (container) via syscalls or accessing also
its contents.

> One explanation, also in February, was that you want to have
> (file-remote-p "file:///toto/titi") => nil
> But with the same reasoning, you could say
> (file-remote-p "/sudo:$USER@localhost:/toto/titi") => nil
> And that doesn't work, IMHO.

I don't have a strong opinion on "/sudo:$USER@localhost:/toto/titi":
I think it would be OK to return nil, but non-nil is OK as well: access
via sudo is probably slower and may even require user-input.
I'm not sure why you say it wouldn't work to return nil for it.


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org,  drew.adams@oracle.com,  jasonr@gnu.org
> Date: Tue, 29 Apr 2008 03:50:28 -0400
> 
> I just have noticed several different concerns that tend to be
> conflated:
> - reliability and performance.
> - ability to access the file via syscalls.
> - whether relative file names can be used from one file name to another.
> - whether a file's contents is directly accessible via syscalls.

I think Drew had yet another concern, which is uncovered by the above.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> I just have noticed several different concerns that tend to be
>> conflated:
>> - reliability and performance.
>> - ability to access the file via syscalls.
>> - whether relative file names can be used from one file name to another.
>> - whether a file's contents is directly accessible via syscalls.

> I think Drew had yet another concern, which is uncovered by the above.

Which one?


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Eli Zaretskii]

> X-Spam-Status: No, score=-1.4 required=5.0 tests=AWL,BAYES_00,
> 	FORGED_RCVD_HELO,SPF_SOFTFAIL,UNPARSEABLE_RELAY autolearn=no 
> 	version=3.1.0
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 29 Apr 2008 16:39:50 -0400
> Cc: emacs-devel@gnu.org, michael.albinus@gmx.de, drew.adams@oracle.com,
> 	jasonr@gnu.org
> 
> >> I just have noticed several different concerns that tend to be
> >> conflated:
> >> - reliability and performance.
> >> - ability to access the file via syscalls.
> >> - whether relative file names can be used from one file name to another.
> >> - whether a file's contents is directly accessible via syscalls.
> 
> > I think Drew had yet another concern, which is uncovered by the above.
> 
> Which one?

The case with remote volumes mounted via the OS facilities such as
NFS.  They are almost identical in reliability and performance to
local volumes, and cannot be distinguished using file-remote-p, but
Drew insisted on being able to distinguish them anyway.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> >> I just have noticed several different concerns that tend to be
>> >> conflated:
>> >> - reliability and performance.
>> >> - ability to access the file via syscalls.
>> >> - whether relative file names can be used from one file name to another.
>> >> - whether a file's contents is directly accessible via syscalls.
>> 
>> > I think Drew had yet another concern, which is uncovered by the above.
>> 
>> Which one?

> The case with remote volumes mounted via the OS facilities such as
> NFS.  They are almost identical in reliability and performance to
> local volumes, and cannot be distinguished using file-remote-p, but
> Drew insisted on being able to distinguish them anyway.

Not sure if it's a different case.  IIUC he's worried about the
different reliability and/or performance of those volumes, but
admittedly, that wasn't stated clearly,


        Stefan

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> >> >> I just have noticed several different concerns that tend to be
> >> >> conflated:
> >> >> - reliability and performance.
> >> >> - ability to access the file via syscalls.
> >> >> - whether relative file names can be used from one file 
> >> >>   name to another.
> >> >> - whether a file's contents is directly accessible via syscalls.
> >> 
> >> > I think Drew had yet another concern, which is uncovered 
> >> > by the above.
> >> 
> >> Which one?
> 
> > The case with remote volumes mounted via the OS facilities such as
> > NFS.  They are almost identical in reliability and performance to
> > local volumes, and cannot be distinguished using file-remote-p, but
> > Drew insisted on being able to distinguish them anyway.
> 
> Not sure if it's a different case.  IIUC he's worried about the
> different reliability and/or performance of those volumes, but
> admittedly, that wasn't stated clearly,

Huh? I stated clearly and explicitly several times that my only concern was
performance.

I want a test that is quick and tells whether a file name names a file whose
access is likely to be slow, compared to a file on a local hard disk (normal
disk, no USB stick or CD etc.).

There are apparently three levels in terms of performance, from fastest to
slowest: (1) local, (2) mapped network drive, (3) truly remote (e.g. ftp/tramp
access). According to Eli's test figures, #2 was three times slower than #1, and
#3 was "two orders of magnitude" slower than #1. I won't argue the numbers -
suffice it to say that we agree that there are three levels.

`file-remote-p' currently distinguishes #1 and #2 from #3. That's reasonable,
since #3 is so much slower, per Eli's figures.

Nevertheless, I want to also be able to distinguish #1 from #2 and #3. For some
applications, a file on a mapped network drive is, like a truly remote file, too
slow, and I want to avoid accessing it. Whether the same predicate works for
both types of distinction or multiple predicates are provided, I don't care.
Whether it is `file-remote-p' or some other predicate that does what I want, I
don't care.

`file-remote-p' is also slower than it could be, at least for some file names.
For file names that `ffap-file-remote-p' determines are remote, calling it is
faster - Michael found it to be about 70 times faster than `file-remote-p'. I
suggested incorporating the ffap test into `file-remote-p'.

In any case, my only concern is performance, in two senses: (a) The main
concern, and the aim, is to avoid the cost of slow file access. (b) A secondary
concern is to make that check itself fast.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> `file-remote-p' is also slower than it could be, at least for some
> file names.  For file names that `ffap-file-remote-p' determines are
> remote, calling it is faster - Michael found it to be about 70 times
> faster than `file-remote-p'. I suggested incorporating the ffap test
> into `file-remote-p'.

I won't worry about the performance of file-remote-p until someone can
show me a real case where it matters.  This is also true of pretty much
anything else than `file-remote-p'.  It's called "don't do premature
optimizations" and there's a good reason for it: you need the concrete
case in order to know for what to optimize.


        Stefan

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > `file-remote-p' is also slower than it could be, at least for some
> > file names.  For file names that `ffap-file-remote-p' determines are
> > remote, calling it is faster - Michael found it to be about 70 times
> > faster than `file-remote-p'. I suggested incorporating the ffap test
> > into `file-remote-p'.
> 
> I won't worry about the performance of file-remote-p until someone can
> show me a real case where it matters.  This is also true of 
> pretty much anything else than `file-remote-p'.  It's called "don't do 
> premature optimizations" and there's a good reason for it: you need the 
> concrete case in order to know for what to optimize.

`file-remote-p' already tests using some regexps - why not add the ffap
regexp(s)? You already agreed to that, in fact:

 >> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
 >> should be added to file-remote-p.

That's all I was saying there.

I offered to describe my use case, but it won't prove anything special in this
regard. Suffice it to say that I would like the test to be about as quick as
`ffap-rfs-regexp' for the regexps that `ffap-rfs-regexp' handles.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I have no idea which change you're talking about.
> I just have noticed several different concerns that tend to be
> conflated:
> - reliability and performance.
> - ability to access the file via syscalls.
> - whether relative file names can be used from one file name to another.
> - whether a file's contents is directly accessible via syscalls.
>
> The first is usually/always answered by calling `file-remote-p'.

At least NOT with the current implementation. `file-remote-p' just
calls file name handlers, if any. And there seem to be only 2 such
handlers until now: `ange-ftp-file-remote-p' and
`tramp-handle-file-remote-p'.

There are also `ediff-file-remote-p' and `ffap-file-remote-p', but
they are not called from `file-remote-p'; they use this function instead.

`file-remote-p', as of today, simply answers the question, whether
there are file name handlers involved accessing a file. If it returns
non-nil, one could deduce that the access to the file might be less
reliable and less performant, but not vice-versa. And even this
interpretation was not the reason why `file-remote-p' was introduced.(*)

Maybe that is the major reason of our misunderstanding: I just read
what is implemented (yet), and I can guess only what you want to have.

I know you have adapted the docstring of `file-remote-p'. But there is
no matching implementation.

Stefan, maybe it is time to list en detail the required changes you
have in mind for this?

> The second is sometimes answered by calling file-remote-p, although it
> does not give the right answer for /:/foo, and it clashes with the
> previous use of file-remote-p if you want to say that "file://" is
> not remote.  A closer match for this is unhandled-file-name-directory,
> except that it historically didn't give useful information (it typically
> returned /tmp rather than nil for handled file names) and it only
> works for directories.

"/:" is handled in `file-name-non-special', there is no problem wrt
syscalls I believe.

"file:///foo" is a special case, indeed. I have no idea, whether the
implementation of `url-handler-mode' is complete (a year ago I've
looked at the webDAV implementation, and it wasn't ready).

If I could understand better what needs to be done for url*.el, I
would be willing to support. Is there a (prioritized) todo list what
shall be done?

> The third is currently answered by file-remote-p.  It's only used in
> file-relative-name, AFAIK, so it doesn't matter much where the info
> comes from, as long as file-relative-name is updated accordingly.
> In any case abusing file-remote-p for that seems like a bad idea.

D'accord.

> The last one is relevant for compressed files, encrypted files, and
> files extracted from tar/zip/ar/rar/... archives.  This is usually
> answered by a bunch of ad-hoc tests listing all the situations of which
> the author was aware.  E.g. it'll need update for auto-encryption-mode.
> We should introduce a new function for that.  Something like
> unhandled-file-contents.  OTOH unhandled-file-contents can only return
> non-nil if unhandled-file-name is non-nil, so maybe we should combine
> the two, and use an argument `contents' to indicate whether we only care
> about accessing the file (container) via syscalls or accessing also
> its contents.

Here I tend to agree as well.

(*): I haven't found a discussion about in the archives; Richard added
     this to files.el the same time Tramp was integrated into Emacs.

>         Stefan

Best regards, Michael.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

"Drew Adams" <drew.adams@oracle.com> writes:

> `file-remote-p' already tests using some regexps - why not add the ffap
> regexp(s)? You already agreed to that, in fact:
>
>  >> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
>  >> should be added to file-remote-p.

Please don't forget that `file-remote-p' is based on file name
handlers. If you want to add `ffap-rfs-regexp', there shall be a
corresponding implementation of such a handler.

Everything else would contradict the clear design, and it would
increase the maintenance burden.

> I offered to describe my use case, but it won't prove anything special in this
> regard. Suffice it to say that I would like the test to be about as quick as
> `ffap-rfs-regexp' for the regexps that `ffap-rfs-regexp' handles.

The test via `file-remote-p' is slower than via `ffap-file-remote-p'.
In my profiling, it was 0.0007140419 sec vs 0.00001117 sec for one
call. A factor of 64, yes, but does it really count?

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> > I offered to describe my use case, but it won't prove 
> > anything special in this regard. Suffice it to say that
> > I would like the test to be about as quick as
> > `ffap-rfs-regexp' for the regexps that `ffap-rfs-regexp' handles.
> 
> The test via `file-remote-p' is slower than via `ffap-file-remote-p'.
> In my profiling, it was 0.0007140419 sec vs 0.00001117 sec for one
> call. A factor of 64, yes, but does it really count?

Yes, for my case it does.

It all depends what the test is used for, when, and how often. This is why I've
argued for something flexible - either more than one test function or an ability
to call the test function differently.

For some applications, a slowdown of 3x or 64x makes a big difference. For
others, it is negligible.

I am not arguing that every use of a file-name test to determine the likely
access time needs to consider network mapped drives or files whose names match
`ffap-rfs-regexp' as "remote" (in the sense of slow). I am looking for a test
that allows that determination, but I'm not asking that that definition of
"slow" be applicable to every use.

Re: testing for a remote file to include file on a Windows mapped drive

[Michael Albinus]

"Drew Adams" <drew.adams@oracle.com> writes:

>> > I offered to describe my use case, but it won't prove 
>> > anything special in this regard. Suffice it to say that
>> > I would like the test to be about as quick as
>> > `ffap-rfs-regexp' for the regexps that `ffap-rfs-regexp' handles.
>> 
>> The test via `file-remote-p' is slower than via `ffap-file-remote-p'.
>> In my profiling, it was 0.0007140419 sec vs 0.00001117 sec for one
>> call. A factor of 64, yes, but does it really count?
>
> Yes, for my case it does.

Oops, maybe there is a misunderstanding? (Forgive me my bad English)
I meant the time a call of `file-remote-p' needs to run. I don't
comment *why* you need such a test.

Even if you have a function which calls 100x `file-remote-p', this would
last 0.07140419 sec only. Does it really count?

Best regards, Michael.

RE: testing for a remote file to include file on a Windows mapped drive

[Drew Adams]

> >> > I offered to describe my use case, but it won't prove 
> >> > anything special in this regard. Suffice it to say that
> >> > I would like the test to be about as quick as
> >> > `ffap-rfs-regexp' for the regexps that `ffap-rfs-regexp' handles.
> >> 
> >> The test via `file-remote-p' is slower than via 
> `ffap-file-remote-p'.
> >> In my profiling, it was 0.0007140419 sec vs 0.00001117 sec for one
> >> call. A factor of 64, yes, but does it really count?
> >
> > Yes, for my case it does.
> 
> Oops, maybe there is a misunderstanding? (Forgive me my bad English)
> I meant the time a call of `file-remote-p' needs to run. I don't
> comment *why* you need such a test.
> 
> Even if you have a function which calls 100x `file-remote-p', 
> this would
> last 0.07140419 sec only. Does it really count?

Yes, I believe so.

Again, I don't care if it is `file-remote-p' that makes such a check or some
other function. I won't enter join discussion with Stefan about where to do
these things and what the "real" purpose of `file-remote-p' should be, because
I'm indifferent about that.

Whatever function (`file-remote-p' or some new function) tests for probable slow
file access, I think it should try to be quick. And if it can sometimes be 64
times quicker by testing the `ffap-rfs-regexp' first, well, that is useful time
gained. IOW, instead of my calling both `ffap-remote-p' and `file-remote-p', in
that order, I would prefer to call a single function. But admittedly, I can
continue to call both.

I think the real discussion about whether to incorporate `ffap-rfs-regexp' into
`file-remote-p' has to do with the disagreement you and Stefan have wrt the
mission (aim) of `file-remote-p'.

My point here is that a test for possibly slow file access should include a test
for `ffap-rfs-regexp', and it should do that before the slower checks used by
the current `file-remote-p'. What the function that tests for possibly slow file
access is called (`file-remote-p' or something else) is another matter, about
which I have no opinion.

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

> There are also `ediff-file-remote-p' and `ffap-file-remote-p', but
> they are not called from `file-remote-p'; they use this function instead.

(defun ediff-file-remote-p (file-name)
  (file-remote-p file-name))

As for ffap-file-remote-p, you're right, but it's just because
file-remote-p didn't exist back then.  Otherwise, it would also have
used file-remote-p.

BTW, when I said "The first is usually/always answered by calling
`file-remote-p'" I did not mean to say that file-remote-p always gives
the right answer, but that whenever someone needs to know this
information, they currently call file-remote-p.

> Maybe that is the major reason of our misunderstanding: I just read
> what is implemented (yet), and I can guess only what you want to have.

Instead, I look at the callers and try to guess what they want.
Since we're talking about deciding what file-remote-p should do, its
current implementation seems much less relevant.

> Stefan, maybe it is time to list en detail the required changes you
> have in mind for this?

Which changes?  The post you reply to is just as far as I've gotten in
trying to figure out what we should have.  I have no hidden plan in
this regard.

>> The second is sometimes answered by calling file-remote-p, although it
>> does not give the right answer for /:/foo, and it clashes with the
>> previous use of file-remote-p if you want to say that "file://" is
>> not remote.  A closer match for this is unhandled-file-name-directory,
>> except that it historically didn't give useful information (it typically
>> returned /tmp rather than nil for handled file names) and it only
>> works for directories.

> "/:" is handled in `file-name-non-special', there is no problem wrt
> syscalls I believe.

There is: if you pass "/:/usr" to a syscall it will not find it.
I.e. wherever the file-name-handlers aren't used such a name will fail
(e.g. external processes, as well as a few cases internally, such as for
display properties specifying images by giving their file name, IIRC).

> "file:///foo" is a special case, indeed.

AFAIK, It's the exact same case as "/:".

> I have no idea, whether the implementation of `url-handler-mode' is
> complete (a year ago I've looked at the webDAV implementation, and it
> wasn't ready).

It's very much non-complete, but I use it all the time to access Elisp
or tarball packages on the web (they get displayed much better than in
Firefox, obviously), so it "does the job" for what I use it for.

> If I could understand better what needs to be done for url*.el, I
> would be willing to support. Is there a (prioritized) todo list what
> shall be done?

Not that I know.

>> The third is currently answered by file-remote-p.  It's only used in
>> file-relative-name, AFAIK, so it doesn't matter much where the info
>> comes from, as long as file-relative-name is updated accordingly.
>> In any case abusing file-remote-p for that seems like a bad idea.
> D'accord.

Wunderbar!


        Stefan

Re: testing for a remote file to include file on a Windows mapped drive

[Stefan Monnier]

>> `file-remote-p' already tests using some regexps - why not add the ffap
>> regexp(s)? You already agreed to that, in fact:
>> 
>> >> That's what ffap-rfs-regexp tries to do.  And yes, I think it 
>> >> should be added to file-remote-p.

> Please don't forget that `file-remote-p' is based on file name
> handlers. If you want to add `ffap-rfs-regexp', there shall be a
> corresponding implementation of such a handler.

I don't think so.  The way I look at it, file-remote-p is a normal file
operation: i.e. it first dispatches to file-name-handlers if any, and if
none it handles the case of files accessed directly via the
OS's filesystem.  Currently, this part of the implementation just always
returns nil, but it should be improved with a file-remote-regexp so
users can tell Emacs that /afs, /net, /sshfs are "OS-local filesystems
but with performance and reliability-semantics of remote files".


        Stefan