Tramp and recursive file operations

Tramp and recursive file operations

[Michael Albinus]

Hi,

recursive copy or deletion of files can last a long time, because all
operations are performed file by file. This could be much faster, if
Tramp would be allowed to apply such operations in one step, for a whole
directory. Think about "scp" or "rsync".

Therefore, I propose to add a file name handler to `dired-copy-file-recursive'.

Recursive file deletion happens via `dired-delete-file'. Here I propose
to add a new function `dired-delete-file-recursive', which takes over
the recursive part of `dired-delete-file', and which calls also a file
name handler when appropriate.

I volunteer to implement such handlers in Tramp :-)

What do people think?

Best regards, Michael.

Re: Tramp and recursive file operations

[joakim]

Michael Albinus <michael.albinus@gmx.de> writes:

> Hi,
>
> recursive copy or deletion of files can last a long time, because all
> operations are performed file by file. This could be much faster, if
> Tramp would be allowed to apply such operations in one step, for a whole
> directory. Think about "scp" or "rsync".
>
> Therefore, I propose to add a file name handler to `dired-copy-file-recursive'.
>
> Recursive file deletion happens via `dired-delete-file'. Here I propose
> to add a new function `dired-delete-file-recursive', which takes over
> the recursive part of `dired-delete-file', and which calls also a file
> name handler when appropriate.
>
> I volunteer to implement such handlers in Tramp :-)
>
> What do people think?

As a user, I welcome all improvements to Tramp! I use it every day.


>
> Best regards, Michael.
>
-- 
Joakim Verona

Re: Tramp and recursive file operations

[Stefan Monnier]

> recursive copy or deletion of files can last a long time, because all
> operations are performed file by file. This could be much faster, if
> Tramp would be allowed to apply such operations in one step, for a whole
> directory. Think about "scp" or "rsync".

> Therefore, I propose to add a file name handler to `dired-copy-file-recursive'.

> Recursive file deletion happens via `dired-delete-file'. Here I propose
> to add a new function `dired-delete-file-recursive', which takes over
> the recursive part of `dired-delete-file', and which calls also a file
> name handler when appropriate.

> I volunteer to implement such handlers in Tramp :-)

> What do people think?

It seems OK, but I'd rather see them renamed to copy-file-recursive and
delete-file-recursive at the same time and made independent from dired.


        Stefan

Re: Tramp and recursive file operations

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> It seems OK, but I'd rather see them renamed to copy-file-recursive and
> delete-file-recursive at the same time and made independent from dired.

OK. Looking outside dired, there is already delete-directory, good for
empty directories. Maybe we add an optional parameter RECURSIVE?

A copy-directory function does not exist. What about allowing the first
parameter of copy-file to be a directory name? This would imply to copy
recursively.

Comments?

>         Stefan

Best regards, Michael.

Re: Tramp and recursive file operations

[Stefan Monnier]

>> It seems OK, but I'd rather see them renamed to copy-file-recursive and
>> delete-file-recursive at the same time and made independent from dired.
> OK. Looking outside dired, there is already delete-directory, good for
> empty directories. Maybe we add an optional parameter RECURSIVE?

Sounds OK, except for the fact that delete-directory is implemented in
C, so you'd have to code the recursion in C as well.

> A copy-directory function does not exist. What about allowing the first
> parameter of copy-file to be a directory name? This would imply to copy
> recursively.

If we were starting over from scratch, that would be fine, but I'm
afraid that some code somewhere relies on the fact that copy-file only
copies files and not directories.  And of course, copy-file is also
implemented in C, so you'd have to code the recursion in C as well.


        Stefan

Re: Tramp and recursive file operations

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> OK. Looking outside dired, there is already delete-directory, good for
>> empty directories. Maybe we add an optional parameter RECURSIVE?
>
> Sounds OK, except for the fact that delete-directory is implemented in
> C, so you'd have to code the recursion in C as well.

I've seen that. I'll do my best ...

>> A copy-directory function does not exist. What about allowing the first
>> parameter of copy-file to be a directory name? This would imply to copy
>> recursively.
>
> If we were starting over from scratch, that would be fine, but I'm
> afraid that some code somewhere relies on the fact that copy-file only
> copies files and not directories.  And of course, copy-file is also
> implemented in C, so you'd have to code the recursion in C as well.

So we go with copy-directory? Just DIRNAME and NEWNAME as parameters, or
the other ones from copy-file as well? I guess that we dont't need
OK-IF-ALREADY-EXISTS, because if directory NEWNAME already exists, we
create a subdirectory.

And this could be a Lisp function.

>         Stefan

Best regards, Michael.

Re: Tramp and recursive file operations

[yary]

On Wed, Sep 30, 2009 at 1:06 AM, Michael Albinus <michael.albinus@gmx.de> wrote:
..
> OK. Looking outside dired, there is already delete-directory, good for
> empty directories. Maybe we add an optional parameter RECURSIVE?
>
> A copy-directory function does not exist. What about allowing the first
> parameter of copy-file to be a directory name? This would imply to copy
> recursively.
>
> Comments?

I like the idea of adding an optional parameter to delete-directory.

Is there any existing code that may be using the error-check in
copy-file to ensure that the first arg is not a directory? I like the
idea of making the existing copy-file smart enough to "do the right
thing" when the first argument is a dir, but it could need an optional
RECURSIVE parameter if and only if other callers rely on it rejecting
directories.

-y

Re: Tramp and recursive file operations

[Stefan Monnier]

>>> OK. Looking outside dired, there is already delete-directory, good for
>>> empty directories. Maybe we add an optional parameter RECURSIVE?
>> Sounds OK, except for the fact that delete-directory is implemented in
>> C, so you'd have to code the recursion in C as well.
> I've seen that. I'll do my best ...

Maybe the best option is to rename delete-directory to
delete-directory-internal and then implement a new delete-directory in
files.el. using it.

>>> A copy-directory function does not exist. What about allowing the first
>>> parameter of copy-file to be a directory name? This would imply to copy
>>> recursively.
>> If we were starting over from scratch, that would be fine, but I'm
>> afraid that some code somewhere relies on the fact that copy-file only
>> copies files and not directories.  And of course, copy-file is also
>> implemented in C, so you'd have to code the recursion in C as well.
> So we go with copy-directory? Just DIRNAME and NEWNAME as parameters, or
> the other ones from copy-file as well?

Whenever I copy a directory, I always use "cp -a", so I'd be tempted to
say that additional args aren't needed, but that the behavior should
preserve as much as we can (symlinks, gid, mtime, you name it).

> I guess that we dont't need OK-IF-ALREADY-EXISTS, because if directory
> NEWNAME already exists, we create a subdirectory.

I'd tend to disagree here: this kind of behavior is very handy
interactively, but not when called from Lisp code, where you might
prefer an error (at which point you can then choose to call
copy-directory again to the subdirectory, or you may prefer to overlay
the copy on top of the existing tree, tho we don't have any code that
does it for us yet).


        Stefan

Re: Tramp and recursive file operations

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Maybe the best option is to rename delete-directory to
> delete-directory-internal and then implement a new delete-directory in
> files.el. using it.

Done.

>> So we go with copy-directory? Just DIRNAME and NEWNAME as parameters, or
>> the other ones from copy-file as well?
>
> Whenever I copy a directory, I always use "cp -a", so I'd be tempted to
> say that additional args aren't needed, but that the behavior should
> preserve as much as we can (symlinks, gid, mtime, you name it).
>
>> I guess that we dont't need OK-IF-ALREADY-EXISTS, because if directory
>> NEWNAME already exists, we create a subdirectory.
>
> I'd tend to disagree here: this kind of behavior is very handy
> interactively, but not when called from Lisp code, where you might
> prefer an error (at which point you can then choose to call
> copy-directory again to the subdirectory, or you may prefer to overlay
> the copy on top of the existing tree, tho we don't have any code that
> does it for us yet).

OK, I'll add it next days.

>         Stefan

Best regards, Michael.

Re: Tramp and recursive file operations

[Michael Albinus]

yary <not.com@gmail.com> writes:

>> A copy-directory function does not exist. What about allowing the first
>> parameter of copy-file to be a directory name? This would imply to copy
>> recursively.
>>
>> Comments?
>
> Is there any existing code that may be using the error-check in
> copy-file to ensure that the first arg is not a directory? I like the
> idea of making the existing copy-file smart enough to "do the right
> thing" when the first argument is a dir, but it could need an optional
> RECURSIVE parameter if and only if other callers rely on it rejecting
> directories.

We don't know what people are doing outside the Emacs core packages with
copy-file. So I cannot answer your question.

Adding a RECURSIVE parameter might be an option; OTOH copy-file has
already a long list, and it would be inconvenient to count the number of
nil until you can say t for RECURSIVE. That's why I prefer to introduce
copy-directory.

> -y

Best regards, Michael.