How speedup find-dired?

How speedup find-dired?

[Oleksandr Gavenko]

I use Cygwin/Windows combination.

Default value of 'find-ls-option' is:

   ("-ls" . "-gilsb")

or which in my case:

   ("-exec ls -ld {} \\;" . "-ld")

Invoking ls for every file is too slow (~= 3 file per sec).

I try set

   (setq find-ls-option '("-print" . ""))

but resulted M-x find-dired buffer does not properly parsed by
'find-dired-filter', which uses 'dired-insert-set-properties'
which depend on 'directory-listing-before-filename-regexp'.

So I try different strings to start match this regex, this one match:

   (string-match directory-listing-before-filename-regexp "-rw-rw-rw- 1 
2010-08-10 23:01 fractal-generator.h")

So I set

(setq find-ls-option '("-printf '-rw-rw-rw- %s %AY-%Am-%Ad %AH:%AM 
%p\n'" . ""))

And all work fine and fast. Further I replace %s with constant 0, to
make less attribute reading operation.

Also I think that replacing file access time with constant also make
running faster.

find with -ls option also work, but output contain many garbage data.

My questions why maintain permission/data/size if I only search for
file names?

How make this search platform independent and independent on installed
utils and their versions?

I previously post how make dired platform/utils independent,
based only on Emacs capabilities:

;;; ----------------------------------------------------------------
;;; ls-lisp, dired ls.

;; If non-nil - use 'insert-directory-program', which I dislike.
(setq ls-lisp-use-insert-directory-program nil)
(setq ls-lisp-ignore-case t)
(setq ls-lisp-dirs-first t)
(if (memq system-type '(windows-nt cygwin))
     (setq ls-lisp-verbosity nil)
   (setq  ls-lisp-verbosity '(links uid gid)))
;; Force use 'ls-lisp-format-time-list'.
(setq ls-lisp-use-localized-time-format t)
(setq ls-lisp-format-time-list
       '("%Y-%m-%d %H:%M:%S"
         "%Y-%m-%d %H:%M   "))
(require 'ls-lisp)

Help me with doing this for find-dired!

Also how about platform/utils independent grep-find/rgrep/lgrep?

-- 
Best regards!

Re: How speedup find-dired?

[Eli Zaretskii]

> From: Oleksandr Gavenko <gavenkoa@gmail.com>
> Date: Mon, 17 Jan 2011 23:59:31 +0200
> 
> I use Cygwin/Windows combination.
> 
> Default value of 'find-ls-option' is:
> 
>    ("-ls" . "-gilsb")
> 
> or which in my case:
> 
>    ("-exec ls -ld {} \\;" . "-ld")
> 
> Invoking ls for every file is too slow (~= 3 file per sec).

I lost you right here: doesn't `("-ls" . "-gilsb")' work for you?
That should cause Emacs to invoke "find ... -ls", instead of invoking
a separate `ls' program for each file.

Also, if your `find' and `ls' ports are slow, I would look for faster
ports rather than for "portable" tweaks of Emacs.  Tweaks that seek to
fix platform-specific bugs or misfeatures can never be portable, in my
experience.

> I try set
> 
>    (setq find-ls-option '("-print" . ""))

Did you try

  (setq find-ls-option '("-ls" . "-gilsb"))

?  If you did, what happened?

Re: How speedup find-dired?

[Oleksandr Gavenko]

On 18.01.2011 0:57, Eli Zaretskii wrote:
>> From: Oleksandr Gavenko<gavenkoa@gmail.com>
>> Date: Mon, 17 Jan 2011 23:59:31 +0200
>>
>> I use Cygwin/Windows combination.
>>
>> Default value of 'find-ls-option' is:
>>
>>     ("-ls" . "-gilsb")
>>
>> or which in my case:
>>
>>     ("-exec ls -ld {} \\;" . "-ld")
>>
>> Invoking ls for every file is too slow (~= 3 file per sec).
>
> I lost you right here: doesn't `("-ls" . "-gilsb")' work for you?
> That should cause Emacs to invoke "find ... -ls", instead of invoking
> a separate `ls' program for each file.
>
Thanks in replay, and sorry for a lot of words ((

I wrote that I try "-ls", but see output:

   $ find . -ls
562949953421501    2 -rw-r--r--   1 user     Отсутствует 
  1539 июн 16  2010 ./.emacs
14355223812252264   24 -rw-r--r--   1 user     Отсутствует 
  47724 янв 18 09:46 ./.emacs-my
844424930132155    2 -rw-r--r--   1 user     Отсутствует 
  828 янв 18 09:44 ./.emacs-post
562949953421500    1 -rw-r--r--   1 user     Отсутствует 
  209 янв 17 15:07 ./.emacs-pre

As I wrote there are many useless data, I need only file names!
There so many useless data that file names truncated to next line!

As wrote it is not easy possible to force parse simple find output by 
dired, like
with 'find . -print'.

Now I stop on GNU find:

(setq find-ls-option '("-printf '-rw-rw-rw- 0 %AY-%Am-%Ad %AH:%AM %p\n'" 
. ""))

This format parsed by 'dired-insert-set-properties' and output aligned 
by column.

> Also, if your `find' and `ls' ports are slow, I would look for faster
> ports rather than for "portable" tweaks of Emacs.  Tweaks that seek to
> fix platform-specific bugs or misfeatures can never be portable, in my
> experience.
>
I use Cygwin and think that it provide very good packages!

It is allow me use native Emacs much easy as on Linux.

ls (GNU coreutils) 8.5
find (GNU findutils) 4.5.9

I happy by making dired platform independent as solution based only on 
Emacs primitives.
This also allow me to make dired look and feel common to all platform
and don't care about 'ls' flavor.

I expect Emacs make things like listening files/dirs,
searching for files and for file's content built in. But it is not true.

Why maintain 'directory-listing-before-filename-regexp' like regexp 
(about 1600 chars
length string) and many 'if' statement instead having internal 'find' 
analog?

My point is when I promote Emacs to my colleagues I must say that they need
find/grep/ls. Some of this people don't know what is this at all.

They say that their favorite IDE make this out of the box and
surprised that Emacs can not just handle such simple things.

>> I try set
>>
>>     (setq find-ls-option '("-print" . ""))
>
> Did you try
>
>    (setq find-ls-option '("-ls" . "-gilsb"))
>
> ?  If you did, what happened?
>
See upper.

Re: How speedup find-dired?

[Eli Zaretskii]

> From: Oleksandr Gavenko <gavenko@bifit.com.ua>
> Date: Tue, 18 Jan 2011 12:08:07 +0200
> 
> I wrote that I try "-ls", but see output:
> 
>    $ find . -ls
> 562949953421501    2 -rw-r--r--   1 user     Отсутствует 
>   1539 июн 16  2010 ./.emacs
> 14355223812252264   24 -rw-r--r--   1 user     Отсутствует 

What's wrong with that?

> As I wrote there are many useless data, I need only file names!

You cannot do with file names alone, since Dired needs the "ls" format
shown above, or else it will not work.

> My point is when I promote Emacs to my colleagues I must say that they need
> find/grep/ls. Some of this people don't know what is this at all.
> 
> They say that their favorite IDE make this out of the box and
> surprised that Emacs can not just handle such simple things.

Emacs has all that, of course.  There's ls-lisp.el, find-lisp.el, "M-x
occur", etc.

Re: How speedup find-dired?

[Oleksandr Gavenko]

On 18.01.2011 17:17, Eli Zaretskii wrote:
>> From: Oleksandr Gavenko<gavenko@bifit.com.ua>
>> Date: Tue, 18 Jan 2011 12:08:07 +0200
>>
>> I wrote that I try "-ls", but see output:
>>
>>     $ find . -ls
>> 562949953421501    2 -rw-r--r--   1 user     Отсутствует
>>    1539 июн 16  2010 ./.emacs
>> 14355223812252264   24 -rw-r--r--   1 user     Отсутствует
>
> What's wrong with that?
>
>> As I wrote there are many useless data, I need only file names!
>
> You cannot do with file names alone, since Dired needs the "ls" format
> shown above, or else it will not work.
>
That I say. It is sad that Emacs historically have dired based on ls.

dired is cool and many based on it, like find-dired.

'find-dired' is work some how and no one interesting in improvement:
get just list on file paths and ability press RET to enter to file!

>> My point is when I promote Emacs to my colleagues I must say that they need
>> find/grep/ls. Some of this people don't know what is this at all.
>>
>> They say that their favorite IDE make this out of the box and
>> surprised that Emacs can not just handle such simple things.
>
> Emacs has all that, of course.  There's ls-lisp.el, find-lisp.el, "M-x
> occur", etc.
>
I try experiment to replace find/ls/grep invocation
with lisp analog.

'occur' is thing that I like because it based on pure Emacs!

I think Emacs need pure Emacs  realization for lgrep/rgrep/find-dired.

Sorry if it already exist, please tell me in that case!

RE: How speedup find-dired?

[Drew Adams]

> > As I wrote there are many useless data, I need only file names!
> 
> You cannot do with file names alone, since Dired needs the "ls" format
> shown above,

But you can hide everything except the file names, if you like. When you do want
to see some of the "useless data" (e.g. modification time), just hit a key to
toggle the display.
http://www.emacswiki.org/emacs/DiredDetails

> or else it will not work.

Dired works just fine with the details hidden - no problem.  You can still do
anything you would do otherwise, including marking files and acting on them,
inserting subdirs, omitting files - anything.

Re: How speedup find-dired?

[Le Wang]

[-- Attachment #1: Type: text/plain, Size: 307 bytes --]

On Tue, Jan 18, 2011 at 11:57 PM, Drew Adams <drew.adams@oracle.com> wrote:

> Dired works just fine with the details hidden - no problem.  You can still
> do
> anything you would do otherwise, including marking files and acting on
> them,
> inserting subdirs, omitting files - anything.
>

wdired?

-- 
Le

[-- Attachment #2: Type: text/html, Size: 585 bytes --]

RE: How speedup find-dired?

[Drew Adams]

>> Dired works just fine with the details hidden - no problem.
>> You can still do anything you would do otherwise, including
>> marking files and acting on them, inserting subdirs, omitting
>> files - anything.
>
> wdired? 

Just toggle to show the details before using `wdired'.

(It could be argued that wdired is not Dired. ;) Try doing any of the normal
Dired things in wdired - e.g. `C' to copy, `B' to byte-compile.)