Re: Indexed search with grep-like output

[Lennart Borgman <lennart.borgman@gmail.com>]

On Sun, Jan 2, 2011 at 2:31 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Lennart Borgman <lennart.borgman@gmail.com>
>> Date: Sun, 2 Jan 2011 12:40:10 +0100
>> Cc: emacs-devel@gnu.org
>>
>> On Sun, Jan 2, 2011 at 12:18 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: Lennart Borgman <lennart.borgman@gmail.com>
>> >> Date: Sun, 2 Jan 2011 05:12:32 +0100
>> >> Cc: emacs-devel@gnu.org
>> >>
>> >> I do not know about other search engines, but at least Windows Desktop
>> >> Search allows you to restrict the search later.
>> >
>> > Because it's dog slow.
>>
>> Is it? Can you give an example?
>
> `lid' takes 0.1sec to return results of a query of a DB that indexes
> 12,800 source files of a large C++ software system.  What's your
> timing of a comparable query with Windows Desktop?

Around 2 seconds.

But that is from within Emacs on a very slow computer. (And less than
8,000 files.)


>> I read that people have had good experience with Lucene and large
>> amounts of files. Have you tested Lucene?
>
> Yes, I use one of the tools that builds on Lucene (see
> http://www.methods.co.nz/docindexer/) to index MS Office documents
> (some 17,000 of them) I have on my office machine.  It is also very
> fast: just a few seconds to return a query.

Is that from within Emacs or?


>> > `lid' is very fast, so if you don't want to
>> > index each tree separately, you can filter the output (which includes
>> > the full absolute file name) in Emacs, in the process filter that
>> > accepts `lid's output.
>>
>> Is not that slow since it requires a lot process switching?
>
> Not slower than grep or compilation -- if you want the results to be
> available to Emacs, and don't want to use synchronous subprocesses (so
> that you could continue working while it runs, and perhaps even lookup
> the first bunch of hits), you will always use this method.  (I don't
> understand what you mean by "a lot of process switching" -- there are
> two processes running in parallel.)

The subprocess must be the current process when it sends the output to
Emacs and Emacs must be the current process when it receives the
output. So the operating system has to switch between them and as I
understands it that operation is rather costly on w32 especially. (But
I do not remember any comparable timings any longer.)