Elisp: Search a File Without Visiting?

Elisp: Search a File Without Visiting?

[taashlo]

I've looked throught the elisp manual without finding the answer to
my question.  I've written some elisp so that when I hit a key
sequence (C-ct) a thesaurus lookup of the word at point (or the
phrase with the mark) is performed with the results of the search
displayed in a separate window (for further searching or dictionary
lookup).

This currently works, but it relies on loading the entire thesaurus
file into a buffer to search for the word.  The problem is that the
thesaurus file (mthesaur.txt[1]) is about 24MiB.

My question: is it possible (using elisp) to search the contents of
the thesaurus file *without* loading it into a buffer?

Thanks,
Tad

[1] ftp://ibiblio.org/pub/docs/books/gutenberg/etext02/mthes10.zip

Re: Elisp: Search a File Without Visiting?

[Sam Halliday]

taashlo AT sandia DOT gov wrote:
> I've written some elisp so that when I hit a key
> sequence (C-ct) a thesaurus lookup of the word
<snip>
> [1] ftp://ibiblio.org/pub/docs/books/gutenberg/etext02/mthes10.zip

this is not a direct answer to your question; but are you aware of the
aiksaurus project? [http://aiksaurus.sourceforge.net] which is a GPL'ed
command line and GUI frontend to the gutenberg thesaurus which you have
referenced (they distribute a much smaller binary data file with their
package). if you did not know of aiksaurus' existence, then maybe its
funtionality can aid your elisp.

i am not aware of any elisp wrapper to that program, but if you intent
to write one, i would most certainly be interrested in using it!! :-D

Re: Elisp: Search a File Without Visiting?

[Kai Großjohann]

taashlo@sandia.gov writes:

> This currently works, but it relies on loading the entire thesaurus
> file into a buffer to search for the word.  The problem is that the
> thesaurus file (mthesaur.txt[1]) is about 24MiB.
>
> My question: is it possible (using elisp) to search the contents of
> the thesaurus file *without* loading it into a buffer?

I can think of two ways to do it.  The first way is to write a
program which does the searching (or use grep) and to have Emacs
invoke that.  The second way is to repeatedly call
insert-file-contents with the right BEG and END arguments to insert
parts of the file into a buffer.

While using an external program makes sense, the other approach is
not useful IMHO.  Emacs is really well optimized for working with
buffers, so why not go with the flow and do it the way it's easiest
in Emacs, why not do it the Emacs way?

-- 
This line is not blank.

Re: Elisp: Search a File Without Visiting?

[Stefan Monnier]

> My question: is it possible (using elisp) to search the contents of
> the thesaurus file *without* loading it into a buffer?

Nope.  How would it work ?
I guess what you're thinking about is not "without loading it" but
"without loading it all at once".  In which case, yes, you can use
the BEG and END args of insert-file-contents to load only a part
of the file, so you can search the file 1MB at a time (for instance).

But maybe you'll be better off using something like grep on
your thesaurus and then fetching the relevant part.


        Stefan

Re: Elisp: Search a File Without Visiting?

[taashlo]

kai.grossjohann@gmx.net (Kai Großjohann) writes:
> taashlo@sandia.gov writes:
> > This currently works, but it relies on loading the entire thesaurus
> > file into a buffer to search for the word.  The problem is that the
> > thesaurus file (mthesaur.txt[1]) is about 24MiB.
> >
> > My question: is it possible (using elisp) to search the contents of
> > the thesaurus file *without* loading it into a buffer?
> 
> I can think of two ways to do it.  The first way is to write a
> program which does the searching (or use grep) and to have Emacs
> invoke that.  The second way is to repeatedly call
> insert-file-contents with the right BEG and END arguments to insert
> parts of the file into a buffer.
> 
> While using an external program makes sense, the other approach is
> not useful IMHO.  Emacs is really well optimized for working with
> buffers, so why not go with the flow and do it the way it's easiest
> in Emacs, why not do it the Emacs way?
> 
> -- 
> This line is not blank.

Thank you Kai!  This is exactly the kind of idea I needed.  I think
that I can use grep to do the searching and return the results for my
elisp to massage for displaying.  My initial tests with grep show
that it will not only use *much* less memory, but it is also *much*
faster.

Thanks again,
Tad

Re: Elisp: Search a File Without Visiting?

[taashlo]

Sam Halliday <devnull@example.com> writes:
> taashlo AT sandia DOT gov wrote:
> > I've written some elisp so that when I hit a key
> > sequence (C-ct) a thesaurus lookup of the word
> <snip>
> > [1] ftp://ibiblio.org/pub/docs/books/gutenberg/etext02/mthes10.zip
> 
> this is not a direct answer to your question; but are you aware of the
> aiksaurus project? [http://aiksaurus.sourceforge.net] which is a GPL'ed
> command line and GUI frontend to the gutenberg thesaurus which you have
> referenced (they distribute a much smaller binary data file with their
> package). if you did not know of aiksaurus' existence, then maybe its
> funtionality can aid your elisp.
> 
> i am not aware of any elisp wrapper to that program, but if you intent
> to write one, i would most certainly be interrested in using it!! :-D

Sam, I will probably look in to this in the future, but for now I'm
going to try using Kai's idea of using grep to do the search.

Thanks,
Tad

Re: Elisp: Search a File Without Visiting?

[Eric Hanchrow]

>>>>> "Tad" == taashlo  <taashlo@sandia.gov> writes:

    Tad> My question: is it possible (using elisp) to search the
    Tad> contents of the thesaurus file *without* loading it into a
    Tad> buffer?

You could use, or duplicate the code of, the `grep' command.  That
simply runs `grep' in a subprocesses and captures the output in a
buffer.
 
-- 
In the practice of computing, where we have so much latitude for
making a mess of it, mathematical elegance is not a dispensable
luxury, but a matter of life and death.

        -- Edsger W. Dijkstra: My Hopes of Computing Science (EWD  709)
        http://www.cs.utexas.edu/users/EWD/ewd07xx/EWD709.PDF

Re: Elisp: Search a File Without Visiting?

[taashlo]

"Stefan Monnier" <monnier+gnu.emacs.help/news/@flint.cs.yale.edu> writes:
> > My question: is it possible (using elisp) to search the contents of
> > the thesaurus file *without* loading it into a buffer?
> 
> Nope.  How would it work ?
> I guess what you're thinking about is not "without loading it" but
> "without loading it all at once".  In which case, yes, you can use
> the BEG and END args of insert-file-contents to load only a part
> of the file, so you can search the file 1MB at a time (for instance).
> 
> But maybe you'll be better off using something like grep on
> your thesaurus and then fetching the relevant part.
> 
>         Stefan

Yeah, Kai Großjohann posted a similar response.  I was too focused on
my original solution to see the right way of getting this problem
solved.

Anyway, I've got the code working as I like:

1) It can pickup the word or phrase to search for from the cursor or
   the mark.

2) It can prompt you to input the word or phrase to search for.

3) It keeps a history of what you searched for.

4) It will preferentially find the word/phrase in the root position.

5) If not found as a root, it will seach for the word/phrase in the list
   of synonyms and related words.

6) With the results of a previous thesaurus search, you can perform a
   subsequent thesaurus search, a dictionary lookup (see
   dictionary-1.8.4.tar.gz), copy, cut & paste, etc.

There's just one minor little nit that is bugging me.  After
formatting the "*Thesaurus Search Results*" buffer, I call
forward-line to position the cursor on a blank line.  This works fine
if the "Results" window is buried or non-existant, but if it is still
visible from a previous search, the cursor is always ends up located
at the beginning of the buffer after the call to display-buffer.

If you would like, I will post the code (to gnu.emacs.sources), or I
can try to make a small example that demonstrates the problem.
(I'm using Gnu Emacs 21.3.1 on Windows NT 4.0 SP 6.)  But before I do
that, I want to check that this isn't a known problem.

Thanks,
Tad Ashlock

Re: Elisp: Search a File Without Visiting?

[Stefan Monnier]

> There's just one minor little nit that is bugging me.  After
> formatting the "*Thesaurus Search Results*" buffer, I call
> forward-line to position the cursor on a blank line.  This works fine
> if the "Results" window is buried or non-existant, but if it is still
> visible from a previous search, the cursor is always ends up located
> at the beginning of the buffer after the call to display-buffer.

Think about what happens when a buffer is shown in two windows:
clearly, there must be more than one `point' per buffer.

So you want to select the window that displays the buffer, and
*then* do forward-line, so that you end up moving the `point' that
you care about, rather than some other `point' of the same buffer.


        Stefan

Re: Elisp: Search a File Without Visiting?

[Tad Ashlock]

"Stefan Monnier" <monnier+gnu.emacs.help/news/@flint.cs.yale.edu> wrote in message news:<5lel1sbm90.fsf@rum.cs.yale.edu>...
> > There's just one minor little nit that is bugging me.  After
> > formatting the "*Thesaurus Search Results*" buffer, I call
> > forward-line to position the cursor on a blank line.  This works fine
> > if the "Results" window is buried or non-existant, but if it is still
> > visible from a previous search, the cursor is always ends up located
> > at the beginning of the buffer after the call to display-buffer.
> 
> Think about what happens when a buffer is shown in two windows:
> clearly, there must be more than one `point' per buffer.
> 
> So you want to select the window that displays the buffer, and
> *then* do forward-line, so that you end up moving the `point' that
> you care about, rather than some other `point' of the same buffer.
> 
>         Stefan

Thanks again Stefan!  That was it.  I'm now satisfied with mthesaur.el
and have posted it to gnu.emacs.sources.  Thanks for all of the help. 
Let me know if you have any ideas for improvements.

Tad