igrep on remote files with tramp?

igrep on remote files with tramp?

[Daniel Berdine]

I need to work with files on a remote server, and tramp so far has 
seemed the best way to get the job done. However, I would very much like 
to be able to use the igrep functions, in particular dired-do-igrep, in 
this context. Unfortunately, igrep doesn't seem to know how to use 
tramp. Looking at igrep.el (2.112), there appears to be at least some 
support of ange-ftp built in. Does there exist a version of igrep.el 
that knows about tramp as well as/instead of ange? Or, does anyone know 
of any other way to make it work? My Google searches so far turn up 
zilch.

Thanks in advance!
-Dan

Re: igrep on remote files with tramp?

[Daniel Berdine]

In article <rothaar-CADE09.13380328072005@news.ispnetbilling.com>,
 Daniel Berdine <rothaar@gmail.com> wrote:

> I need to work with files on a remote server, and tramp so far has 
> seemed the best way to get the job done. However, I would very much like 
> to be able to use the igrep functions, in particular dired-do-igrep, in 
> this context. Unfortunately, igrep doesn't seem to know how to use 
> tramp. Looking at igrep.el (2.112), there appears to be at least some 
> support of ange-ftp built in. Does there exist a version of igrep.el 
> that knows about tramp as well as/instead of ange? Or, does anyone know 
> of any other way to make it work? My Google searches so far turn up 
> zilch.

Oops, left out the output, which follows.

-*- mode: igrep; default-directory: 
"/ssh:berdine@fermi.pas.rochester.edu:/home/berdine/MADGRAPH/Code.testing
/" -*-
grep -n  -e alpha_s coupl.inc /dev/null
grep: coupl.inc: No such file or directory

Igrep exited abnormally with code 2 at Thu Jul 28 13:50:47

Re: igrep on remote files with tramp?

[Kevin Rodgers]

Daniel Berdine wrote:
 > I need to work with files on a remote server, and tramp so far has
 > seemed the best way to get the job done. However, I would very much like
 > to be able to use the igrep functions, in particular dired-do-igrep, in
 > this context. Unfortunately, igrep doesn't seem to know how to use
 > tramp. Looking at igrep.el (2.112), there appears to be at least some
 > support of ange-ftp built in. Does there exist a version of igrep.el
 > that knows about tramp as well as/instead of ange? Or, does anyone know
 > of any other way to make it work? My Google searches so far turn up
 > zilch.

igrep only supports ange-ftp (and efs) to the extent that the
igrep-visited-files and Buffer-menu-igrep commands explicitly ignore
remote files.

To properly support remote files, igrep would need to invoke the grep
command in a remote shell.  I know there has been discussion about
whether Emacs should implement process handlers, so that tramp could run
programs on a remote host when invoked from a buffer visiting a remote
file.  But I don't know what was decided and implemented (if anything).

Thanks,
-- 
Kevin Rodgers

Re: igrep on remote files with tramp?

[Michael Albinus]

Daniel Berdine <rothaar@gmail.com> writes:

> Or, does anyone know of any other way to make it work?

I haven't worked with igrep yet. But for my own needs, I have trained
Tramp to support `find-grep-dired'. HTH.


> Thanks in advance!
> -Dan

Best regards, Michael.

Re: igrep on remote files with tramp?

[Michael Albinus]

Kevin Rodgers <ihs_4664@yahoo.com> writes:

> To properly support remote files, igrep would need to invoke the grep
> command in a remote shell.  I know there has been discussion about
> whether Emacs should implement process handlers, so that tramp could run
> programs on a remote host when invoked from a buffer visiting a remote
> file.  But I don't know what was decided and implemented (if anything).

Discussion is postponed until GNU Emacs 22 will be released.

> Thanks,

Best regards, Michael.

Re: igrep on remote files with tramp?

[Kevin Rodgers]

Michael Albinus wrote:
 > Kevin Rodgers <ihs_4664@yahoo.com> writes:
 >>To properly support remote files, igrep would need to invoke the grep
 >>command in a remote shell.  I know there has been discussion about
 >>whether Emacs should implement process handlers, so that tramp could run
 >>programs on a remote host when invoked from a buffer visiting a remote
 >>file.  But I don't know what was decided and implemented (if anything).
 >
 > Discussion is postponed until GNU Emacs 22 will be released.

Perhaps some future generation will get to use that feature then. :-)

-- 
Kevin Rodgers

Re: igrep on remote files with tramp?

[Daniel Berdine]

On Thu, 28 Jul 2005 15:28:54 -0400, Kevin Rodgers wrote
(in article <mailman.1998.1122579165.20277.help-gnu-emacs@gnu.org>):

> Daniel Berdine wrote:
>  > I need to work with files on a remote server, and tramp so far has
>  > seemed the best way to get the job done. However, I would very much like
>  > to be able to use the igrep functions, in particular dired-do-igrep, in
>  > this context. Unfortunately, igrep doesn't seem to know how to use
>  > tramp. Looking at igrep.el (2.112), there appears to be at least some
>  > support of ange-ftp built in. Does there exist a version of igrep.el
>  > that knows about tramp as well as/instead of ange? Or, does anyone know
>  > of any other way to make it work? My Google searches so far turn up
>  > zilch.
> 
> igrep only supports ange-ftp (and efs) to the extent that the
> igrep-visited-files and Buffer-menu-igrep commands explicitly ignore
> remote files.
> 
> To properly support remote files, igrep would need to invoke the grep
> command in a remote shell.  I know there has been discussion about
> whether Emacs should implement process handlers, so that tramp could run
> programs on a remote host when invoked from a buffer visiting a remote
> file.  But I don't know what was decided and implemented (if anything).

Ok, thanks.

I'm obviously ignorant here, but why is this so different than, say, 
incremental searches across several remote files with tramp? Can't a local 
igrep be run on the local tmp files that get used for so many other purposes?
In any case, being able to run remote processes with tramp would be great. 
I've frequently wished I could do this with gdb within emacs, for instance.

Thanks again for the information,
-Dan

Re: igrep on remote files with tramp?

[Daniel Berdine]

On Thu, 28 Jul 2005 15:43:30 -0400, Michael Albinus wrote
(in article <mailman.2000.1122579980.20277.help-gnu-emacs@gnu.org>):

> Daniel Berdine <rothaar@gmail.com> writes:
> 
>> Or, does anyone know of any other way to make it work?
> 
> I haven't worked with igrep yet. But for my own needs, I have trained
> Tramp to support `find-grep-dired'. HTH.

find-grep-dired works, but unfortunately what I need is the grep output of 
matching lines within each file. There are generally many many individual 
matches for any given search, and narrowing it down to searching 30 files 
instead of 50 isn't sufficient. The dired-do-igrep (or grep-find tho I can't 
see how to limit the search to marked files as with igrep) results are 
exactly what I need. I haven't been able to find any remote search solution 
that does anything similar. In fact the only remote search that seems to work 
is the incremental dired-do-search, which is great in some instances, but not 
generally sufficient.

Thanks for your help, tho!

Cheers,
-Dan

Re: igrep on remote files with tramp? (~Solved)

[Daniel Berdine]

In case it helps anyone else, I've found an acceptable solution to this 
problem through color-moccur.el 
(http://www.bookshelf.jp/elc/color-moccur.el).

For cases where there are likely to be several searches, it seems the fastest 
method is to first load all the files one wishes to search (C-u 
dired-do-find-marked-files in my case) and then run moccur. For a one-off 
search, or for local searches where read-times aren't so prohibitive, 
dired-do-moccur works nicely.

Other than the necessity of loading all the files, this actually seems 
preferable to an igrep solution, and short of running the search remotely, 
some such delay or memory hit is inevitable.

Thanks for the input, and hopefully this is useful for someone else.

-Dan

Re: igrep on remote files with tramp?

[Kevin Rodgers]

Daniel Berdine wrote:
 > On Thu, 28 Jul 2005 15:28:54 -0400, Kevin Rodgers wrote
 > (in article <mailman.1998.1122579165.20277.help-gnu-emacs@gnu.org>):
 > > igrep only supports ange-ftp (and efs) to the extent that the
 > > igrep-visited-files and Buffer-menu-igrep commands explicitly ignore
 > > remote files.
 > >
 > > To properly support remote files, igrep would need to invoke the grep
 > > command in a remote shell.  I know there has been discussion about
 > > whether Emacs should implement process handlers, so that tramp 
could run
 > > programs on a remote host when invoked from a buffer visiting a remote
 > > file.  But I don't know what was decided and implemented (if anything).
 >
 > Ok, thanks.
 >
 > I'm obviously ignorant here, but why is this so different than, say,
 > incremental searches across several remote files with tramp?

I'm completely ignorant about tramp.  Do you mean incremental search
across several buffers visiting remote files via tramp?  Or does tramp
provide its own incremental search command?

 > Can't a local igrep be run on the local tmp files that get used for so
 > many other purposes?

I don't know how tramp works, but ange-ftp evidently does not leave the
local tmp files on disk once their contents have been inserted into the
buffer.

 > In any case, being able to run remote processes with tramp would be 
great.

It turns out that ange-ftp provides file name handlers for
shell-command, which uses remote-shell-program (e.g.  /bin/remsh) to
execute the command on the other host when visiting a remote file or
directory.  So I would be suprised if tramp does not provide the same
functionality when you use e.g. `! grep foo *.java'.

But igrep and grep are based on compile, which uses start-process
instead of shell-command.  But since tramp provides commands for
compilation of files on remote hosts (see
http://www.gnu.org/software/tramp/#Compilation), perhaps tramp-util.el
just needs something like this (cribbed from tramp-compile):

(defun tramp-grep (command)
   "Run grep COMMAND on remote host."
   (interactive
    (list (read-from-minibuffer "Run grep (like this): "
                                (or (car grep-history) grep-command)
                                nil nil 'grep-history)))
   (setq grep-command command)
   (unless (string-match "[ \t]*&[ \t]*\\'" command)
     (setq command (concat command "&")))
   (save-some-buffers (not compilation-ask-about-save) nil)
   (let ((d default-directory))
     (save-excursion
       (pop-to-buffer (get-buffer-create "*grep*") t)
       (erase-buffer)
       (setq default-directory d)))
   (tramp-handle-shell-command command (get-buffer "*grep*"))
   (pop-to-buffer (get-buffer "*grep*"))
   (compilation-mode)
   (compilation-minor-mode 1)
   (tramp-minor-mode 1))

(tramp-remap-command 'grep 'tramp-grep)

(when (fboundp 'add-menu-button)
   (funcall 'add-menu-button
    '("Tools" "Grep")
    ["Search Files (Grep)..."
     (command-execute (if tramp-minor-mode 'tramp-grep 'grep))
     :active (fboundp 'grep)]))

 > I've frequently wished I could do this with gdb within emacs, for 
instance.

You are not alone.

-- 
Kevin Rodgers

Re: igrep on remote files with tramp?

[Kevin Rodgers]

Kevin Rodgers wrote:
 > Daniel Berdine wrote:
 >  > In any case, being able to run remote processes with tramp would be
 > great.
 >
 > It turns out that ange-ftp provides file name handlers for
 > shell-command, which uses remote-shell-program (e.g.  /bin/remsh) to
 > execute the command on the other host when visiting a remote file or
 > directory.  So I would be suprised if tramp does not provide the same
 > functionality when you use e.g. `! grep foo *.java'.

Of course, I meant `M-!'.

-- 
Kevin Rodgers

Re: igrep on remote files with tramp?

[Michael Albinus]

Kevin Rodgers <ihs_4664@yahoo.com> writes:

>  > Can't a local igrep be run on the local tmp files that get used for so
>  > many other purposes?
>
> I don't know how tramp works, but ange-ftp evidently does not leave the
> local tmp files on disk once their contents have been inserted into the
> buffer.

So does Tramp.

>  > In any case, being able to run remote processes with tramp would be
> great.
>
> It turns out that ange-ftp provides file name handlers for
> shell-command, which uses remote-shell-program (e.g.  /bin/remsh) to
> execute the command on the other host when visiting a remote file or
> directory.  So I would be suprised if tramp does not provide the same
> functionality when you use e.g. `! grep foo *.java'.

So does Tramp.

> But igrep and grep are based on compile, which uses start-process
> instead of shell-command.  But since tramp provides commands for
> compilation of files on remote hosts (see
> http://www.gnu.org/software/tramp/#Compilation), perhaps tramp-util.el
> just needs something like this (cribbed from tramp-compile):

[...]

It could go this way. But as I wrote already, it would be an endless
task to provide such Tramp pendants for all usefull
commands. Therefore it won't be done inside main Tramp development
(except the integration of `compile', which already exists).

Instead of I hope to convince the emacs-devel list to extend
`call-process' and `start-process' supporting something like file
handlers. There was already a step into that direction with
`process-file', which should act like `call-process' on remote
hosts. But this approach is not sufficient I fear.

>  > I've frequently wished I could do this with gdb within emacs, for
> instance.
>
> You are not alone.

That will be the proof of concept (prototype implementation). If it
doesn't work for gdb out of the box, the concept is not worth to be
applied.

Best regards, Michael.