simple useful functions

simple useful functions

[Tak Ota]

If you think the following two functions are universally useful please
consider incorporating them in simple.el or any appropriate package.
If not disregard.

-Tak

(defun collect-string (regexp &optional num)
  "Collect strings of REGEXP (or optional NUM paren) from the
current buffer into a collection buffer."
  (interactive "sCollect string (regexp): \nP")
  (let ((collection-buffer
	 (get-buffer-create (format "*Collection of \"%s\" *" regexp))))
    (with-current-buffer collection-buffer (erase-buffer))
    (save-excursion
      (goto-char (point-min))
      (while (re-search-forward regexp nil t)
	(let ((str (match-string (or num 0))))
	  (if str
	      (with-current-buffer collection-buffer
		(insert str)
		(or (zerop (current-column))
		    (insert "\n")))))))
    (pop-to-buffer collection-buffer)
    (goto-char (point-min))))

(defun source (script &optional shell keep-current-directory)
  "Source the specified shell script.
Source the shell SCRIPT and import the environment into this
emacs.  The optional SHELL specifies the shell other than the
default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
which is the default, the current directory is temporarily
changed to the directory where the script resides while sourcing
the script."
  (interactive "fscript file: ")
  (if (null shell)
      (setq shell shell-file-name))
  (with-temp-buffer
    (unless keep-current-directory
      (setq default-directory (file-name-directory script)))
    (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
    (while (re-search-backward "^\\([^=]+\\)=\\(.*\\)$" nil t)
      (setenv (match-string 1) (match-string 2)))))

simple useful functions

[Stephen J. Turnbull]

Tak Ota writes:
 > If you think the following two functions are universally useful please
 > consider incorporating them in simple.el or any appropriate package.
 > If not disregard.
 > 
 > -Tak
 > 
 > (defun collect-string (regexp &optional num)
 >   "Collect strings of REGEXP (or optional NUM paren) from the
 > current buffer into a collection buffer."

What does this do that M-x occur doesn't do?  Could it be added to
occur or to occur-mode?

 > (defun source (script &optional shell keep-current-directory)
 >   "Source the specified shell script.
 > Source the shell SCRIPT and import the environment into this
 > emacs.  The optional SHELL specifies the shell other than the
 > default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
 > which is the default, the current directory is temporarily
 > changed to the directory where the script resides while sourcing
 > the script."

Probably this should be in the "shell-" namespace.

I can see why the side effect of importing the environment might be
useful, but I think that should be explicit in the name.

 >     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))

Isn't "." rather than "source" the portable idiom here?

Re: simple useful functions

[Tak Ota]

Thanks for the quick feedback.

Thu, 28 Oct 2010 20:39:38 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

> Tak Ota writes:
>  > If you think the following two functions are universally useful please
>  > consider incorporating them in simple.el or any appropriate package.
>  > If not disregard.
>  > 
>  > -Tak
>  > 
>  > (defun collect-string (regexp &optional num)
>  >   "Collect strings of REGEXP (or optional NUM paren) from the
>  > current buffer into a collection buffer."
> 
> What does this do that M-x occur doesn't do?  Could it be added to
> occur or to occur-mode?

I don't think you can perform M-1 M-x collect-string with an input
string of (defun \([^ ]+\)

It's handy to collect function names from source code or make a list
of image URLs from a given HTML file and so on.

Correct me if I'm wrong.  To my understanding occur is like running
grep to the current buffer while collect-string is running a special
case of sed or awk to the current buffer.

>  > (defun source (script &optional shell keep-current-directory)
>  >   "Source the specified shell script.
>  > Source the shell SCRIPT and import the environment into this
>  > emacs.  The optional SHELL specifies the shell other than the
>  > default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
>  > which is the default, the current directory is temporarily
>  > changed to the directory where the script resides while sourcing
>  > the script."
> 
> Probably this should be in the "shell-" namespace.

I agree.  That makes sense.

> I can see why the side effect of importing the environment might be
> useful, but I think that should be explicit in the name.
> 
>  >     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
> 
> Isn't "." rather than "source" the portable idiom here?

You are correct.  "." is better.

-Tak

RE: simple useful functions

[Drew Adams]

> > > (call-process shell nil t nil "-c"
> > >               (concat "source " script "; printenv"))
> > 
> > Isn't "." rather than "source" the portable idiom here?
> 
> You are correct.  "." is better.

`.' doesn't work for csh, does it?
Isn't `source' appropriate for csh?

And `man bash' shows that `source' is supported as well as `.', at least for
that flavor of `sh' (not for original `sh').  In multiple places the `bash' man
page refers to "the . or source builtins" or "the . (source) command".  And `.'
and `source' are listed together, with the same description, under `Shell
Builtin Commands'.

So what makes `.' more portable than `source'?

(But I'm no expert on this.)

Re: simple useful functions

[Andreas Schwab]

"Drew Adams" <drew.adams@oracle.com> writes:

> So what makes `.' more portable than `source'?

It works in any Bourne or POSIX (derived) shell.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

RE: simple useful functions

[Drew Adams]

> > So what makes `.' more portable than `source'?
> 
> It works in any Bourne or POSIX (derived) shell.

Swahili is a universal language ... among people who speak Swahili.

`.' works in the Bourne-shell family.
`source' works in the C-shell family.
`bash' and `zsh' support both.

If you want it to be _portable_, then test the value of argument SHELL and DTRT.

Re: simple useful functions

[Andreas Schwab]

"Drew Adams" <drew.adams@oracle.com> writes:

> `.' works in the Bourne-shell family.
> `source' works in the C-shell family.
> `bash' and `zsh' support both.

Which makes "." more portable.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: simple useful functions

[Chad Brown]

On Oct 29, 2010, at 1:47 PM, Andreas Schwab wrote:

> "Drew Adams" <drew.adams@oracle.com> writes:
> 
>> `.' works in the Bourne-shell family.
>> `source' works in the C-shell family.
>> `bash' and `zsh' support both.
> 
> Which makes "." more portable.

Which seems to be largely pointless, since the goal of the function 
is to interact with a specific script.  

If the function doesn't want to detect the shell (ideally, parsing the 
interpreter from the shebang line, perhaps it'd be better just to
name the function(s) something like `source-sh-script' or 
`source-csh-script'.

*Chad
..who has used login shells that didn't support `printenv', and once
even used emacs as his login shell to see if it would work.

RE: simple useful functions

[Drew Adams]

> If the function doesn't want to detect the shell (ideally, 
> parsing the interpreter from the shebang line...

The Emacs function `source' in question is passed an optional SHELL argument.
If nil, then the value of option `shell-file-name' is used.

That's why I suggested acting conditionally based on this SHELL arg.

RE: simple useful functions

[Stephen J. Turnbull]

Drew Adams writes:

 > So what makes `.' more portable than `source'?

The "P" in "POSIX".

True, that doesn't generalize to csh.  But then, nothing does.  At
that point, you're not talking "a simple function we can just add
somewhere" anymore, you're talking about a whole package.

Re: simple useful functions

[Thierry Volpiatto]

Tak Ota <Takaaki.Ota@am.sony.com> writes:

> Thanks for the quick feedback.
>
> Thu, 28 Oct 2010 20:39:38 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
>
>> Tak Ota writes:
>>  > If you think the following two functions are universally useful please
>>  > consider incorporating them in simple.el or any appropriate package.
>>  > If not disregard.
>>  > 
>>  > -Tak
>>  > 
>>  > (defun collect-string (regexp &optional num)
>>  >   "Collect strings of REGEXP (or optional NUM paren) from the
>>  > current buffer into a collection buffer."
>> 
>> What does this do that M-x occur doesn't do?  Could it be added to
>> occur or to occur-mode?
>
> I don't think you can perform M-1 M-x collect-string with an input
> string of (defun \([^ ]+\)
>
> It's handy to collect function names from source code or make a list
> of image URLs from a given HTML file and so on.
>
> Correct me if I'm wrong.  To my understanding occur is like running
> grep to the current buffer while collect-string is running a special
> case of sed or awk to the current buffer.

Have a look at ioccur.el that does what you want but is incremental, allow
changing search method during session (regexp/litteral) is faster than
occur and many other things.

http://mercurial.intuxication.org/hg/ioccur

>>  > (defun source (script &optional shell keep-current-directory)
>>  >   "Source the specified shell script.
>>  > Source the shell SCRIPT and import the environment into this
>>  > emacs.  The optional SHELL specifies the shell other than the
>>  > default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
>>  > which is the default, the current directory is temporarily
>>  > changed to the directory where the script resides while sourcing
>>  > the script."
>> 
>> Probably this should be in the "shell-" namespace.
>
> I agree.  That makes sense.
>
>> I can see why the side effect of importing the environment might be
>> useful, but I think that should be explicit in the name.
>> 
>>  >     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
>> 
>> Isn't "." rather than "source" the portable idiom here?
>
> You are correct.  "." is better.
>
> -Tak
>
>
>

-- 
A+ Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

Re: simple useful functions

[Tak Ota]

Could you teach me how to perform collect-string equivalent operation
with occur mode?

-Tak

Fri, 29 Oct 2010 11:13:46 -0700: Tak Ota <Takaaki.Ota@am.sony.com> wrote:

> Thanks for the quick feedback.
> 
> Thu, 28 Oct 2010 20:39:38 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
> 
> > Tak Ota writes:
> >  > If you think the following two functions are universally useful please
> >  > consider incorporating them in simple.el or any appropriate package.
> >  > If not disregard.
> >  > 
> >  > -Tak
> >  > 
> >  > (defun collect-string (regexp &optional num)
> >  >   "Collect strings of REGEXP (or optional NUM paren) from the
> >  > current buffer into a collection buffer."
> > 
> > What does this do that M-x occur doesn't do?  Could it be added to
> > occur or to occur-mode?
> 
> I don't think you can perform M-1 M-x collect-string with an input
> string of (defun \([^ ]+\)
> 
> It's handy to collect function names from source code or make a list
> of image URLs from a given HTML file and so on.
> 
> Correct me if I'm wrong.  To my understanding occur is like running
> grep to the current buffer while collect-string is running a special
> case of sed or awk to the current buffer.
> 
> >  > (defun source (script &optional shell keep-current-directory)
> >  >   "Source the specified shell script.
> >  > Source the shell SCRIPT and import the environment into this
> >  > emacs.  The optional SHELL specifies the shell other than the
> >  > default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
> >  > which is the default, the current directory is temporarily
> >  > changed to the directory where the script resides while sourcing
> >  > the script."
> > 
> > Probably this should be in the "shell-" namespace.
> 
> I agree.  That makes sense.
> 
> > I can see why the side effect of importing the environment might be
> > useful, but I think that should be explicit in the name.
> > 
> >  >     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
> > 
> > Isn't "." rather than "source" the portable idiom here?
> 
> You are correct.  "." is better.
> 
> -Tak

Re: simple useful functions

[Stephen J. Turnbull]

Tak Ota writes:
 > Could you teach me how to perform collect-string equivalent operation
 > with occur mode?

I don't know if you can, maybe there's something in Emacs's version of
`occur' that would allow it.  I didn't understand your requirements,
that's why I asked.

Rather than a separate function, I think it would be useful to augment
occur-mode here, by saving the occur regexp (say to occur-mode-regexp)
and adding the following command to occur mode (untested):

(defun occur-mode-matchers-only ()
  "Leave only text that matches the first group in the occur regexp.
If there is no group, use the whole match.
With more than one group, you should use shy groups for the others.
Be careful with repeated groups; only one of them will be left."
  (interactive)
  (save-excursion
    (toggle-read-only -1)
    (goto-char (point-min))
    (save-match-data
      ;; don't look at this code lest you be turned to stone, bletch
      (re-search-forward occur-mode-regexp nil t)
      (let* ((group (if (match-end 1) 1 0))
             (previous-end (progn (goto-char (match-end group))
                                  (point-marker)))
             temp)
        (delete-region (point-min) (match-beginning group))
        (while (re-search-forward occur-mode-regexp nil t)
          (setq temp (progn (goto-char (match-end group)) (point-marker)))
          (delete-region previous-end (match-beginning group))
          (goto-char previous-end)
          (insert ?\n)
          (setq previous-end temp))
        (delete-region previous-end (point-max))
        (insert ?\n)
        (goto-char (point-max))
        (insert ?\n)
        (toggle-read-only 1)))))

Re: simple useful functions

[Tak Ota]

Don't you like the simplicity?  How about simply adding collect-string
to replace.el instead of trying to associate it with occur in a clumsy
way?

-Tak

Mon, 1 Nov 2010 19:22:11 -0700: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp> wrote:

> Tak Ota writes:
>  > Could you teach me how to perform collect-string equivalent operation
>  > with occur mode?
> 
> I don't know if you can, maybe there's something in Emacs's version of
> `occur' that would allow it.  I didn't understand your requirements,
> that's why I asked.
> 
> Rather than a separate function, I think it would be useful to augment
> occur-mode here, by saving the occur regexp (say to occur-mode-regexp)
> and adding the following command to occur mode (untested):
> 
> (defun occur-mode-matchers-only ()
>   "Leave only text that matches the first group in the occur regexp.
> If there is no group, use the whole match.
> With more than one group, you should use shy groups for the others.
> Be careful with repeated groups; only one of them will be left."
>   (interactive)
>   (save-excursion
>     (toggle-read-only -1)
>     (goto-char (point-min))
>     (save-match-data
>       ;; don't look at this code lest you be turned to stone, bletch
>       (re-search-forward occur-mode-regexp nil t)
>       (let* ((group (if (match-end 1) 1 0))
>              (previous-end (progn (goto-char (match-end group))
>                                   (point-marker)))
>              temp)
>         (delete-region (point-min) (match-beginning group))
>         (while (re-search-forward occur-mode-regexp nil t)
>           (setq temp (progn (goto-char (match-end group)) (point-marker)))
>           (delete-region previous-end (match-beginning group))
>           (goto-char previous-end)
>           (insert ?\n)
>           (setq previous-end temp))
>         (delete-region previous-end (point-max))
>         (insert ?\n)
>         (goto-char (point-max))
>         (insert ?\n)
>         (toggle-read-only 1)))))
> 

Re: simple useful functions

[Stephen J. Turnbull]

Tak Ota writes:

 > Don't you like the simplicity?

Proliferating unrelated functions is a complexity of its own.  Occur
already does this kind of thing; that's where I would look for this
functionality.  Sure, you can add the function, but unless you can
document it for other users in a discoverable way, you should add it
to your init file, not to Emacs. ;-)

 > How about simply adding collect-string to replace.el instead of
 > trying to associate it with occur in a clumsy way?

Uh, what "replace" functionality does "collect-string" provide?

Re: simple useful functions

[Andreas Röhler]

Am 03.11.2010 06:27, schrieb Stephen J. Turnbull:
> Tak Ota writes:
>
>   >  Don't you like the simplicity?
>
> Proliferating unrelated functions is a complexity of its own.  Occur
> already does this kind of thing;

Hi Stephen,

AFAIU `occur' selects lines, not strings.

Isn't `collect-string' an instructive example how to write simple and 
effective code in Emacs Lisp?

Andreas


that's where I would look for this
> functionality.  Sure, you can add the function, but unless you can
> document it for other users in a discoverable way, you should add it
> to your init file, not to Emacs. ;-)
>
>   >  How about simply adding collect-string to replace.el instead of
>   >  trying to associate it with occur in a clumsy way?
>
> Uh, what "replace" functionality does "collect-string" provide?
>
>
>

Re: simple useful functions

[Stephen J. Turnbull]

Andreas Röhler writes:

 > AFAIU `occur' selects lines, not strings.

Sure, but the words "occur" and "occurance" don't imply
line-oriented.  IOW, that's an implementation detail AFAICS.

You can argue this functionality doesn't belong in occur, but I don't
think its current line-oriented implementation is a reason not to add
it there.  More to the point would be that `collect-string' might be
most useful in Lisp programs, but `occur' is interactive.  There may
be other reasons, too.  Note that the interactive vs. programmatic
issue might very well be a problem with my clumsy implementation --
that was just a proof of concept.

Even so, I think it belongs in the "occur-" namespace.  I think it's
most discoverable there.

 > Isn't `collect-string' an instructive example how to write simple and 
 > effective code in Emacs Lisp?

"Instructive examples" belong in the Elisp tutorial, not in the core
code.

The questions about `collect-string' should be, "Does this
*functionality* belong in the core codebase?  If so, where?"

Re: simple useful functions

[Tak Ota]

Wed, 3 Nov 2010 03:13:36 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

> Andreas Röhler writes:
> 
>  > AFAIU `occur' selects lines, not strings.
> 
> Sure, but the words "occur" and "occurance" don't imply
> line-oriented.  IOW, that's an implementation detail AFAICS.
> 
> You can argue this functionality doesn't belong in occur, but I don't
> think its current line-oriented implementation is a reason not to add
> it there.  More to the point would be that `collect-string' might be
> most useful in Lisp programs, but `occur' is interactive.  There may
> be other reasons, too.  Note that the interactive vs. programmatic
> issue might very well be a problem with my clumsy implementation --
> that was just a proof of concept.

Try M-x collect-string.  It is interactive.  And recently in regard to
my work I most frequently use collect-string to extract specific URLs
from XML and HTML files.  I rarely use it in Lisp programming.


> Even so, I think it belongs in the "occur-" namespace.  I think it's
> most discoverable there.
> 
>  > Isn't `collect-string' an instructive example how to write simple and 
>  > effective code in Emacs Lisp?
> 
> "Instructive examples" belong in the Elisp tutorial, not in the core
> code.
> 
> The questions about `collect-string' should be, "Does this
> *functionality* belong in the core codebase?  If so, where?"

This logic makes sense.

-Tak

Re: simple useful functions

[Tak Ota]

Tue, 2 Nov 2010 22:27:23 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

> Tak Ota writes:
> 
>  > Don't you like the simplicity?
> 
> Proliferating unrelated functions is a complexity of its own.

I agree in general.

> Occur
> already does this kind of thing; that's where I would look for this
> functionality.  Sure, you can add the function, but unless you can
> document it for other users in a discoverable way, you should add it
> to your init file, not to Emacs. ;-)

How about rename the command as collect-occur instead of
collect-string and document it as next?

`M-x collect-occur'
     Prompt for a regexp, and collect the matching string into a
     collection buffer.  A numeric argument N specifies to extract the
     Nth recorded matched substring.

Does this address your concern in simpler fashion?

>  > How about simply adding collect-string to replace.el instead of
>  > trying to associate it with occur in a clumsy way?
> 
> Uh, what "replace" functionality does "collect-string" provide?

I followed your logic as you insist associating this command to occur
which is defined in replace.el.

-Tak

Re: simple useful functions

[Stephen J. Turnbull]

Tak Ota writes:
 > Tue, 2 Nov 2010 22:27:23 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:
 > 
 > How about rename the command as collect-occur instead of
 > collect-string and document it as next?

Wrong namespace.  IMO, the name should *start* with occur (but Stefan
and Yidong are final authorities on that).

 > `M-x collect-occur'

 > Does this address your concern in simpler fashion?

Not really. :-)  I *would* like this feature added to occur-mode, but
if you can get it past the maintainers as a standalone command, more
power to you.  I think it is a useful function, but it would be more
useful (and used) with occur integration.

 > > Uh, what "replace" functionality does "collect-string" provide?
 > 
 > I followed your logic as you insist associating this command to occur
 > which is defined in replace.el.

Ah.  "Irreconcilable technical differences" raise their heads once in
again.  In *my* (X)Emacs, occur is defined in occur.el, and that's the
way it should be. :-)

Re: simple useful functions

[Tak Ota]

Wed, 3 Nov 2010 19:10:28 -0700: "Stephen J. Turnbull" <stephen@xemacs.org> wrote:

>  > > Uh, what "replace" functionality does "collect-string" provide?
>  > 
>  > I followed your logic as you insist associating this command to occur
>  > which is defined in replace.el.
> 
> Ah.  "Irreconcilable technical differences" raise their heads once in
> again.  In *my* (X)Emacs, occur is defined in occur.el, and that's the
> way it should be. :-)

Ah.  That explains the mutual misunderstanding.  I was wondering why.

-Tak

collect-string (was: simple useful functions)

[Stefan Monnier]

>> How about rename the command as collect-occur instead of
>> collect-string and document it as next?
> Wrong namespace.  IMO, the name should *start* with occur (but Stefan
> and Yidong are final authorities on that).

Agreed, namespace cleanliness is one of my favorite forms of
anal retentiveness.

The way I see it, the suggested collect-string is a variant of occur
where the result buffer contains none of the regexp matches's context.
So it would make sense to integrate it very tightly with `occur',
i.e. make M-x occur do the job of collect-strings for some particular
value of its argument NLINES.  Currently, NLINES is assumed to be
a number and all values of that number have a useful meaning, so we'd
have to add this new feature via a non-number value of NLINES.

E.g. C-u M-x occur could do the collect-string thingy (which is still an
incompatible change since some people may like to use C-u M-x occur to get
4 lines of context, but you can make omelets without breaking eggs).


        Stefan

Re: collect-string (was: simple useful functions)

[Tak Ota]

Thu, 4 Nov 2010 06:58:20 -0700: Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> >> How about rename the command as collect-occur instead of
> >> collect-string and document it as next?
> > Wrong namespace.  IMO, the name should *start* with occur (but Stefan
> > and Yidong are final authorities on that).
> 
> Agreed, namespace cleanliness is one of my favorite forms of
> anal retentiveness.
> 
> The way I see it, the suggested collect-string is a variant of occur
> where the result buffer contains none of the regexp matches's context.
> So it would make sense to integrate it very tightly with `occur',
> i.e. make M-x occur do the job of collect-strings for some particular
> value of its argument NLINES.  Currently, NLINES is assumed to be
> a number and all values of that number have a useful meaning, so we'd
> have to add this new feature via a non-number value of NLINES.
> 
> E.g. C-u M-x occur could do the collect-string thingy (which is still an
> incompatible change since some people may like to use C-u M-x occur to get
> 4 lines of context, but you can make omelets without breaking eggs).
> 
> 
>         Stefan
> 

Now I am convinced.  How about the change below?  In conventional
occur zero or negative value for nlines is meaningless correct?  We
can use that for collection purpose.  i.e. C-u 0 M-x occur does the
collection of the matching pattern.  C-u -1 M-x occur performs the
collection of the recorded pattern 1.

-Tak

(defun occur-1 (regexp nlines bufs &optional buf-name)
  (unless (and regexp (not (equal regexp "")))
    (error "Occur doesn't work with the empty regexp"))
  (unless buf-name
    (setq buf-name "*Occur*"))
  (let (occur-buf
	(active-bufs (delq nil (mapcar #'(lambda (buf)
					   (when (buffer-live-p buf) buf))
				       bufs))))
    ;; Handle the case where one of the buffers we're searching is the
    ;; output buffer.  Just rename it.
    (when (member buf-name (mapcar 'buffer-name active-bufs))
      (with-current-buffer (get-buffer buf-name)
	(rename-uniquely)))

    ;; Now find or create the output buffer.
    ;; If we just renamed that buffer, we will make a new one here.
    (setq occur-buf (get-buffer-create buf-name))

    (if (or (null (integerp nlines))
	    (> nlines 0))
	;; nlines is not zero or negative so perform nomal occur
	(with-current-buffer occur-buf
	  (occur-mode)
	  (let ((inhibit-read-only t)
		;; Don't generate undo entries for creation of the initial contents.
		(buffer-undo-list t))
	    (erase-buffer)
	    (let ((count (occur-engine
			  regexp active-bufs occur-buf
			  (or nlines list-matching-lines-default-context-lines)
			  (if (and case-fold-search search-upper-case)
			      (isearch-no-upper-case-p regexp t)
			    case-fold-search)
			  list-matching-lines-buffer-name-face
			  nil list-matching-lines-face
			  (not (eq occur-excluded-properties t)))))
	      (let* ((bufcount (length active-bufs))
		     (diff (- (length bufs) bufcount)))
		(message "Searched %d buffer%s%s; %s match%s for `%s'"
			 bufcount (if (= bufcount 1) "" "s")
			 (if (zerop diff) "" (format " (%d killed)" diff))
			 (if (zerop count) "no" (format "%d" count))
			 (if (= count 1) "" "es")
			 regexp))
	      (setq occur-revert-arguments (list regexp nlines bufs))
	      (if (= count 0)
		  (kill-buffer occur-buf)
		(display-buffer occur-buf)
		(setq next-error-last-buffer occur-buf)
		(setq buffer-read-only t)
		(set-buffer-modified-p nil)
		(run-hooks 'occur-hook)))))
      ;; nlines is zero or negative integer perform collect-string
      (with-current-buffer occur-buf
	(setq nlines (- nlines))
	(fundamental-mode)
	(let ((inhibit-read-only t)
	      (buffer-undo-list t))
	  (erase-buffer)
	  (while active-bufs
	    (with-current-buffer (car active-bufs)
	      (save-excursion
		(goto-char (point-min))
		(while (re-search-forward regexp nil t)
		  (let ((str (match-string nlines)))
		    (if str
			(with-current-buffer occur-buf
			  (insert str)
			  (or (zerop (current-column))
			      (insert "\n"))))))))
	    (setq active-bufs (cdr active-bufs))))
	(display-buffer occur-buf)))))

Re: collect-string (was: simple useful functions)

[Tak Ota]

The documentation can be something like this.  I just added the last
paragraph below.

-Tak

`M-x occur'
     Prompt for a regexp, and display a list showing each line in the
     buffer that contains a match for it.  To limit the search to part
     of the buffer, narrow to that part (*note Narrowing::).  A numeric
     argument N specifies that N lines of context are to be displayed
     before and after each matching line.  Currently, `occur' can not
     correctly handle multiline matches.

     The buffer `*Occur*' containing the output serves as a menu for
     finding the occurrences in their original context.  Click
     `Mouse-2' on an occurrence listed in `*Occur*', or position point
     there and type <RET>; this switches to the buffer that was
     searched and moves point to the original of the chosen occurrence.
     `o' and `C-o' display the match in another window; `C-o' does not
     select it.

     After using `M-x occur', you can use `next-error' to visit the
     occurrences found, one by one.  *note Compilation Mode::.

     When the numeric argument N is 0 or negative the buffer `*Occur*'
     collects all the matched strings.  When N is 0 the entire text
     matched is collected.  When N is negative the text in the -Nth
     parenthesized expression in the regexp is collected.


Thu, 04 Nov 2010 11:36:54 -0700: Tak Ota <Takaaki.Ota@am.sony.com> wrote:

> Thu, 4 Nov 2010 06:58:20 -0700: Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> > >> How about rename the command as collect-occur instead of
> > >> collect-string and document it as next?
> > > Wrong namespace.  IMO, the name should *start* with occur (but Stefan
> > > and Yidong are final authorities on that).
> > 
> > Agreed, namespace cleanliness is one of my favorite forms of
> > anal retentiveness.
> > 
> > The way I see it, the suggested collect-string is a variant of occur
> > where the result buffer contains none of the regexp matches's context.
> > So it would make sense to integrate it very tightly with `occur',
> > i.e. make M-x occur do the job of collect-strings for some particular
> > value of its argument NLINES.  Currently, NLINES is assumed to be
> > a number and all values of that number have a useful meaning, so we'd
> > have to add this new feature via a non-number value of NLINES.
> > 
> > E.g. C-u M-x occur could do the collect-string thingy (which is still an
> > incompatible change since some people may like to use C-u M-x occur to get
> > 4 lines of context, but you can make omelets without breaking eggs).
> > 
> > 
> >         Stefan
> > 
> 
> Now I am convinced.  How about the change below?  In conventional
> occur zero or negative value for nlines is meaningless correct?  We
> can use that for collection purpose.  i.e. C-u 0 M-x occur does the
> collection of the matching pattern.  C-u -1 M-x occur performs the
> collection of the recorded pattern 1.
> 
> -Tak
> 
> (defun occur-1 (regexp nlines bufs &optional buf-name)
>   (unless (and regexp (not (equal regexp "")))
>     (error "Occur doesn't work with the empty regexp"))
>   (unless buf-name
>     (setq buf-name "*Occur*"))
>   (let (occur-buf
> 	(active-bufs (delq nil (mapcar #'(lambda (buf)
> 					   (when (buffer-live-p buf) buf))
> 				       bufs))))
>     ;; Handle the case where one of the buffers we're searching is the
>     ;; output buffer.  Just rename it.
>     (when (member buf-name (mapcar 'buffer-name active-bufs))
>       (with-current-buffer (get-buffer buf-name)
> 	(rename-uniquely)))
> 
>     ;; Now find or create the output buffer.
>     ;; If we just renamed that buffer, we will make a new one here.
>     (setq occur-buf (get-buffer-create buf-name))
> 
>     (if (or (null (integerp nlines))
> 	    (> nlines 0))
> 	;; nlines is not zero or negative so perform nomal occur
> 	(with-current-buffer occur-buf
> 	  (occur-mode)
> 	  (let ((inhibit-read-only t)
> 		;; Don't generate undo entries for creation of the initial contents.
> 		(buffer-undo-list t))
> 	    (erase-buffer)
> 	    (let ((count (occur-engine
> 			  regexp active-bufs occur-buf
> 			  (or nlines list-matching-lines-default-context-lines)
> 			  (if (and case-fold-search search-upper-case)
> 			      (isearch-no-upper-case-p regexp t)
> 			    case-fold-search)
> 			  list-matching-lines-buffer-name-face
> 			  nil list-matching-lines-face
> 			  (not (eq occur-excluded-properties t)))))
> 	      (let* ((bufcount (length active-bufs))
> 		     (diff (- (length bufs) bufcount)))
> 		(message "Searched %d buffer%s%s; %s match%s for `%s'"
> 			 bufcount (if (= bufcount 1) "" "s")
> 			 (if (zerop diff) "" (format " (%d killed)" diff))
> 			 (if (zerop count) "no" (format "%d" count))
> 			 (if (= count 1) "" "es")
> 			 regexp))
> 	      (setq occur-revert-arguments (list regexp nlines bufs))
> 	      (if (= count 0)
> 		  (kill-buffer occur-buf)
> 		(display-buffer occur-buf)
> 		(setq next-error-last-buffer occur-buf)
> 		(setq buffer-read-only t)
> 		(set-buffer-modified-p nil)
> 		(run-hooks 'occur-hook)))))
>       ;; nlines is zero or negative integer perform collect-string
>       (with-current-buffer occur-buf
> 	(setq nlines (- nlines))
> 	(fundamental-mode)
> 	(let ((inhibit-read-only t)
> 	      (buffer-undo-list t))
> 	  (erase-buffer)
> 	  (while active-bufs
> 	    (with-current-buffer (car active-bufs)
> 	      (save-excursion
> 		(goto-char (point-min))
> 		(while (re-search-forward regexp nil t)
> 		  (let ((str (match-string nlines)))
> 		    (if str
> 			(with-current-buffer occur-buf
> 			  (insert str)
> 			  (or (zerop (current-column))
> 			      (insert "\n"))))))))
> 	    (setq active-bufs (cdr active-bufs))))
> 	(display-buffer occur-buf)))))

Re: collect-string (was: simple useful functions)

[Tak Ota]

And here is the change log update.

2010-11-04  Takaaki Ota  <Takaaki.Ota@am.sony.com>

	* replace.el (occur-1): Add string collection feature when NLINES
	is zero or negative integer value.

-Tak

Thu, 04 Nov 2010 13:18:20 -0700: Tak Ota <Takaaki.Ota@am.sony.com> wrote:

> The documentation can be something like this.  I just added the last
> paragraph below.
> 
> -Tak
> 
> `M-x occur'
>      Prompt for a regexp, and display a list showing each line in the
>      buffer that contains a match for it.  To limit the search to part
>      of the buffer, narrow to that part (*note Narrowing::).  A numeric
>      argument N specifies that N lines of context are to be displayed
>      before and after each matching line.  Currently, `occur' can not
>      correctly handle multiline matches.
> 
>      The buffer `*Occur*' containing the output serves as a menu for
>      finding the occurrences in their original context.  Click
>      `Mouse-2' on an occurrence listed in `*Occur*', or position point
>      there and type <RET>; this switches to the buffer that was
>      searched and moves point to the original of the chosen occurrence.
>      `o' and `C-o' display the match in another window; `C-o' does not
>      select it.
> 
>      After using `M-x occur', you can use `next-error' to visit the
>      occurrences found, one by one.  *note Compilation Mode::.
> 
>      When the numeric argument N is 0 or negative the buffer `*Occur*'
>      collects all the matched strings.  When N is 0 the entire text
>      matched is collected.  When N is negative the text in the -Nth
>      parenthesized expression in the regexp is collected.
> 
> 
> Thu, 04 Nov 2010 11:36:54 -0700: Tak Ota <Takaaki.Ota@am.sony.com> wrote:
> 
> > Thu, 4 Nov 2010 06:58:20 -0700: Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > 
> > > >> How about rename the command as collect-occur instead of
> > > >> collect-string and document it as next?
> > > > Wrong namespace.  IMO, the name should *start* with occur (but Stefan
> > > > and Yidong are final authorities on that).
> > > 
> > > Agreed, namespace cleanliness is one of my favorite forms of
> > > anal retentiveness.
> > > 
> > > The way I see it, the suggested collect-string is a variant of occur
> > > where the result buffer contains none of the regexp matches's context.
> > > So it would make sense to integrate it very tightly with `occur',
> > > i.e. make M-x occur do the job of collect-strings for some particular
> > > value of its argument NLINES.  Currently, NLINES is assumed to be
> > > a number and all values of that number have a useful meaning, so we'd
> > > have to add this new feature via a non-number value of NLINES.
> > > 
> > > E.g. C-u M-x occur could do the collect-string thingy (which is still an
> > > incompatible change since some people may like to use C-u M-x occur to get
> > > 4 lines of context, but you can make omelets without breaking eggs).
> > > 
> > > 
> > >         Stefan
> > > 
> > 
> > Now I am convinced.  How about the change below?  In conventional
> > occur zero or negative value for nlines is meaningless correct?  We
> > can use that for collection purpose.  i.e. C-u 0 M-x occur does the
> > collection of the matching pattern.  C-u -1 M-x occur performs the
> > collection of the recorded pattern 1.
> > 
> > -Tak
> > 
> > (defun occur-1 (regexp nlines bufs &optional buf-name)
> >   (unless (and regexp (not (equal regexp "")))
> >     (error "Occur doesn't work with the empty regexp"))
> >   (unless buf-name
> >     (setq buf-name "*Occur*"))
> >   (let (occur-buf
> > 	(active-bufs (delq nil (mapcar #'(lambda (buf)
> > 					   (when (buffer-live-p buf) buf))
> > 				       bufs))))
> >     ;; Handle the case where one of the buffers we're searching is the
> >     ;; output buffer.  Just rename it.
> >     (when (member buf-name (mapcar 'buffer-name active-bufs))
> >       (with-current-buffer (get-buffer buf-name)
> > 	(rename-uniquely)))
> > 
> >     ;; Now find or create the output buffer.
> >     ;; If we just renamed that buffer, we will make a new one here.
> >     (setq occur-buf (get-buffer-create buf-name))
> > 
> >     (if (or (null (integerp nlines))
> > 	    (> nlines 0))
> > 	;; nlines is not zero or negative so perform nomal occur
> > 	(with-current-buffer occur-buf
> > 	  (occur-mode)
> > 	  (let ((inhibit-read-only t)
> > 		;; Don't generate undo entries for creation of the initial contents.
> > 		(buffer-undo-list t))
> > 	    (erase-buffer)
> > 	    (let ((count (occur-engine
> > 			  regexp active-bufs occur-buf
> > 			  (or nlines list-matching-lines-default-context-lines)
> > 			  (if (and case-fold-search search-upper-case)
> > 			      (isearch-no-upper-case-p regexp t)
> > 			    case-fold-search)
> > 			  list-matching-lines-buffer-name-face
> > 			  nil list-matching-lines-face
> > 			  (not (eq occur-excluded-properties t)))))
> > 	      (let* ((bufcount (length active-bufs))
> > 		     (diff (- (length bufs) bufcount)))
> > 		(message "Searched %d buffer%s%s; %s match%s for `%s'"
> > 			 bufcount (if (= bufcount 1) "" "s")
> > 			 (if (zerop diff) "" (format " (%d killed)" diff))
> > 			 (if (zerop count) "no" (format "%d" count))
> > 			 (if (= count 1) "" "es")
> > 			 regexp))
> > 	      (setq occur-revert-arguments (list regexp nlines bufs))
> > 	      (if (= count 0)
> > 		  (kill-buffer occur-buf)
> > 		(display-buffer occur-buf)
> > 		(setq next-error-last-buffer occur-buf)
> > 		(setq buffer-read-only t)
> > 		(set-buffer-modified-p nil)
> > 		(run-hooks 'occur-hook)))))
> >       ;; nlines is zero or negative integer perform collect-string
> >       (with-current-buffer occur-buf
> > 	(setq nlines (- nlines))
> > 	(fundamental-mode)
> > 	(let ((inhibit-read-only t)
> > 	      (buffer-undo-list t))
> > 	  (erase-buffer)
> > 	  (while active-bufs
> > 	    (with-current-buffer (car active-bufs)
> > 	      (save-excursion
> > 		(goto-char (point-min))
> > 		(while (re-search-forward regexp nil t)
> > 		  (let ((str (match-string nlines)))
> > 		    (if str
> > 			(with-current-buffer occur-buf
> > 			  (insert str)
> > 			  (or (zerop (current-column))
> > 			      (insert "\n"))))))))
> > 	    (setq active-bufs (cdr active-bufs))))
> > 	(display-buffer occur-buf)))))

Re: collect-string (was: simple useful functions)

[Andreas Röhler]

Am 04.11.2010 21:27, schrieb Tak Ota:
> And here is the change log update.
>
> 2010-11-04  Takaaki Ota<Takaaki.Ota@am.sony.com>
>
> 	* replace.el (occur-1): Add string collection feature when NLINES
> 	is zero or negative integer value.
>
> -Tak
>
> Thu, 04 Nov 2010 13:18:20 -0700: Tak Ota<Takaaki.Ota@am.sony.com>  wrote:
>
>> The documentation can be something like this.  I just added the last
>> paragraph below.
>>
>> -Tak
>>
>> `M-x occur'
>>       Prompt for a regexp, and display a list showing each line in the
>>       buffer that contains a match for it.  To limit the search to part
>>       of the buffer, narrow to that part (*note Narrowing::).  A numeric
>>       argument N specifies that N lines of context are to be displayed
>>       before and after each matching line.  Currently, `occur' can not
>>       correctly handle multiline matches.
>>
>>       The buffer `*Occur*' containing the output serves as a menu for
>>       finding the occurrences in their original context.  Click
>>       `Mouse-2' on an occurrence listed in `*Occur*', or position point
>>       there and type<RET>; this switches to the buffer that was
>>       searched and moves point to the original of the chosen occurrence.
>>       `o' and `C-o' display the match in another window; `C-o' does not
>>       select it.
>>
>>       After using `M-x occur', you can use `next-error' to visit the
>>       occurrences found, one by one.  *note Compilation Mode::.
>>
>>       When the numeric argument N is 0 or negative the buffer `*Occur*'
>>       collects all the matched strings.  When N is 0 the entire text
>>       matched is collected.  When N is negative the text in the -Nth
>>       parenthesized expression in the regexp is collected.
>>
>>
>> Thu, 04 Nov 2010 11:36:54 -0700: Tak Ota<Takaaki.Ota@am.sony.com>  wrote:
>>
>>> Thu, 4 Nov 2010 06:58:20 -0700: Stefan Monnier<monnier@iro.umontreal.ca>  wrote:
>>>
>>>>>> How about rename the command as collect-occur instead of
>>>>>> collect-string and document it as next?
>>>>> Wrong namespace.  IMO, the name should *start* with occur (but Stefan
>>>>> and Yidong are final authorities on that).
>>>>
>>>> Agreed, namespace cleanliness is one of my favorite forms of
>>>> anal retentiveness.
>>>>
>>>> The way I see it, the suggested collect-string is a variant of occur
>>>> where the result buffer contains none of the regexp matches's context.
>>>> So it would make sense to integrate it very tightly with `occur',
>>>> i.e. make M-x occur do the job of collect-strings for some particular
>>>> value of its argument NLINES.  Currently, NLINES is assumed to be
>>>> a number and all values of that number have a useful meaning, so we'd
>>>> have to add this new feature via a non-number value of NLINES.
>>>>
>>>> E.g. C-u M-x occur could do the collect-string thingy (which is still an
>>>> incompatible change since some people may like to use C-u M-x occur to get
>>>> 4 lines of context, but you can make omelets without breaking eggs).
>>>>
>>>>
>>>>          Stefan
>>>>
>>>
>>> Now I am convinced.  How about the change below?  In conventional
>>> occur zero or negative value for nlines is meaningless correct?  We
>>> can use that for collection purpose.  i.e. C-u 0 M-x occur does the
>>> collection of the matching pattern.  C-u -1 M-x occur performs the
>>> collection of the recorded pattern 1.
>>>
>>> -Tak
>>>
>>> (defun occur-1 (regexp nlines bufs&optional buf-name)
>>>    (unless (and regexp (not (equal regexp "")))
>>>      (error "Occur doesn't work with the empty regexp"))
>>>    (unless buf-name
>>>      (setq buf-name "*Occur*"))
>>>    (let (occur-buf
>>> 	(active-bufs (delq nil (mapcar #'(lambda (buf)
>>> 					   (when (buffer-live-p buf) buf))
>>> 				       bufs))))
>>>      ;; Handle the case where one of the buffers we're searching is the
>>>      ;; output buffer.  Just rename it.
>>>      (when (member buf-name (mapcar 'buffer-name active-bufs))
>>>        (with-current-buffer (get-buffer buf-name)
>>> 	(rename-uniquely)))
>>>
>>>      ;; Now find or create the output buffer.
>>>      ;; If we just renamed that buffer, we will make a new one here.
>>>      (setq occur-buf (get-buffer-create buf-name))
>>>
>>>      (if (or (null (integerp nlines))
>>> 	    (>  nlines 0))
>>> 	;; nlines is not zero or negative so perform nomal occur
>>> 	(with-current-buffer occur-buf
>>> 	  (occur-mode)
>>> 	  (let ((inhibit-read-only t)
>>> 		;; Don't generate undo entries for creation of the initial contents.
>>> 		(buffer-undo-list t))
>>> 	    (erase-buffer)
>>> 	    (let ((count (occur-engine
>>> 			  regexp active-bufs occur-buf
>>> 			  (or nlines list-matching-lines-default-context-lines)
>>> 			  (if (and case-fold-search search-upper-case)
>>> 			      (isearch-no-upper-case-p regexp t)
>>> 			    case-fold-search)
>>> 			  list-matching-lines-buffer-name-face
>>> 			  nil list-matching-lines-face
>>> 			  (not (eq occur-excluded-properties t)))))
>>> 	      (let* ((bufcount (length active-bufs))
>>> 		     (diff (- (length bufs) bufcount)))
>>> 		(message "Searched %d buffer%s%s; %s match%s for `%s'"
>>> 			 bufcount (if (= bufcount 1) "" "s")
>>> 			 (if (zerop diff) "" (format " (%d killed)" diff))
>>> 			 (if (zerop count) "no" (format "%d" count))
>>> 			 (if (= count 1) "" "es")
>>> 			 regexp))
>>> 	      (setq occur-revert-arguments (list regexp nlines bufs))
>>> 	      (if (= count 0)
>>> 		  (kill-buffer occur-buf)
>>> 		(display-buffer occur-buf)
>>> 		(setq next-error-last-buffer occur-buf)
>>> 		(setq buffer-read-only t)
>>> 		(set-buffer-modified-p nil)
>>> 		(run-hooks 'occur-hook)))))
>>>        ;; nlines is zero or negative integer perform collect-string
>>>        (with-current-buffer occur-buf
>>> 	(setq nlines (- nlines))
>>> 	(fundamental-mode)
>>> 	(let ((inhibit-read-only t)
>>> 	      (buffer-undo-list t))
>>> 	  (erase-buffer)
>>> 	  (while active-bufs
>>> 	    (with-current-buffer (car active-bufs)
>>> 	      (save-excursion
>>> 		(goto-char (point-min))
>>> 		(while (re-search-forward regexp nil t)
>>> 		  (let ((str (match-string nlines)))
>>> 		    (if str
>>> 			(with-current-buffer occur-buf
>>> 			  (insert str)
>>> 			  (or (zerop (current-column))
>>> 			      (insert "\n"))))))))
>>> 	    (setq active-bufs (cdr active-bufs))))
>>> 	(display-buffer occur-buf)))))
>
>
>

Hi Tak,

please permit a consideration, which affects Emacs usability as such:

do you recognise the difference between

;;;;;;;;;;;;;;;;;;;;;
  When the numeric argument N is 0 or negative the buffer `*Occur*'
      collects all the matched strings.  When N is 0 the entire text
      matched is collected.  When N is negative the text in the -Nth
      parenthesized expression in the regexp is collected.
;;;;;;;;;;;;;;;;;;;;

and your original describtion

;;;;;;;;;;;;;;;;;;;
  "Collect strings of REGEXP (or optional NUM paren) from the
current buffer into a collection buffer."
;;;;;;;;;;;;;;;;;;;

Which one would you think is more readable?

Cheers

Andreas

Re: collect-string

[Stefan Monnier]

> Now I am convinced.  How about the change below?

Could you send it as a context patch?

> In conventional occur zero or negative value for nlines is meaningless
> correct?

Hmmm....no, C-h f occur RET says:

   Each line is displayed with NLINES lines before and after, or -NLINES
   before if NLINES is negative.

So negative values are not meaningless and neither is zero.  Zero is
unnecessary in the default config, but if the user has set
list-matching-lines-default-context-lines to a non-zero value, then even
zero is useful (I'd even say, it's *very* useful in that case).

Which is why I suggested to use a non-numeric value (just a C-u) to
trigger the "collect-string" behavior.


        Stefan

Re: collect-string

[Tak Ota]

Mon, 8 Nov 2010 10:36:08 -0800: Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > Now I am convinced.  How about the change below?
> 
> Could you send it as a context patch?

I will once we figure out the right way.

> > In conventional occur zero or negative value for nlines is meaningless
> > correct?
> 
> Hmmm....no, C-h f occur RET says:
> 
>    Each line is displayed with NLINES lines before and after, or -NLINES
>    before if NLINES is negative.
> 
> So negative values are not meaningless and neither is zero.  Zero is
> unnecessary in the default config, but if the user has set
> list-matching-lines-default-context-lines to a non-zero value, then even
> zero is useful (I'd even say, it's *very* useful in that case).

You are right.  I don't know how missed that.

> Which is why I suggested to use a non-numeric value (just a C-u) to
> trigger the "collect-string" behavior.

But isn't C-u equivalent to numeric value 4?

-Tak

Re: collect-string

[Stephen J. Turnbull]

Tak Ota writes:

 > But isn't C-u equivalent to numeric value 4?

No.  C-u delivers a non-numeric value (a list, actually) for the
corresponding arg if "P" is used in the interactive spec.  If "N" or
"p" is specified, it is automatically converted to a number (a power
of 4 depending on how many times C-u was pressed).

C-h f interactive RET

or

(defun show-me-C-u (arg)
  (interactive "P")
  (message "ARG is %s." arg))

Regards,

Re: collect-string

[Tak Ota]

Mon, 8 Nov 2010 10:36:08 -0800: Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > Now I am convinced.  How about the change below?
> 
> Could you send it as a context patch?


How about this?

-Tak


*** replace.el.orig	Tue Nov  9 11:41:08 2010
--- replace.el	Tue Nov  9 18:09:30 2010
***************
*** 527,532 ****
--- 527,535 ----
  Maximum length of the history list is determined by the value
  of `history-length', which see.")
  
+ (defvar last-collect-number 0
+   "The last parenthesized expression number.")
+ 
  (defun read-regexp (prompt &optional default-value)
    "Read regexp as a string using the regexp history and some useful defaults.
  Prompt for a regular expression with PROMPT (without a colon and
***************
*** 1031,1037 ****
    (list (read-regexp "List lines matching regexp"
  		     (car regexp-history))
  	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
--- 1034,1044 ----
    (list (read-regexp "List lines matching regexp"
  		     (car regexp-history))
  	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))
! 	(and current-prefix-arg
! 	     (listp current-prefix-arg)
! 	     (setq last-collect-number
! 		   (read-number "Collect" last-collect-number)))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
***************
*** 1050,1056 ****
                             "*")
                     (or unique-p (not interactive-p)))))
  
! (defun occur (regexp &optional nlines)
    "Show all lines in the current buffer containing a match for REGEXP.
  This function can not handle matches that span more than one line.
  
--- 1057,1063 ----
                             "*")
                     (or unique-p (not interactive-p)))))
  
! (defun occur (regexp &optional nlines collect)
    "Show all lines in the current buffer containing a match for REGEXP.
  This function can not handle matches that span more than one line.
  
***************
*** 1064,1074 ****
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive."
    (interactive (occur-read-primary-args))
!   (occur-1 regexp nlines (list (current-buffer))))
  
! (defun multi-occur (bufs regexp &optional nlines)
    "Show all lines in buffers BUFS containing a match for REGEXP.
  This function acts on multiple buffers; otherwise, it is exactly like
  `occur'.  When you invoke this command interactively, you must specify
--- 1071,1087 ----
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive.
! 
! When prefix argument is provided without a number (`C-u' alone as
! prefix) the matching strings are collected into the `*Occur*'
! buffer.  When COLLECT is zero the entire matched strings are
! collected otherwise COLLECTth pair of parenthesized match in the
! REGEXP is collected."
    (interactive (occur-read-primary-args))
!   (occur-1 regexp nlines (list (current-buffer)) nil collect))
  
! (defun multi-occur (bufs regexp &optional nlines collect)
    "Show all lines in buffers BUFS containing a match for REGEXP.
  This function acts on multiple buffers; otherwise, it is exactly like
  `occur'.  When you invoke this command interactively, you must specify
***************
*** 1090,1098 ****
  	(setq ido-ignore-item-temp-list bufs))
        (nreverse (mapcar #'get-buffer bufs)))
      (occur-read-primary-args)))
!   (occur-1 regexp nlines bufs))
  
! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs)
    "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
  Normally BUFREGEXP matches against each buffer's visited file name,
  but if you specify a prefix argument, it matches against the buffer name.
--- 1103,1111 ----
  	(setq ido-ignore-item-temp-list bufs))
        (nreverse (mapcar #'get-buffer bufs)))
      (occur-read-primary-args)))
!   (occur-1 regexp nlines bufs nil collect))
  
! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs collect)
    "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
  Normally BUFREGEXP matches against each buffer's visited file name,
  but if you specify a prefix argument, it matches against the buffer name.
***************
*** 1124,1132 ****
  					  (string-match bufregexp
  							(buffer-file-name buf))))
  			       buf))
! 			   (buffer-list))))))
  
! (defun occur-1 (regexp nlines bufs &optional buf-name)
    (unless (and regexp (not (equal regexp "")))
      (error "Occur doesn't work with the empty regexp"))
    (unless buf-name
--- 1137,1146 ----
  					  (string-match bufregexp
  							(buffer-file-name buf))))
  			       buf))
! 			   (buffer-list)))
! 	     nil collect)))
  
! (defun occur-1 (regexp nlines bufs &optional buf-name collect)
    (unless (and regexp (not (equal regexp "")))
      (error "Occur doesn't work with the empty regexp"))
    (unless buf-name
***************
*** 1145,1181 ****
      ;; If we just renamed that buffer, we will make a new one here.
      (setq occur-buf (get-buffer-create buf-name))
  
!     (with-current-buffer occur-buf
!       (occur-mode)
!       (let ((inhibit-read-only t)
! 	    ;; Don't generate undo entries for creation of the initial contents.
! 	    (buffer-undo-list t))
! 	(erase-buffer)
! 	(let ((count (occur-engine
! 		      regexp active-bufs occur-buf
! 		      (or nlines list-matching-lines-default-context-lines)
! 		      (if (and case-fold-search search-upper-case)
! 			  (isearch-no-upper-case-p regexp t)
! 			case-fold-search)
! 		      list-matching-lines-buffer-name-face
! 		      nil list-matching-lines-face
! 		      (not (eq occur-excluded-properties t)))))
! 	  (let* ((bufcount (length active-bufs))
! 		 (diff (- (length bufs) bufcount)))
! 	    (message "Searched %d buffer%s%s; %s match%s for `%s'"
! 		     bufcount (if (= bufcount 1) "" "s")
! 		     (if (zerop diff) "" (format " (%d killed)" diff))
! 		     (if (zerop count) "no" (format "%d" count))
! 		     (if (= count 1) "" "es")
! 		     regexp))
! 	  (setq occur-revert-arguments (list regexp nlines bufs))
!           (if (= count 0)
!               (kill-buffer occur-buf)
!             (display-buffer occur-buf)
!             (setq next-error-last-buffer occur-buf)
!             (setq buffer-read-only t)
!             (set-buffer-modified-p nil)
!             (run-hooks 'occur-hook)))))))
  
  (defun occur-engine-add-prefix (lines)
    (mapcar
--- 1159,1214 ----
      ;; If we just renamed that buffer, we will make a new one here.
      (setq occur-buf (get-buffer-create buf-name))
  
!     (if collect
! 	(with-current-buffer occur-buf
! 	  (fundamental-mode)
! 	  (let ((inhibit-read-only t)
! 		(buffer-undo-list t))
! 	    (erase-buffer)
! 	    (while active-bufs
! 	      (with-current-buffer (car active-bufs)
! 		(save-excursion
! 		  (goto-char (point-min))
! 		  (while (re-search-forward regexp nil t)
! 		    (let ((str (match-string collect)))
! 		      (if str
! 			  (with-current-buffer occur-buf
! 			    (insert str)
! 			    (or (zerop (current-column))
! 				(insert "\n"))))))))
! 	      (setq active-bufs (cdr active-bufs))))
! 	  (display-buffer occur-buf))
!       (with-current-buffer occur-buf
! 	(occur-mode)
! 	(let ((inhibit-read-only t)
! 	      ;; Don't generate undo entries for creation of the initial contents.
! 	      (buffer-undo-list t))
! 	  (erase-buffer)
! 	  (let ((count (occur-engine
! 			regexp active-bufs occur-buf
! 			(or nlines list-matching-lines-default-context-lines)
! 			(if (and case-fold-search search-upper-case)
! 			    (isearch-no-upper-case-p regexp t)
! 			  case-fold-search)
! 			list-matching-lines-buffer-name-face
! 			nil list-matching-lines-face
! 			(not (eq occur-excluded-properties t)))))
! 	    (let* ((bufcount (length active-bufs))
! 		   (diff (- (length bufs) bufcount)))
! 	      (message "Searched %d buffer%s%s; %s match%s for `%s'"
! 		       bufcount (if (= bufcount 1) "" "s")
! 		       (if (zerop diff) "" (format " (%d killed)" diff))
! 		       (if (zerop count) "no" (format "%d" count))
! 		       (if (= count 1) "" "es")
! 		       regexp))
! 	    (setq occur-revert-arguments (list regexp nlines bufs))
! 	    (if (= count 0)
! 		(kill-buffer occur-buf)
! 	      (display-buffer occur-buf)
! 	      (setq next-error-last-buffer occur-buf)
! 	      (setq buffer-read-only t)
! 	      (set-buffer-modified-p nil)
! 	      (run-hooks 'occur-hook))))))))
  
  (defun occur-engine-add-prefix (lines)
    (mapcar

Re: collect-string

[Tak Ota]

Please somebody comment on this.

-Tak

Tue, 09 Nov 2010 18:12:50 -0800: Tak Ota <Takaaki.Ota@am.sony.com> wrote:

> Mon, 8 Nov 2010 10:36:08 -0800: Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> > > Now I am convinced.  How about the change below?
> > 
> > Could you send it as a context patch?
> 
> 
> How about this?
> 
> -Tak
> 
> 
> *** replace.el.orig	Tue Nov  9 11:41:08 2010
> --- replace.el	Tue Nov  9 18:09:30 2010
> ***************
> *** 527,532 ****
> --- 527,535 ----
>   Maximum length of the history list is determined by the value
>   of `history-length', which see.")
>   
> + (defvar last-collect-number 0
> +   "The last parenthesized expression number.")
> + 
>   (defun read-regexp (prompt &optional default-value)
>     "Read regexp as a string using the regexp history and some useful defaults.
>   Prompt for a regular expression with PROMPT (without a colon and
> ***************
> *** 1031,1037 ****
>     (list (read-regexp "List lines matching regexp"
>   		     (car regexp-history))
>   	(when current-prefix-arg
> ! 	  (prefix-numeric-value current-prefix-arg))))
>   
>   (defun occur-rename-buffer (&optional unique-p interactive-p)
>     "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
> --- 1034,1044 ----
>     (list (read-regexp "List lines matching regexp"
>   		     (car regexp-history))
>   	(when current-prefix-arg
> ! 	  (prefix-numeric-value current-prefix-arg))
> ! 	(and current-prefix-arg
> ! 	     (listp current-prefix-arg)
> ! 	     (setq last-collect-number
> ! 		   (read-number "Collect" last-collect-number)))))
>   
>   (defun occur-rename-buffer (&optional unique-p interactive-p)
>     "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
> ***************
> *** 1050,1056 ****
>                              "*")
>                      (or unique-p (not interactive-p)))))
>   
> ! (defun occur (regexp &optional nlines)
>     "Show all lines in the current buffer containing a match for REGEXP.
>   This function can not handle matches that span more than one line.
>   
> --- 1057,1063 ----
>                              "*")
>                      (or unique-p (not interactive-p)))))
>   
> ! (defun occur (regexp &optional nlines collect)
>     "Show all lines in the current buffer containing a match for REGEXP.
>   This function can not handle matches that span more than one line.
>   
> ***************
> *** 1064,1074 ****
>   \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
>   
>   If REGEXP contains upper case characters (excluding those preceded by `\\')
> ! and `search-upper-case' is non-nil, the matching is case-sensitive."
>     (interactive (occur-read-primary-args))
> !   (occur-1 regexp nlines (list (current-buffer))))
>   
> ! (defun multi-occur (bufs regexp &optional nlines)
>     "Show all lines in buffers BUFS containing a match for REGEXP.
>   This function acts on multiple buffers; otherwise, it is exactly like
>   `occur'.  When you invoke this command interactively, you must specify
> --- 1071,1087 ----
>   \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
>   
>   If REGEXP contains upper case characters (excluding those preceded by `\\')
> ! and `search-upper-case' is non-nil, the matching is case-sensitive.
> ! 
> ! When prefix argument is provided without a number (`C-u' alone as
> ! prefix) the matching strings are collected into the `*Occur*'
> ! buffer.  When COLLECT is zero the entire matched strings are
> ! collected otherwise COLLECTth pair of parenthesized match in the
> ! REGEXP is collected."
>     (interactive (occur-read-primary-args))
> !   (occur-1 regexp nlines (list (current-buffer)) nil collect))
>   
> ! (defun multi-occur (bufs regexp &optional nlines collect)
>     "Show all lines in buffers BUFS containing a match for REGEXP.
>   This function acts on multiple buffers; otherwise, it is exactly like
>   `occur'.  When you invoke this command interactively, you must specify
> ***************
> *** 1090,1098 ****
>   	(setq ido-ignore-item-temp-list bufs))
>         (nreverse (mapcar #'get-buffer bufs)))
>       (occur-read-primary-args)))
> !   (occur-1 regexp nlines bufs))
>   
> ! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs)
>     "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
>   Normally BUFREGEXP matches against each buffer's visited file name,
>   but if you specify a prefix argument, it matches against the buffer name.
> --- 1103,1111 ----
>   	(setq ido-ignore-item-temp-list bufs))
>         (nreverse (mapcar #'get-buffer bufs)))
>       (occur-read-primary-args)))
> !   (occur-1 regexp nlines bufs nil collect))
>   
> ! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs collect)
>     "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
>   Normally BUFREGEXP matches against each buffer's visited file name,
>   but if you specify a prefix argument, it matches against the buffer name.
> ***************
> *** 1124,1132 ****
>   					  (string-match bufregexp
>   							(buffer-file-name buf))))
>   			       buf))
> ! 			   (buffer-list))))))
>   
> ! (defun occur-1 (regexp nlines bufs &optional buf-name)
>     (unless (and regexp (not (equal regexp "")))
>       (error "Occur doesn't work with the empty regexp"))
>     (unless buf-name
> --- 1137,1146 ----
>   					  (string-match bufregexp
>   							(buffer-file-name buf))))
>   			       buf))
> ! 			   (buffer-list)))
> ! 	     nil collect)))
>   
> ! (defun occur-1 (regexp nlines bufs &optional buf-name collect)
>     (unless (and regexp (not (equal regexp "")))
>       (error "Occur doesn't work with the empty regexp"))
>     (unless buf-name
> ***************
> *** 1145,1181 ****
>       ;; If we just renamed that buffer, we will make a new one here.
>       (setq occur-buf (get-buffer-create buf-name))
>   
> !     (with-current-buffer occur-buf
> !       (occur-mode)
> !       (let ((inhibit-read-only t)
> ! 	    ;; Don't generate undo entries for creation of the initial contents.
> ! 	    (buffer-undo-list t))
> ! 	(erase-buffer)
> ! 	(let ((count (occur-engine
> ! 		      regexp active-bufs occur-buf
> ! 		      (or nlines list-matching-lines-default-context-lines)
> ! 		      (if (and case-fold-search search-upper-case)
> ! 			  (isearch-no-upper-case-p regexp t)
> ! 			case-fold-search)
> ! 		      list-matching-lines-buffer-name-face
> ! 		      nil list-matching-lines-face
> ! 		      (not (eq occur-excluded-properties t)))))
> ! 	  (let* ((bufcount (length active-bufs))
> ! 		 (diff (- (length bufs) bufcount)))
> ! 	    (message "Searched %d buffer%s%s; %s match%s for `%s'"
> ! 		     bufcount (if (= bufcount 1) "" "s")
> ! 		     (if (zerop diff) "" (format " (%d killed)" diff))
> ! 		     (if (zerop count) "no" (format "%d" count))
> ! 		     (if (= count 1) "" "es")
> ! 		     regexp))
> ! 	  (setq occur-revert-arguments (list regexp nlines bufs))
> !           (if (= count 0)
> !               (kill-buffer occur-buf)
> !             (display-buffer occur-buf)
> !             (setq next-error-last-buffer occur-buf)
> !             (setq buffer-read-only t)
> !             (set-buffer-modified-p nil)
> !             (run-hooks 'occur-hook)))))))
>   
>   (defun occur-engine-add-prefix (lines)
>     (mapcar
> --- 1159,1214 ----
>       ;; If we just renamed that buffer, we will make a new one here.
>       (setq occur-buf (get-buffer-create buf-name))
>   
> !     (if collect
> ! 	(with-current-buffer occur-buf
> ! 	  (fundamental-mode)
> ! 	  (let ((inhibit-read-only t)
> ! 		(buffer-undo-list t))
> ! 	    (erase-buffer)
> ! 	    (while active-bufs
> ! 	      (with-current-buffer (car active-bufs)
> ! 		(save-excursion
> ! 		  (goto-char (point-min))
> ! 		  (while (re-search-forward regexp nil t)
> ! 		    (let ((str (match-string collect)))
> ! 		      (if str
> ! 			  (with-current-buffer occur-buf
> ! 			    (insert str)
> ! 			    (or (zerop (current-column))
> ! 				(insert "\n"))))))))
> ! 	      (setq active-bufs (cdr active-bufs))))
> ! 	  (display-buffer occur-buf))
> !       (with-current-buffer occur-buf
> ! 	(occur-mode)
> ! 	(let ((inhibit-read-only t)
> ! 	      ;; Don't generate undo entries for creation of the initial contents.
> ! 	      (buffer-undo-list t))
> ! 	  (erase-buffer)
> ! 	  (let ((count (occur-engine
> ! 			regexp active-bufs occur-buf
> ! 			(or nlines list-matching-lines-default-context-lines)
> ! 			(if (and case-fold-search search-upper-case)
> ! 			    (isearch-no-upper-case-p regexp t)
> ! 			  case-fold-search)
> ! 			list-matching-lines-buffer-name-face
> ! 			nil list-matching-lines-face
> ! 			(not (eq occur-excluded-properties t)))))
> ! 	    (let* ((bufcount (length active-bufs))
> ! 		   (diff (- (length bufs) bufcount)))
> ! 	      (message "Searched %d buffer%s%s; %s match%s for `%s'"
> ! 		       bufcount (if (= bufcount 1) "" "s")
> ! 		       (if (zerop diff) "" (format " (%d killed)" diff))
> ! 		       (if (zerop count) "no" (format "%d" count))
> ! 		       (if (= count 1) "" "es")
> ! 		       regexp))
> ! 	    (setq occur-revert-arguments (list regexp nlines bufs))
> ! 	    (if (= count 0)
> ! 		(kill-buffer occur-buf)
> ! 	      (display-buffer occur-buf)
> ! 	      (setq next-error-last-buffer occur-buf)
> ! 	      (setq buffer-read-only t)
> ! 	      (set-buffer-modified-p nil)
> ! 	      (run-hooks 'occur-hook))))))))
>   
>   (defun occur-engine-add-prefix (lines)
>     (mapcar

Re: collect-string

[Stephen J. Turnbull]

Tak Ota writes:

 > > ! When prefix argument is provided without a number (`C-u' alone as
 > > ! prefix) the matching strings are collected into the `*Occur*'
 > > ! buffer.  When COLLECT is zero the entire matched strings are
 > > ! collected otherwise COLLECTth pair of parenthesized match in the
 > > ! REGEXP is collected."

I don't think this describes the behavior of the prefix argument
correctly (you will be prompted for a number, IIUC).  What happens
when the prefix argument is a number?

 > > ! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs collect)
 > >     "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
 > >   Normally BUFREGEXP matches against each buffer's visited file name,
 > >   but if you specify a prefix argument, it matches against the buffer name.

The COLLECT argument is undocumented.

Aside to the maintainers: the ALLBUFS argument could use further
clarification, but that's not Tak's responsibility.

Re: collect-string

[Tak Ota]

Hi Stephen,

Thanks for the comments.

Mon, 29 Nov 2010 21:27:12 -0800: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp> wrote:

> Tak Ota writes:
> 
>  > > ! When prefix argument is provided without a number (`C-u' alone as
>  > > ! prefix) the matching strings are collected into the `*Occur*'
>  > > ! buffer.  When COLLECT is zero the entire matched strings are
>  > > ! collected otherwise COLLECTth pair of parenthesized match in the
>  > > ! REGEXP is collected."
> 
> I don't think this describes the behavior of the prefix argument
> correctly (you will be prompted for a number, IIUC).  What happens
> when the prefix argument is a number?

I modified the description slightly.

>  > > ! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs collect)
>  > >     "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
>  > >   Normally BUFREGEXP matches against each buffer's visited file name,
>  > >   but if you specify a prefix argument, it matches against the buffer name.
> 
> The COLLECT argument is undocumented.

I just replicated the same paragraph from occur to multi-occur and
multi-occur-in-matching-buffers below.  I added some examples to
demonstrate how this feature is useful for some tasks.

*** ../../../d/pub/emacs/emacs-23.2.90/lisp/replace.el	Tue Oct 12 15:43:12 2010
--- replace.el	Wed Dec  1 17:55:58 2010
***************
*** 527,532 ****
--- 527,535 ----
  Maximum length of the history list is determined by the value
  of `history-length', which see.")
  
+ (defvar last-collect-number 0
+   "The last parenthesized expression number.")
+ 
  (defun read-regexp (prompt &optional default-value)
    "Read regexp as a string using the regexp history and some useful defaults.
  Prompt for a regular expression with PROMPT (without a colon and
***************
*** 1031,1037 ****
    (list (read-regexp "List lines matching regexp"
  		     (car regexp-history))
  	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
--- 1034,1044 ----
    (list (read-regexp "List lines matching regexp"
  		     (car regexp-history))
  	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))
! 	(and current-prefix-arg
! 	     (listp current-prefix-arg)
! 	     (setq last-collect-number
! 		   (read-number "Collect" last-collect-number)))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
***************
*** 1050,1056 ****
                             "*")
                     (or unique-p (not interactive-p)))))
  
! (defun occur (regexp &optional nlines)
    "Show all lines in the current buffer containing a match for REGEXP.
  This function can not handle matches that span more than one line.
  
--- 1057,1063 ----
                             "*")
                     (or unique-p (not interactive-p)))))
  
! (defun occur (regexp &optional nlines collect)
    "Show all lines in the current buffer containing a match for REGEXP.
  This function can not handle matches that span more than one line.
  
***************
*** 1064,1078 ****
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive."
    (interactive (occur-read-primary-args))
!   (occur-1 regexp nlines (list (current-buffer))))
  
! (defun multi-occur (bufs regexp &optional nlines)
    "Show all lines in buffers BUFS containing a match for REGEXP.
  This function acts on multiple buffers; otherwise, it is exactly like
  `occur'.  When you invoke this command interactively, you must specify
! the buffer names that you want, one by one."
    (interactive
     (cons
      (let* ((bufs (list (read-buffer "First buffer to search: "
--- 1071,1105 ----
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive.
! 
! When the prefix argument is a list (you type `C-u' alone without
! a number) the matching strings are collected into the `*Occur*'
! buffer.  When COLLECT is zero the entire matched strings are
! collected, which is not very useful, otherwise COLLECTth pair of
! parenthesized match in REGEXP is collected.  For example,
! providing \"defun \\(\\S +\\)\" for REGEXP and 1 for COLLECT
! collects all the function names in the lisp program.  Another
! example, \"<img src=\"\\([^\"]+\\)\" for the REGEXP and 1 for
! COLLECT collects all the image URLs in the HTML file"
    (interactive (occur-read-primary-args))
!   (occur-1 regexp nlines (list (current-buffer)) nil collect))
  
! (defun multi-occur (bufs regexp &optional nlines collect)
    "Show all lines in buffers BUFS containing a match for REGEXP.
  This function acts on multiple buffers; otherwise, it is exactly like
  `occur'.  When you invoke this command interactively, you must specify
! the buffer names that you want, one by one.
! 
! When the prefix argument is a list (you type `C-u' alone without
! a number) the matching strings are collected into the `*Occur*'
! buffer.  When COLLECT is zero the entire matched strings are
! collected, which is not very useful, otherwise COLLECTth pair of
! parenthesized match in REGEXP is collected.  For example,
! providing \"defun \\(\\S +\\)\" for REGEXP and 1 for COLLECT
! collects all the function names in the lisp program.  Another
! example, \"<img src=\"\\([^\"]+\\)\" for the REGEXP and 1 for
! COLLECT collects all the image URLs in the HTML file"
    (interactive
     (cons
      (let* ((bufs (list (read-buffer "First buffer to search: "
***************
*** 1090,1101 ****
  	(setq ido-ignore-item-temp-list bufs))
        (nreverse (mapcar #'get-buffer bufs)))
      (occur-read-primary-args)))
!   (occur-1 regexp nlines bufs))
  
! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs)
    "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
  Normally BUFREGEXP matches against each buffer's visited file name,
  but if you specify a prefix argument, it matches against the buffer name.
  See also `multi-occur'."
    (interactive
     (cons
--- 1117,1139 ----
  	(setq ido-ignore-item-temp-list bufs))
        (nreverse (mapcar #'get-buffer bufs)))
      (occur-read-primary-args)))
!   (occur-1 regexp nlines bufs nil collect))
  
! (defun multi-occur-in-matching-buffers (bufregexp regexp &optional allbufs collect)
    "Show all lines matching REGEXP in buffers specified by BUFREGEXP.
  Normally BUFREGEXP matches against each buffer's visited file name,
  but if you specify a prefix argument, it matches against the buffer name.
+ 
+ When the prefix argument is a list (you type `C-u' alone without
+ a number) the matching strings are collected into the `*Occur*'
+ buffer.  When COLLECT is zero the entire matched strings are
+ collected, which is not very useful, otherwise COLLECTth pair of
+ parenthesized match in REGEXP is collected.  For example,
+ providing \"defun \\(\\S +\\)\" for REGEXP and 1 for COLLECT
+ collects all the function names in the lisp program.  Another
+ example, \"<img src=\"\\([^\"]+\\)\" for the REGEXP and 1 for
+ COLLECT collects all the image URLs in the HTML file
+ 
  See also `multi-occur'."
    (interactive
     (cons
***************
*** 1124,1132 ****
  					  (string-match bufregexp
  							(buffer-file-name buf))))
  			       buf))
! 			   (buffer-list))))))
  
! (defun occur-1 (regexp nlines bufs &optional buf-name)
    (unless (and regexp (not (equal regexp "")))
      (error "Occur doesn't work with the empty regexp"))
    (unless buf-name
--- 1162,1171 ----
  					  (string-match bufregexp
  							(buffer-file-name buf))))
  			       buf))
! 			   (buffer-list)))
! 	     nil collect)))
  
! (defun occur-1 (regexp nlines bufs &optional buf-name collect)
    (unless (and regexp (not (equal regexp "")))
      (error "Occur doesn't work with the empty regexp"))
    (unless buf-name
***************
*** 1145,1181 ****
      ;; If we just renamed that buffer, we will make a new one here.
      (setq occur-buf (get-buffer-create buf-name))
  
!     (with-current-buffer occur-buf
!       (occur-mode)
!       (let ((inhibit-read-only t)
! 	    ;; Don't generate undo entries for creation of the initial contents.
! 	    (buffer-undo-list t))
! 	(erase-buffer)
! 	(let ((count (occur-engine
! 		      regexp active-bufs occur-buf
! 		      (or nlines list-matching-lines-default-context-lines)
! 		      (if (and case-fold-search search-upper-case)
! 			  (isearch-no-upper-case-p regexp t)
! 			case-fold-search)
! 		      list-matching-lines-buffer-name-face
! 		      nil list-matching-lines-face
! 		      (not (eq occur-excluded-properties t)))))
! 	  (let* ((bufcount (length active-bufs))
! 		 (diff (- (length bufs) bufcount)))
! 	    (message "Searched %d buffer%s%s; %s match%s for `%s'"
! 		     bufcount (if (= bufcount 1) "" "s")
! 		     (if (zerop diff) "" (format " (%d killed)" diff))
! 		     (if (zerop count) "no" (format "%d" count))
! 		     (if (= count 1) "" "es")
! 		     regexp))
! 	  (setq occur-revert-arguments (list regexp nlines bufs))
!           (if (= count 0)
!               (kill-buffer occur-buf)
!             (display-buffer occur-buf)
!             (setq next-error-last-buffer occur-buf)
!             (setq buffer-read-only t)
!             (set-buffer-modified-p nil)
!             (run-hooks 'occur-hook)))))))
  
  (defun occur-engine-add-prefix (lines)
    (mapcar
--- 1184,1239 ----
      ;; If we just renamed that buffer, we will make a new one here.
      (setq occur-buf (get-buffer-create buf-name))
  
!     (if collect
! 	(with-current-buffer occur-buf
! 	  (fundamental-mode)
! 	  (let ((inhibit-read-only t)
! 		(buffer-undo-list t))
! 	    (erase-buffer)
! 	    (while active-bufs
! 	      (with-current-buffer (car active-bufs)
! 		(save-excursion
! 		  (goto-char (point-min))
! 		  (while (re-search-forward regexp nil t)
! 		    (let ((str (match-string collect)))
! 		      (if str
! 			  (with-current-buffer occur-buf
! 			    (insert str)
! 			    (or (zerop (current-column))
! 				(insert "\n"))))))))
! 	      (setq active-bufs (cdr active-bufs))))
! 	  (display-buffer occur-buf))
!       (with-current-buffer occur-buf
! 	(occur-mode)
! 	(let ((inhibit-read-only t)
! 	      ;; Don't generate undo entries for creation of the initial contents.
! 	      (buffer-undo-list t))
! 	  (erase-buffer)
! 	  (let ((count (occur-engine
! 			regexp active-bufs occur-buf
! 			(or nlines list-matching-lines-default-context-lines)
! 			(if (and case-fold-search search-upper-case)
! 			    (isearch-no-upper-case-p regexp t)
! 			  case-fold-search)
! 			list-matching-lines-buffer-name-face
! 			nil list-matching-lines-face
! 			(not (eq occur-excluded-properties t)))))
! 	    (let* ((bufcount (length active-bufs))
! 		   (diff (- (length bufs) bufcount)))
! 	      (message "Searched %d buffer%s%s; %s match%s for `%s'"
! 		       bufcount (if (= bufcount 1) "" "s")
! 		       (if (zerop diff) "" (format " (%d killed)" diff))
! 		       (if (zerop count) "no" (format "%d" count))
! 		       (if (= count 1) "" "es")
! 		       regexp))
! 	    (setq occur-revert-arguments (list regexp nlines bufs))
! 	    (if (= count 0)
! 		(kill-buffer occur-buf)
! 	      (display-buffer occur-buf)
! 	      (setq next-error-last-buffer occur-buf)
! 	      (setq buffer-read-only t)
! 	      (set-buffer-modified-p nil)
! 	      (run-hooks 'occur-hook))))))))
  
  (defun occur-engine-add-prefix (lines)
    (mapcar

> 
> Aside to the maintainers: the ALLBUFS argument could use further
> clarification, but that's not Tak's responsibility.

That's right :)

-Tak

Re: collect-string

[Stephen J. Turnbull]

Tak Ota writes:
 > Hi Stephen,
 > 
 > Thanks for the comments.
 > 
 > Mon, 29 Nov 2010 21:27:12 -0800: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp> wrote:
 > > Tak Ota writes:

 > > I don't think this describes the behavior of the prefix argument [...]

 > I modified the description slightly.

Much better, thank you (but it's Stefan's opinion that matters, of
course).

 > > The COLLECT argument is undocumented.
 > 
 > I just replicated the same paragraph from occur to multi-occur and
 > multi-occur-in-matching-buffers below.  I added some examples to
 > demonstrate how this feature is useful for some tasks.

I'm not sure whether either that kind of duplication, or such
examples, are considered good style in Emacs docstrings or not.
You'll need to wait for Stefan's comment.  But:

 > ! buffer.  When COLLECT is zero the entire matched strings are
 > ! collected, which is not very useful,

Hmm ... it occurs to me that M-0 and M-1 are really the most useful
numeric values.  Otherwise, you have to plan ahead.  A regexp with
multiple capturing parens is really pretty complex; I suspect most of
the time if you actually need such a thing you'll only notice it as
you type the regexp (after entering the numeric value).  If you don't
need capturing groups, although shy groups are a little more typing
they have the advantage that you can go, "oh, I need a group here" and
insert it without invalidating your prefix argument.

Maybe you could use M-0 for the natural application of "collect the
whole match" and C-u (without numerical value) for "prompt for group
after reading the regexp"?

Re: collect-string

[Stefan Monnier]

>> > Now I am convinced.  How about the change below?
>> Could you send it as a context patch?
> How about this?

IIUC the new arg `collect' overrides `lines' in the sense that when
collect is non-nil lines is simply unused.  So I suggest to just use
a single argument instead (`lines') and simply give additional semantics
to values that weren't supported until now.
E.g. if (numberp lines) then do as before, but if (consp lines) then
only keep the actual match, where (car lines) could specify which
submatch to keep.

>     (list (read-regexp "List lines matching regexp"
>   		     (car regexp-history))
>   	(when current-prefix-arg
> ! 	  (prefix-numeric-value current-prefix-arg))
> ! 	(and current-prefix-arg
> ! 	     (listp current-prefix-arg)
> ! 	     (setq last-collect-number
> ! 		   (read-number "Collect" last-collect-number)))))

I recommend you first check (with regexp-opt-depth) whether there is
a submatch, and if not, don't `read-number'.  It's likely to be a very
common case.

> !     (if collect
> ! 	(with-current-buffer occur-buf
> ! 	  (fundamental-mode)
> ! 	  (let ((inhibit-read-only t)
> ! 		(buffer-undo-list t))
> ! 	    (erase-buffer)
> ! 	    (while active-bufs
> ! 	      (with-current-buffer (car active-bufs)
> ! 		(save-excursion
> ! 		  (goto-char (point-min))
> ! 		  (while (re-search-forward regexp nil t)
> ! 		    (let ((str (match-string collect)))
> ! 		      (if str
> ! 			  (with-current-buffer occur-buf
> ! 			    (insert str)
> ! 			    (or (zerop (current-column))
> ! 				(insert "\n"))))))))
> ! 	      (setq active-bufs (cdr active-bufs))))
> ! 	  (display-buffer occur-buf))
> !       (with-current-buffer occur-buf
> ! 	(occur-mode)
> ! 	(let ((inhibit-read-only t)
> ! 	      ;; Don't generate undo entries for creation of the initial contents.
> ! 	      (buffer-undo-list t))
> ! 	  (erase-buffer)
> ! 	  (let ((count (occur-engine
> ! 			regexp active-bufs occur-buf
> ! 			(or nlines list-matching-lines-default-context-lines)
> ! 			(if (and case-fold-search search-upper-case)
> ! 			    (isearch-no-upper-case-p regexp t)
> ! 			  case-fold-search)
> ! 			list-matching-lines-buffer-name-face
> ! 			nil list-matching-lines-face
> ! 			(not (eq occur-excluded-properties t)))))
> ! 	    (let* ((bufcount (length active-bufs))
> ! 		   (diff (- (length bufs) bufcount)))
> ! 	      (message "Searched %d buffer%s%s; %s match%s for `%s'"
> ! 		       bufcount (if (= bufcount 1) "" "s")
> ! 		       (if (zerop diff) "" (format " (%d killed)" diff))
> ! 		       (if (zerop count) "no" (format "%d" count))
> ! 		       (if (= count 1) "" "es")
> ! 		       regexp))
> ! 	    (setq occur-revert-arguments (list regexp nlines bufs))
> ! 	    (if (= count 0)
> ! 		(kill-buffer occur-buf)
> ! 	      (display-buffer occur-buf)
> ! 	      (setq next-error-last-buffer occur-buf)
> ! 	      (setq buffer-read-only t)
> ! 	      (set-buffer-modified-p nil)
> ! 	      (run-hooks 'occur-hook))))))))

The "if collect" test should probably not be placed so far out: the
`collect' case should output the same message, run the same hooks, etc
as the non-collect case.


        Stefan

Re: collect-string

[Tak Ota]

Thanks Stefan.

Thu, 2 Dec 2010 06:16:16 -0800: Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> >> > Now I am convinced.  How about the change below?
> >> Could you send it as a context patch?
> > How about this?
> 
> IIUC the new arg `collect' overrides `lines' in the sense that when
> collect is non-nil lines is simply unused.  So I suggest to just use
> a single argument instead (`lines') and simply give additional semantics
> to values that weren't supported until now.
> E.g. if (numberp lines) then do as before, but if (consp lines) then
> only keep the actual match, where (car lines) could specify which
> submatch to keep.

Very good point.  It made the change much cleaner.

> 
> >     (list (read-regexp "List lines matching regexp"
> >   		     (car regexp-history))
> >   	(when current-prefix-arg
> > ! 	  (prefix-numeric-value current-prefix-arg))
> > ! 	(and current-prefix-arg
> > ! 	     (listp current-prefix-arg)
> > ! 	     (setq last-collect-number
> > ! 		   (read-number "Collect" last-collect-number)))))
> 
> I recommend you first check (with regexp-opt-depth) whether there is
> a submatch, and if not, don't `read-number'.  It's likely to be a very
> common case.

Another excellent suggestion.

> 
> > !     (if collect
> > ! 	(with-current-buffer occur-buf
> > ! 	  (fundamental-mode)
> > ! 	  (let ((inhibit-read-only t)
> > ! 		(buffer-undo-list t))
> > ! 	    (erase-buffer)
> > ! 	    (while active-bufs
> > ! 	      (with-current-buffer (car active-bufs)
> > ! 		(save-excursion
> > ! 		  (goto-char (point-min))
> > ! 		  (while (re-search-forward regexp nil t)
> > ! 		    (let ((str (match-string collect)))
> > ! 		      (if str
> > ! 			  (with-current-buffer occur-buf
> > ! 			    (insert str)
> > ! 			    (or (zerop (current-column))
> > ! 				(insert "\n"))))))))
> > ! 	      (setq active-bufs (cdr active-bufs))))
> > ! 	  (display-buffer occur-buf))
> > !       (with-current-buffer occur-buf
> > ! 	(occur-mode)
> > ! 	(let ((inhibit-read-only t)
> > ! 	      ;; Don't generate undo entries for creation of the initial contents.
> > ! 	      (buffer-undo-list t))
> > ! 	  (erase-buffer)
> > ! 	  (let ((count (occur-engine
> > ! 			regexp active-bufs occur-buf
> > ! 			(or nlines list-matching-lines-default-context-lines)
> > ! 			(if (and case-fold-search search-upper-case)
> > ! 			    (isearch-no-upper-case-p regexp t)
> > ! 			  case-fold-search)
> > ! 			list-matching-lines-buffer-name-face
> > ! 			nil list-matching-lines-face
> > ! 			(not (eq occur-excluded-properties t)))))
> > ! 	    (let* ((bufcount (length active-bufs))
> > ! 		   (diff (- (length bufs) bufcount)))
> > ! 	      (message "Searched %d buffer%s%s; %s match%s for `%s'"
> > ! 		       bufcount (if (= bufcount 1) "" "s")
> > ! 		       (if (zerop diff) "" (format " (%d killed)" diff))
> > ! 		       (if (zerop count) "no" (format "%d" count))
> > ! 		       (if (= count 1) "" "es")
> > ! 		       regexp))
> > ! 	    (setq occur-revert-arguments (list regexp nlines bufs))
> > ! 	    (if (= count 0)
> > ! 		(kill-buffer occur-buf)
> > ! 	      (display-buffer occur-buf)
> > ! 	      (setq next-error-last-buffer occur-buf)
> > ! 	      (setq buffer-read-only t)
> > ! 	      (set-buffer-modified-p nil)
> > ! 	      (run-hooks 'occur-hook))))))))
> 
> The "if collect" test should probably not be placed so far out: the
> `collect' case should output the same message, run the same hooks, etc
> as the non-collect case.

How about this?  See below.

-Tak

*** ../../../d/pub/emacs/emacs-23.2.90/lisp/replace.el	Thu Dec  2 16:33:19 2010
--- replace.el	Thu Dec  2 17:00:42 2010
***************
*** 527,532 ****
--- 527,536 ----
  Maximum length of the history list is determined by the value
  of `history-length', which see.")
  
+ (defvar occur-collect-submatch-history '("1")
+   "The history of list of parenthesized expression numbers as a
+   string in occur's collect operation")
+ 
  (defun read-regexp (prompt &optional default-value)
    "Read regexp as a string using the regexp history and some useful defaults.
  Prompt for a regular expression with PROMPT (without a colon and
***************
*** 1028,1037 ****
        (nreverse result))))
  
  (defun occur-read-primary-args ()
!   (list (read-regexp "List lines matching regexp"
! 		     (car regexp-history))
! 	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
--- 1032,1058 ----
        (nreverse result))))
  
  (defun occur-read-primary-args ()
!   (let ((regexp)
! 	(do-collect (consp current-prefix-arg)))
!     (list (setq regexp
! 		(read-regexp (if do-collect
! 				 "Collect strings matching regexp"
! 			       "List lines matching regexp")
! 			     (car regexp-history)))
! 	  (if do-collect
! 	      (if (zerop (regexp-opt-depth regexp))
! 		  ;; no subexpression so collect entire the match
! 		  '(0)
! 		;; construct a list of subexpression integers
! 		(mapcar 'string-to-number
! 			(split-string
! 			 (let ((default (car occur-collect-submatch-history)))
! 			   (read-from-minibuffer
! 			    (format "Subexpressions to collect (default %s): " default)
! 			    "" nil nil 'occur-collect-submatch-history default)
! 			   (car occur-collect-submatch-history)))))
! 	    (when current-prefix-arg
! 	      (prefix-numeric-value current-prefix-arg))))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
***************
*** 1064,1070 ****
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive."
    (interactive (occur-read-primary-args))
    (occur-1 regexp nlines (list (current-buffer))))
  
--- 1085,1101 ----
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive.
! 
! When NLINES is a list of integers or when the function is called
! interactively with prefix argument without a number (`C-u' alone
! as prefix) the matching strings are collected into the `*Occur*'
! buffer.  The parenthesized match strings in REGEXP indicated by
! the integers in NLINES are collected.  For example, providing
! \"defun \\(\\S +\\)\" for REGEXP and (1) for NLINES collects all
! the function names in a lisp program.  When there is no
! parenthesized subexpressions in REGEXP the entire match is
! collected."
    (interactive (occur-read-primary-args))
    (occur-1 regexp nlines (list (current-buffer))))
  
***************
*** 1146,1165 ****
      (setq occur-buf (get-buffer-create buf-name))
  
      (with-current-buffer occur-buf
!       (occur-mode)
        (let ((inhibit-read-only t)
  	    ;; Don't generate undo entries for creation of the initial contents.
  	    (buffer-undo-list t))
  	(erase-buffer)
! 	(let ((count (occur-engine
! 		      regexp active-bufs occur-buf
! 		      (or nlines list-matching-lines-default-context-lines)
! 		      (if (and case-fold-search search-upper-case)
! 			  (isearch-no-upper-case-p regexp t)
! 			case-fold-search)
! 		      list-matching-lines-buffer-name-face
! 		      nil list-matching-lines-face
! 		      (not (eq occur-excluded-properties t)))))
  	  (let* ((bufcount (length active-bufs))
  		 (diff (- (length bufs) bufcount)))
  	    (message "Searched %d buffer%s%s; %s match%s for `%s'"
--- 1177,1226 ----
      (setq occur-buf (get-buffer-create buf-name))
  
      (with-current-buffer occur-buf
!       (if (consp nlines)
! 	  (fundamental-mode) ;; for collect opeartion
! 	(occur-mode))
        (let ((inhibit-read-only t)
  	    ;; Don't generate undo entries for creation of the initial contents.
  	    (buffer-undo-list t))
  	(erase-buffer)
! 	(let ((count
! 	       (if (consp nlines)
!                    ;; nlines is a list of subexpression integers to collect
! 		   (let ((bufs active-bufs)
! 			 (count 0))
! 		     (while bufs
! 		       (with-current-buffer (car bufs)
! 			 (save-excursion
! 			   (goto-char (point-min))
! 			   (while (re-search-forward regexp nil t)
!                              ;; insert the mached subexpressions in a line
! 			     (let ((first-subexp t))
! 			       (mapc
! 				(lambda (subexp)
! 				  (let ((str (match-string subexp)))
! 				    (if str
! 					(with-current-buffer occur-buf
! 					  (if first-subexp
! 					      (setq first-subexp nil)
! 					    (insert " ")) ;; delimiter
! 					  (insert str)))))
! 				nlines)
! 			       (setq count (1+ count)))
! 			     (with-current-buffer occur-buf
! 			       (or (zerop (current-column))
! 				   (insert "\n"))))))
!                        (setq bufs (cdr bufs)))
!                      count)
! 		 (occur-engine
! 		  regexp active-bufs occur-buf
! 		  (or nlines list-matching-lines-default-context-lines)
! 		  (if (and case-fold-search search-upper-case)
! 		      (isearch-no-upper-case-p regexp t)
! 		    case-fold-search)
! 		  list-matching-lines-buffer-name-face
! 		  nil list-matching-lines-face
! 		  (not (eq occur-excluded-properties t))))))
  	  (let* ((bufcount (length active-bufs))
  		 (diff (- (length bufs) bufcount)))
  	    (message "Searched %d buffer%s%s; %s match%s for `%s'"

Re: collect-string

[Stefan Monnier]

Thanks, that looks pretty good now.  A few more nitpicks while I'm here:

> + (defvar occur-collect-submatch-history '("1")
> +   "The history of list of parenthesized expression numbers as a
> +   string in occur's collect operation")

Usually history vars start at nil (and you can save them via savehist
if you so prefer).

>   (defun occur-read-primary-args ()
> !   (let ((regexp)
> ! 	(do-collect (consp current-prefix-arg)))
> !     (list (setq regexp
> ! 		(read-regexp (if do-collect
> ! 				 "Collect strings matching regexp"
> ! 			       "List lines matching regexp")
> ! 			     (car regexp-history)))

Always try to set the var directly in the let rather than via
a separate setq:

  (let* ((do-collect (consp current-prefix-arg))
         (regexp (read-regexp (if do-collect
                                  "Collect strings matching regexp"
                                "List lines matching regexp")
                              (car regexp-history))))
    (list regexp ...))

> ! 	  (if do-collect
> ! 	      (if (zerop (regexp-opt-depth regexp))
> ! 		  ;; no subexpression so collect entire the match

Comments should start with a capital letter and end with a ".".

> ! 		  '(0)
> ! 		;; construct a list of subexpression integers
> ! 		(mapcar 'string-to-number
> ! 			(split-string
> ! 			 (let ((default (car occur-collect-submatch-history)))
> ! 			   (read-from-minibuffer
> ! 			    (format "Subexpressions to collect (default %s): " default)
> ! 			    "" nil nil 'occur-collect-submatch-history default)
> ! 			   (car occur-collect-submatch-history)))))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This should be `default', right?

> ! When NLINES is a list of integers or when the function is called
> ! interactively with prefix argument without a number (`C-u' alone
> ! as prefix) the matching strings are collected into the `*Occur*'
> ! buffer.  The parenthesized match strings in REGEXP indicated by
> ! the integers in NLINES are collected.  For example, providing
> ! \"defun \\(\\S +\\)\" for REGEXP and (1) for NLINES collects all
> ! the function names in a lisp program.  When there is no
> ! parenthesized subexpressions in REGEXP the entire match is
> ! collected."

Hmm... I didn't think of NLINES turning into a list of submatch-numbers,
but that makes me think that maybe we should go one step further and
turn it into a string, like the ones used in replace-match.


        Stefan

Re: collect-string

[Tak Ota]

Stefan,

Thanks for your quick review.

Fri, 3 Dec 2010 11:17:50 -0800: Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> Thanks, that looks pretty good now.  A few more nitpicks while I'm here:
> 
> > + (defvar occur-collect-submatch-history '("1")
> > +   "The history of list of parenthesized expression numbers as a
> > +   string in occur's collect operation")
> 
> Usually history vars start at nil (and you can save them via savehist
> if you so prefer).

If a user takes the default, which is the first item in the history,
it makes no sense.  This value is asked only when there is at least
one subexpression so I thought having something would suggest the
usage to the user.

> 
> >   (defun occur-read-primary-args ()
> > !   (let ((regexp)
> > ! 	(do-collect (consp current-prefix-arg)))
> > !     (list (setq regexp
> > ! 		(read-regexp (if do-collect
> > ! 				 "Collect strings matching regexp"
> > ! 			       "List lines matching regexp")
> > ! 			     (car regexp-history)))
> 
> Always try to set the var directly in the let rather than via
> a separate setq:
> 
>   (let* ((do-collect (consp current-prefix-arg))
>          (regexp (read-regexp (if do-collect
>                                   "Collect strings matching regexp"
>                                 "List lines matching regexp")
>                               (car regexp-history))))
>     (list regexp ...))

This I realized after sent out the message and made exactly same
change as you suggest.

> 
> > ! 	  (if do-collect
> > ! 	      (if (zerop (regexp-opt-depth regexp))
> > ! 		  ;; no subexpression so collect entire the match
> 
> Comments should start with a capital letter and end with a ".".

You are right.

> 
> > ! 		  '(0)
> > ! 		;; construct a list of subexpression integers
> > ! 		(mapcar 'string-to-number
> > ! 			(split-string
> > ! 			 (let ((default (car occur-collect-submatch-history)))
> > ! 			   (read-from-minibuffer
> > ! 			    (format "Subexpressions to collect (default %s): " default)
> > ! 			    "" nil nil 'occur-collect-submatch-history default)
> > ! 			   (car occur-collect-submatch-history)))))
>                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> This should be `default', right?

No, this is the new input value as read-from-minibuffer prepends this
value to the history.

> > ! When NLINES is a list of integers or when the function is called
> > ! interactively with prefix argument without a number (`C-u' alone
> > ! as prefix) the matching strings are collected into the `*Occur*'
> > ! buffer.  The parenthesized match strings in REGEXP indicated by
> > ! the integers in NLINES are collected.  For example, providing
> > ! \"defun \\(\\S +\\)\" for REGEXP and (1) for NLINES collects all
> > ! the function names in a lisp program.  When there is no
> > ! parenthesized subexpressions in REGEXP the entire match is
> > ! collected."
> 
> Hmm... I didn't think of NLINES turning into a list of submatch-numbers,
> but that makes me think that maybe we should go one step further and
> turn it into a string, like the ones used in replace-match.

This is truly brilliant.  Thank you very much.  I love this idea
bouncing process.  See below.

-Tak

*** ../../../d/pub/emacs/emacs-23.2.90/lisp/replace.el	Thu Dec  2 16:33:19 2010
--- replace.el	Fri Dec  3 14:28:27 2010
***************
*** 527,532 ****
--- 527,535 ----
  Maximum length of the history list is determined by the value
  of `history-length', which see.")
  
+ (defvar occur-collect-regexp-history '("\\1")
+   "History of regexp for occur's collect operation")
+ 
  (defun read-regexp (prompt &optional default-value)
    "Read regexp as a string using the regexp history and some useful defaults.
  Prompt for a regular expression with PROMPT (without a colon and
***************
*** 1028,1037 ****
        (nreverse result))))
  
  (defun occur-read-primary-args ()
!   (list (read-regexp "List lines matching regexp"
! 		     (car regexp-history))
! 	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
--- 1031,1056 ----
        (nreverse result))))
  
  (defun occur-read-primary-args ()
!   (let* ((perform-collect (consp current-prefix-arg))
!          (regexp (read-regexp (if perform-collect
!                                   "Collect strings matching regexp"
!                                 "List lines matching regexp")
!                               (car regexp-history))))
!     (list regexp
! 	  (if perform-collect
! 	      ;; Perform collect operation
! 	      (if (zerop (regexp-opt-depth regexp))
! 		  ;; No subexpression so collect the entire match.
! 		  "\\&"
! 		;; Get the regexp for collection pattern.
! 		(let ((default (car occur-collect-regexp-history)))
! 		  (read-from-minibuffer
! 		   (format "Regexp to collect (default %s): " default)
! 		   "" nil nil 'occur-collect-regexp-history default)
! 		  (car occur-collect-regexp-history)))
! 	    ;; Otherwise normal occur takes numerical prefix argument.
! 	    (when current-prefix-arg
! 	      (prefix-numeric-value current-prefix-arg))))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
***************
*** 1064,1070 ****
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive."
    (interactive (occur-read-primary-args))
    (occur-1 regexp nlines (list (current-buffer))))
  
--- 1083,1100 ----
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive.
! 
! When NLINES is a string or when the function is called
! interactively with prefix argument without a number (`C-u' alone
! as prefix) the matching strings are collected into the `*Occur*'
! buffer by using NLINES as a replacement regexp.  NLINES may
! contain \\& and \\N which convention follows `replace-match'.
! For example, providing \"defun\\s +\\(\\S +\\)\" for REGEXP and
! \"\\1\" for NLINES collects all the function names in a lisp
! program.  When there is no parenthesized subexpressions in REGEXP
! the entire match is collected.  In any case the searched buffers
! are not modified."
    (interactive (occur-read-primary-args))
    (occur-1 regexp nlines (list (current-buffer))))
  
***************
*** 1146,1165 ****
      (setq occur-buf (get-buffer-create buf-name))
  
      (with-current-buffer occur-buf
!       (occur-mode)
        (let ((inhibit-read-only t)
  	    ;; Don't generate undo entries for creation of the initial contents.
  	    (buffer-undo-list t))
  	(erase-buffer)
! 	(let ((count (occur-engine
! 		      regexp active-bufs occur-buf
! 		      (or nlines list-matching-lines-default-context-lines)
! 		      (if (and case-fold-search search-upper-case)
! 			  (isearch-no-upper-case-p regexp t)
! 			case-fold-search)
! 		      list-matching-lines-buffer-name-face
! 		      nil list-matching-lines-face
! 		      (not (eq occur-excluded-properties t)))))
  	  (let* ((bufcount (length active-bufs))
  		 (diff (- (length bufs) bufcount)))
  	    (message "Searched %d buffer%s%s; %s match%s for `%s'"
--- 1176,1218 ----
      (setq occur-buf (get-buffer-create buf-name))
  
      (with-current-buffer occur-buf
!       (if (stringp nlines)
! 	  (fundamental-mode) ;; This is for collect opeartion.
! 	(occur-mode))
        (let ((inhibit-read-only t)
  	    ;; Don't generate undo entries for creation of the initial contents.
  	    (buffer-undo-list t))
  	(erase-buffer)
! 	(let ((count
! 	       (if (stringp nlines)
!                    ;; Treat nlines as a regexp to collect.
! 		   (let ((bufs active-bufs)
! 			 (count 0))
! 		     (while bufs
! 		       (with-current-buffer (car bufs)
! 			 (save-excursion
! 			   (goto-char (point-min))
! 			   (while (re-search-forward regexp nil t)
!                              ;; Insert the replacement regexp.
! 			     (let ((str (match-substitute-replacement nlines)))
! 			       (if str
! 				   (with-current-buffer occur-buf
! 				     (insert str)
! 				     (setq count (1+ count))
! 				     (or (zerop (current-column))
! 					 (insert "\n"))))))))
!                        (setq bufs (cdr bufs)))
!                      count)
! 		 ;; Perform normal occur.
! 		 (occur-engine
! 		  regexp active-bufs occur-buf
! 		  (or nlines list-matching-lines-default-context-lines)
! 		  (if (and case-fold-search search-upper-case)
! 		      (isearch-no-upper-case-p regexp t)
! 		    case-fold-search)
! 		  list-matching-lines-buffer-name-face
! 		  nil list-matching-lines-face
! 		  (not (eq occur-excluded-properties t))))))
  	  (let* ((bufcount (length active-bufs))
  		 (diff (- (length bufs) bufcount)))
  	    (message "Searched %d buffer%s%s; %s match%s for `%s'"

Re: collect-string

[Davis Herring]

> ! 		  ;; No subexpression so collect the entire match.
> ! 		  "\\&"

Once we have the replace-match functionality, I don't think this
assumption makes sense anymore; the user might want to frame the matches
with something.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: collect-string

[Tak Ota]

I thought about that too.  But this may be the most common usage
without answering one extra question.  If the user wants to decorate
the entire match the regexp can be a single big subexpression.
"\\(...\\)"

-Tak

Fri, 3 Dec 2010 14:40:47 -0800: Davis Herring <herring@lanl.gov> wrote:

> > ! 		  ;; No subexpression so collect the entire match.
> > ! 		  "\\&"
> 
> Once we have the replace-match functionality, I don't think this
> assumption makes sense anymore; the user might want to frame the matches
> with something.
> 
> Davis
> 
> -- 
> This product is sold by volume, not by mass.  If it appears too dense or
> too sparse, it is because mass-energy conversion has occurred during
> shipping.
> 

Re: collect-string

[Stefan Monnier]

>> > ! 		  '(0)
>> > ! 		;; construct a list of subexpression integers
>> > ! 		(mapcar 'string-to-number
>> > ! 			(split-string
>> > ! 			 (let ((default (car occur-collect-submatch-history)))
>> > ! 			   (read-from-minibuffer
>> > ! 			    (format "Subexpressions to collect (default %s): " default)
>> > ! 			    "" nil nil 'occur-collect-submatch-history default)
>> > ! 			   (car occur-collect-submatch-history)))))
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> 
>> This should be `default', right?

> No, this is the new input value as read-from-minibuffer prepends this
> value to the history.

Oh, right, there's more parens than I thought.  You should use
`read-string' instead, which will take care of this ugliness.

>> Hmm... I didn't think of NLINES turning into a list of submatch-numbers,
>> but that makes me think that maybe we should go one step further and
>> turn it into a string, like the ones used in replace-match.
> This is truly brilliant.  Thank you very much.  I love this idea
> bouncing process.  See below.

It's not always as successful, sadly.  But yes, it turned out very well
this time.


        Stefan

Re: collect-string

[Tak Ota]

Fri, 3 Dec 2010 14:56:51 -0800: Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> >> > ! 		  '(0)
> >> > ! 		;; construct a list of subexpression integers
> >> > ! 		(mapcar 'string-to-number
> >> > ! 			(split-string
> >> > ! 			 (let ((default (car occur-collect-submatch-history)))
> >> > ! 			   (read-from-minibuffer
> >> > ! 			    (format "Subexpressions to collect (default %s): " default)
> >> > ! 			    "" nil nil 'occur-collect-submatch-history default)
> >> > ! 			   (car occur-collect-submatch-history)))))
> >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >> 
> >> This should be `default', right?
> 
> > No, this is the new input value as read-from-minibuffer prepends this
> > value to the history.
> 
> Oh, right, there's more parens than I thought.  You should use
> `read-string' instead, which will take care of this ugliness.

OK, done.  Indeed its cleaner.

*** ../../../d/pub/emacs/emacs-23.2.90/lisp/replace.el	Thu Dec  2 16:33:19 2010
--- replace.el	Fri Dec  3 15:07:56 2010
***************
*** 527,532 ****
--- 527,535 ----
  Maximum length of the history list is determined by the value
  of `history-length', which see.")
  
+ (defvar occur-collect-regexp-history '("\\1")
+   "History of regexp for occur's collect operation")
+ 
  (defun read-regexp (prompt &optional default-value)
    "Read regexp as a string using the regexp history and some useful defaults.
  Prompt for a regular expression with PROMPT (without a colon and
***************
*** 1028,1037 ****
        (nreverse result))))
  
  (defun occur-read-primary-args ()
!   (list (read-regexp "List lines matching regexp"
! 		     (car regexp-history))
! 	(when current-prefix-arg
! 	  (prefix-numeric-value current-prefix-arg))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
--- 1031,1055 ----
        (nreverse result))))
  
  (defun occur-read-primary-args ()
!   (let* ((perform-collect (consp current-prefix-arg))
!          (regexp (read-regexp (if perform-collect
!                                   "Collect strings matching regexp"
!                                 "List lines matching regexp")
!                               (car regexp-history))))
!     (list regexp
! 	  (if perform-collect
! 	      ;; Perform collect operation
! 	      (if (zerop (regexp-opt-depth regexp))
! 		  ;; No subexpression so collect the entire match.
! 		  "\\&"
! 		;; Get the regexp for collection pattern.
! 		(let ((default (car occur-collect-regexp-history)))
! 		  (read-string
! 		   (format "Regexp to collect (default %s): " default)
! 		   nil 'occur-collect-regexp-history default)))
! 	    ;; Otherwise normal occur takes numerical prefix argument.
! 	    (when current-prefix-arg
! 	      (prefix-numeric-value current-prefix-arg))))))
  
  (defun occur-rename-buffer (&optional unique-p interactive-p)
    "Rename the current *Occur* buffer to *Occur: original-buffer-name*.
***************
*** 1064,1070 ****
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive."
    (interactive (occur-read-primary-args))
    (occur-1 regexp nlines (list (current-buffer))))
  
--- 1082,1099 ----
  \\<occur-mode-map>\\[describe-mode] in that buffer will explain how.
  
  If REGEXP contains upper case characters (excluding those preceded by `\\')
! and `search-upper-case' is non-nil, the matching is case-sensitive.
! 
! When NLINES is a string or when the function is called
! interactively with prefix argument without a number (`C-u' alone
! as prefix) the matching strings are collected into the `*Occur*'
! buffer by using NLINES as a replacement regexp.  NLINES may
! contain \\& and \\N which convention follows `replace-match'.
! For example, providing \"defun\\s +\\(\\S +\\)\" for REGEXP and
! \"\\1\" for NLINES collects all the function names in a lisp
! program.  When there is no parenthesized subexpressions in REGEXP
! the entire match is collected.  In any case the searched buffers
! are not modified."
    (interactive (occur-read-primary-args))
    (occur-1 regexp nlines (list (current-buffer))))
  
***************
*** 1146,1165 ****
      (setq occur-buf (get-buffer-create buf-name))
  
      (with-current-buffer occur-buf
!       (occur-mode)
        (let ((inhibit-read-only t)
  	    ;; Don't generate undo entries for creation of the initial contents.
  	    (buffer-undo-list t))
  	(erase-buffer)
! 	(let ((count (occur-engine
! 		      regexp active-bufs occur-buf
! 		      (or nlines list-matching-lines-default-context-lines)
! 		      (if (and case-fold-search search-upper-case)
! 			  (isearch-no-upper-case-p regexp t)
! 			case-fold-search)
! 		      list-matching-lines-buffer-name-face
! 		      nil list-matching-lines-face
! 		      (not (eq occur-excluded-properties t)))))
  	  (let* ((bufcount (length active-bufs))
  		 (diff (- (length bufs) bufcount)))
  	    (message "Searched %d buffer%s%s; %s match%s for `%s'"
--- 1175,1217 ----
      (setq occur-buf (get-buffer-create buf-name))
  
      (with-current-buffer occur-buf
!       (if (stringp nlines)
! 	  (fundamental-mode) ;; This is for collect opeartion.
! 	(occur-mode))
        (let ((inhibit-read-only t)
  	    ;; Don't generate undo entries for creation of the initial contents.
  	    (buffer-undo-list t))
  	(erase-buffer)
! 	(let ((count
! 	       (if (stringp nlines)
!                    ;; Treat nlines as a regexp to collect.
! 		   (let ((bufs active-bufs)
! 			 (count 0))
! 		     (while bufs
! 		       (with-current-buffer (car bufs)
! 			 (save-excursion
! 			   (goto-char (point-min))
! 			   (while (re-search-forward regexp nil t)
!                              ;; Insert the replacement regexp.
! 			     (let ((str (match-substitute-replacement nlines)))
! 			       (if str
! 				   (with-current-buffer occur-buf
! 				     (insert str)
! 				     (setq count (1+ count))
! 				     (or (zerop (current-column))
! 					 (insert "\n"))))))))
!                        (setq bufs (cdr bufs)))
!                      count)
! 		 ;; Perform normal occur.
! 		 (occur-engine
! 		  regexp active-bufs occur-buf
! 		  (or nlines list-matching-lines-default-context-lines)
! 		  (if (and case-fold-search search-upper-case)
! 		      (isearch-no-upper-case-p regexp t)
! 		    case-fold-search)
! 		  list-matching-lines-buffer-name-face
! 		  nil list-matching-lines-face
! 		  (not (eq occur-excluded-properties t))))))
  	  (let* ((bufcount (length active-bufs))
  		 (diff (- (length bufs) bufcount)))
  	    (message "Searched %d buffer%s%s; %s match%s for `%s'"


> >> Hmm... I didn't think of NLINES turning into a list of submatch-numbers,
> >> but that makes me think that maybe we should go one step further and
> >> turn it into a string, like the ones used in replace-match.
> > This is truly brilliant.  Thank you very much.  I love this idea
> > bouncing process.  See below.
> 
> It's not always as successful, sadly.  But yes, it turned out very well
> this time.

As long as people behave rationally with patience and not carried too
much away by own belief it should work fine most of the time.  I'm
glad I haven't been involved in sad cases.

-Tak

Re: collect-string

[Stefan Monnier]

> OK, done.  Indeed its cleaner.

Thanks, installed in the trunk.

The next step would probably be to try and handle the remaining
differences between normal occur behavior and the new "collect"
behavior, mostly the presence of line numbers.  Maybe we could try and
move the line numbers out of the buffer's content into the margin to
help resolve this problem.


        Stefan

Re: collect-string

[Tak Ota]

Fri, 3 Dec 2010 18:01:01 -0800: Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> > OK, done.  Indeed its cleaner.
> 
> Thanks, installed in the trunk.

Thank you.

> The next step would probably be to try and handle the remaining
> differences between normal occur behavior and the new "collect"
> behavior, mostly the presence of line numbers.  Maybe we could try and
> move the line numbers out of the buffer's content into the margin to
> help resolve this problem.

For now I'll leave that work to someone who is more familiar with how
occur works than I :)

-Tak

Re: collect-string

[Glenn Morris]

Stefan Monnier wrote:

> Oh, right, there's more parens than I thought.

Out of context, I think this is a great quotation. :)

Re: simple useful functions

[Andreas Schwab]

Tak Ota <Takaaki.Ota@am.sony.com> writes:

>     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))

That lacks proper quoting.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: simple useful functions

[Andreas Röhler]

Am 28.10.2010 20:56, schrieb Tak Ota:
> If you think the following two functions are universally useful please
> consider incorporating them in simple.el or any appropriate package.
> If not disregard.
>
> -Tak
>
> (defun collect-string (regexp&optional num)
>    "Collect strings of REGEXP (or optional NUM paren) from the
> current buffer into a collection buffer."
>    (interactive "sCollect string (regexp): \nP")
>    (let ((collection-buffer
> 	 (get-buffer-create (format "*Collection of \"%s\" *" regexp))))
>      (with-current-buffer collection-buffer (erase-buffer))
>      (save-excursion
>        (goto-char (point-min))
>        (while (re-search-forward regexp nil t)
> 	(let ((str (match-string (or num 0))))
> 	  (if str
> 	      (with-current-buffer collection-buffer
> 		(insert str)
> 		(or (zerop (current-column))
> 		    (insert "\n")))))))
>      (pop-to-buffer collection-buffer)
>      (goto-char (point-min))))
>
> (defun source (script&optional shell keep-current-directory)
>    "Source the specified shell script.
> Source the shell SCRIPT and import the environment into this
> emacs.  The optional SHELL specifies the shell other than the
> default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
> which is the default, the current directory is temporarily
> changed to the directory where the script resides while sourcing
> the script."
>    (interactive "fscript file: ")
>    (if (null shell)
>        (setq shell shell-file-name))
>    (with-temp-buffer
>      (unless keep-current-directory
>        (setq default-directory (file-name-directory script)))
>      (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
>      (while (re-search-backward "^\\([^=]+\\)=\\(.*\\)$" nil t)
>        (setenv (match-string 1) (match-string 2)))))
>
>
>

Hi Tak,

thanks a lot.

Took your stuff into my personal collection misc-utils.el at

https://code.launchpad.net/s-x-emacs-werkstatt/

also into a git-repo collecting interesting stuff

http://repo.or.cz/w/elbb.git

Andreas

--
https://code.launchpad.net/~a-roehler/python-mode/python-mode-components

Re: simple useful functions

[Tak Ota]

Now `collect-string' seems to have found an appropriate place to fit
in `occur' can we revisit `source' command?  It would be useful if
this is a part of shell.el or somewhere with more appropriate name.

-Tak

Thu, 28 Oct 2010 11:56:15 -0700: Tak Ota <Takaaki.Ota@am.sony.com> wrote:

> If you think the following two functions are universally useful please
> consider incorporating them in simple.el or any appropriate package.
> If not disregard.
> 
> -Tak
> 
> (defun collect-string (regexp &optional num)
>   "Collect strings of REGEXP (or optional NUM paren) from the
> current buffer into a collection buffer."
>   (interactive "sCollect string (regexp): \nP")
>   (let ((collection-buffer
> 	 (get-buffer-create (format "*Collection of \"%s\" *" regexp))))
>     (with-current-buffer collection-buffer (erase-buffer))
>     (save-excursion
>       (goto-char (point-min))
>       (while (re-search-forward regexp nil t)
> 	(let ((str (match-string (or num 0))))
> 	  (if str
> 	      (with-current-buffer collection-buffer
> 		(insert str)
> 		(or (zerop (current-column))
> 		    (insert "\n")))))))
>     (pop-to-buffer collection-buffer)
>     (goto-char (point-min))))
> 
> (defun source (script &optional shell keep-current-directory)
>   "Source the specified shell script.
> Source the shell SCRIPT and import the environment into this
> emacs.  The optional SHELL specifies the shell other than the
> default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
> which is the default, the current directory is temporarily
> changed to the directory where the script resides while sourcing
> the script."
>   (interactive "fscript file: ")
>   (if (null shell)
>       (setq shell shell-file-name))
>   (with-temp-buffer
>     (unless keep-current-directory
>       (setq default-directory (file-name-directory script)))
>     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
>     (while (re-search-backward "^\\([^=]+\\)=\\(.*\\)$" nil t)
>       (setenv (match-string 1) (match-string 2)))))

Re: simple useful functions

[Stefan Monnier]

> (defun source (script &optional shell keep-current-directory)
>   "Source the specified shell script.
> Source the shell SCRIPT and import the environment into this
> emacs.  The optional SHELL specifies the shell other than the
> default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
> which is the default, the current directory is temporarily
> changed to the directory where the script resides while sourcing
> the script."
>   (interactive "fscript file: ")
>   (if (null shell)
>       (setq shell shell-file-name))
>   (with-temp-buffer
>     (unless keep-current-directory
>       (setq default-directory (file-name-directory script)))
>     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
>     (while (re-search-backward "^\\([^=]+\\)=\\(.*\\)$" nil t)
>       (setenv (match-string 1) (match-string 2)))))

Some problems I see with it, by order of decreasing importance:
- I fail to see in which circumstance(s) this is useful.
- What if the user uses scsh/emacs for his shell.
- Most scripts don't setup much of environment.
- AFAIC things like ~/.cshrc are not a scripts but a config files.
- The call-process needs to quote `script' (and should probably be
  replaced by process-file).
- The prompt should start with a capital.


        Stefan

Re: simple useful functions

[Tak Ota]

Fri, 3 Dec 2010 18:36:31 -0800: Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> > (defun source (script &optional shell keep-current-directory)
> >   "Source the specified shell script.
> > Source the shell SCRIPT and import the environment into this
> > emacs.  The optional SHELL specifies the shell other than the
> > default `shell-file-name'.  When KEEP-CURRENT-DIRECTORY is nil,
> > which is the default, the current directory is temporarily
> > changed to the directory where the script resides while sourcing
> > the script."
> >   (interactive "fscript file: ")
> >   (if (null shell)
> >       (setq shell shell-file-name))
> >   (with-temp-buffer
> >     (unless keep-current-directory
> >       (setq default-directory (file-name-directory script)))
> >     (call-process shell nil t nil "-c" (concat "source " script "; printenv"))
> >     (while (re-search-backward "^\\([^=]+\\)=\\(.*\\)$" nil t)
> >       (setenv (match-string 1) (match-string 2)))))
> 
> Some problems I see with it, by order of decreasing importance:
> - I fail to see in which circumstance(s) this is useful.

OK, I see this may not be useful for everybody.  The reason I wrote
this command was to support embedded software development.  In
embedded development work we are usually required to use some cross
compilation tools.  Such tools often come with a setup script which
includes setting up necessary environment variables for command paths
and library paths.  In order to compile software in this environment
from emacs I need to start emacs from the shell that has run the
script.  But to me emacs is not a text editor but a working
environment.  I don't want to kill the current emacs or start another
emacs for this purpose.  My emacs has been running before running the
script.  I take care of shell needs from inferior shell.  But running
the setup script from an inferior shell doesn't help.  Therefore this
command became handy.

After writing this up I now realize it is useful for minority groups.
Thanks for the comments anyway.

-Tak

> - What if the user uses scsh/emacs for his shell.
> - Most scripts don't setup much of environment.
> - AFAIC things like ~/.cshrc are not a scripts but a config files.
> - The call-process needs to quote `script' (and should probably be
>   replaced by process-file).
> - The prompt should start with a capital.
> 
> 
>         Stefan
> 

Re: simple useful functions

[Stefan Monnier]

>> - I fail to see in which circumstance(s) this is useful.
> OK, I see this may not be useful for everybody.

That's not necessarily a problem, tho.  Most packages in Emacs are not
useful for everybody, after all.  Still, it needs to be useful to more
than a handful; and, more importantly, it should provide a clear
functionality in a "generic" way (this is what is likely to make it
useful to more than a handful of users).

> The reason I wrote this command was to support embedded software
> development.  In embedded development work we are usually required to
> use some cross compilation tools.  Such tools often come with a setup
> script which includes setting up necessary environment variables for
> command paths and library paths.

IIUC these scripts are written for /bin/sh, right?  How do (t)csh users
handle that?
[ Sorry, I'm not that familiar with cross-compiling: OpenWRT is about
  as far as I got into this, and it "takes care of things" in ways
  I haven't tried to understand.  They don't use such setup scripts, at
  least in a user-visible way (although the user does perform the
  cross-compiling).  ]

> In order to compile software in this environment from Emacs I need to
> start Emacs from the shell that has run the script.  But to me Emacs
> is not a text editor but a working environment.

I partly understand the context.  But right here I think your script
fails to serve you: it uses `setenv' which affects all inferior
processes, so if you're working on several projects at the same time
from the same Emacs session (likely if you're using it as a work
environment, which is also the intended use to a large extent), you get
pollution from one project to the other.  I think this issue is
important, and I suspect that figuring out how it should be solved is
necessary to figure out what is a good/generic way to provide this
"source" feature.


        Stefan

Re: simple useful functions

[David Kastrup]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> IIUC these scripts are written for /bin/sh, right?  How do (t)csh
> users handle that?

They use XEmacs.

SCNR

-- 
David Kastrup

Re: simple useful functions

[René Kyllingstad]

On Sat, Dec 4, 2010 at 5:36 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> I partly understand the context.  But right here I think your script
> fails to serve you: it uses `setenv' which affects all inferior
> processes, so if you're working on several projects at the same time
> from the same Emacs session (likely if you're using it as a work
> environment, which is also the intended use to a large extent), you get
> pollution from one project to the other.  I think this issue is
> important, and I suspect that figuring out how it should be solved is
> necessary to figure out what is a good/generic way to provide this
> "source" feature.

As long as the login script sets up the environment needed you just
need to run a source command when switching between the projects to
override the environment.

A cleaner solution would be to attach the environment to a project
concept within Emacs, but when someone offers a shed it seems unfair
to ask them to build the whole city around it.


-- René

Re: simple useful functions

[Stefan Monnier]

>> I partly understand the context.  But right here I think your script
>> fails to serve you: it uses `setenv' which affects all inferior
>> processes, so if you're working on several projects at the same time
>> from the same Emacs session (likely if you're using it as a work
>> environment, which is also the intended use to a large extent), you get
>> pollution from one project to the other.  I think this issue is
>> important, and I suspect that figuring out how it should be solved is
>> necessary to figure out what is a good/generic way to provide this
>> "source" feature.

> As long as the login script sets up the environment needed you just
> need to run a source command when switching between the projects to
> override the environment.

You may use the various projects "at the same time", so "switching" may
be annoying if not problematic.  Furthermore, sometimes you can't just
accumulate the changes and hope that things still work, so instead you
need to first undo one of the envs and then apply the other env.

> A cleaner solution would be to attach the environment to a project
> concept within Emacs, but when someone offers a shed it seems unfair
> to ask them to build the whole city around it.

If it sounded like I was asking him to write an IDE as a prerequisite
for his change, I'm sorry for it, that was not my intention.  We already
have .dir-local.el which we might be able to use for that: a command
that extracts the env from a setup script and puts the result in some
var in a .dir-local.el; and then tweak things like M-x compile to pay
attention to that var.


        Stefan

Re: simple useful functions

[Tak Ota]

Fri, 3 Dec 2010 20:36:09 -0800: Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> >> - I fail to see in which circumstance(s) this is useful.
> > OK, I see this may not be useful for everybody.
> 
> That's not necessarily a problem, tho.  Most packages in Emacs are not
> useful for everybody, after all.  Still, it needs to be useful to more
> than a handful; and, more importantly, it should provide a clear
> functionality in a "generic" way (this is what is likely to make it
> useful to more than a handful of users).
> 
> > The reason I wrote this command was to support embedded software
> > development.  In embedded development work we are usually required to
> > use some cross compilation tools.  Such tools often come with a setup
> > script which includes setting up necessary environment variables for
> > command paths and library paths.

I understand.

> IIUC these scripts are written for /bin/sh, right?  How do (t)csh users
> handle that?
> [ Sorry, I'm not that familiar with cross-compiling: OpenWRT is about
>   as far as I got into this, and it "takes care of things" in ways
>   I haven't tried to understand.  They don't use such setup scripts, at
>   least in a user-visible way (although the user does perform the
>   cross-compiling).  ]

Those scripts are written for the shell the tool provider intends user
to use.  The shell to use is not always our choice.

> > In order to compile software in this environment from Emacs I need to
> > start Emacs from the shell that has run the script.  But to me Emacs
> > is not a text editor but a working environment.
> 
> I partly understand the context.  But right here I think your script
> fails to serve you: it uses `setenv' which affects all inferior
> processes, so if you're working on several projects at the same time
> from the same Emacs session (likely if you're using it as a work
> environment, which is also the intended use to a large extent), you get
> pollution from one project to the other.  I think this issue is
> important, and I suspect that figuring out how it should be solved is
> necessary to figure out what is a good/generic way to provide this
> "source" feature.

This is a good point.  I am contaminating the whole emacs.  It made me
review compile.el and I learned the existence of
compilation-environment which I think is more appropriate than
`setenv' function.


(defun source (script &optional shell keep-current-directory)
  "Source the specified shell script and set `compilation-environment'.
Source the shell SCRIPT and import the environment into
`compilation-environment'.  The optional SHELL specifies the
shell other than the default `shell-file-name'.  When
KEEP-CURRENT-DIRECTORY is nil, which is the default, the current
directory is temporarily changed to the directory where the
script resides while sourcing the it."
  (interactive "fscript file: ")
  (if (null shell)
      (setq shell shell-file-name))
  (with-temp-buffer
    (unless keep-current-directory
      (setq default-directory (file-name-directory script)))
    (call-process shell nil t nil "-c" (concat ". " (shell-quote-argument script) "; printenv"))
    (setq compilation-environment
	  (split-string
	   (buffer-substring-no-properties (point-min) (point-max))
	   "\n"))))

Re: simple useful functions

[Stefan Monnier]

>> IIUC these scripts are written for /bin/sh, right?  How do (t)csh users
>> handle that?
>> [ Sorry, I'm not that familiar with cross-compiling: OpenWRT is about
>> as far as I got into this, and it "takes care of things" in ways
>> I haven't tried to understand.  They don't use such setup scripts, at
>> least in a user-visible way (although the user does perform the
>> cross-compiling).  ]

> Those scripts are written for the shell the tool provider intends user
> to use.  The shell to use is not always our choice.

In the case of the scripts you've used, was there some way to
mechanically figure out which shell was intended?  I'm thinking that
using shell-file-name is probably not the right choice, and we should
instead default to /bin/sh (which I'd expect to be the most common
case).

> This is a good point.  I am contaminating the whole emacs.  It made me
> review compile.el and I learned the existence of
> compilation-environment which I think is more appropriate than
> `setenv' function.

So only `compile' needs to know about these env-vars?


        Stefan

Re: simple useful functions

[Tak Ota]

Mon, 6 Dec 2010 12:23:21 -0800: Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> >> IIUC these scripts are written for /bin/sh, right?  How do (t)csh users
> >> handle that?
> >> [ Sorry, I'm not that familiar with cross-compiling: OpenWRT is about
> >> as far as I got into this, and it "takes care of things" in ways
> >> I haven't tried to understand.  They don't use such setup scripts, at
> >> least in a user-visible way (although the user does perform the
> >> cross-compiling).  ]
> 
> > Those scripts are written for the shell the tool provider intends user
> > to use.  The shell to use is not always our choice.
> 
> In the case of the scripts you've used, was there some way to
> mechanically figure out which shell was intended?  I'm thinking that
> using shell-file-name is probably not the right choice, and we should
> instead default to /bin/sh (which I'd expect to be the most common
> case).
> 
> > This is a good point.  I am contaminating the whole emacs.  It made me
> > review compile.el and I learned the existence of
> > compilation-environment which I think is more appropriate than
> > `setenv' function.
> 
> So only `compile' needs to know about these env-vars?

No, the debugger also needs.  I don't find similar consideration in
gdb-ui.el.  Do you have a better suggestion?

-Tak

Re: simple useful functions

[Stefan Monnier]

>> >> IIUC these scripts are written for /bin/sh, right?  How do (t)csh users
>> >> handle that?
>> >> [ Sorry, I'm not that familiar with cross-compiling: OpenWRT is about
>> >> as far as I got into this, and it "takes care of things" in ways
>> >> I haven't tried to understand.  They don't use such setup scripts, at
>> >> least in a user-visible way (although the user does perform the
>> >> cross-compiling).  ]
>> 
>> > Those scripts are written for the shell the tool provider intends user
>> > to use.  The shell to use is not always our choice.
>> 
>> In the case of the scripts you've used, was there some way to
>> mechanically figure out which shell was intended?  I'm thinking that
>> using shell-file-name is probably not the right choice, and we should
>> instead default to /bin/sh (which I'd expect to be the most common
>> case).
>> 
>> > This is a good point.  I am contaminating the whole emacs.  It made me
>> > review compile.el and I learned the existence of
>> > compilation-environment which I think is more appropriate than
>> > `setenv' function.
>> So only `compile' needs to know about these env-vars?
> No, the debugger also needs.  I don't find similar consideration in
> gdb-ui.el.  Do you have a better suggestion?

Well, we could imagine a new var (like compilation-environment) and make
all relevant commands obey it (maybe it could be implemented directly
within call/start-process and the new var set via dir-locals.el).


        Stefan