paragraph-start in Shell mode and Ielm.

paragraph-start in Shell mode and Ielm.

[Luc Teirlinck]

I propose to change paragraph-start in Shell mode and Ielm from:

(setq paragraph-start comint-prompt-regexp)

to:

(setq paragraph-start (concat "[ \t\f]*$\\|" comint-prompt-regexp))

This will make no user visible difference, for reasons explained
below.  But the current value is misleading given the paragraph-start
docstring:

  *Regexp for beginning of a line that starts OR separates paragraphs.
  This regexp should match lines that separate paragraphs
  and should also match lines that start a paragraph
  (and are part of that paragraph).

In Shell mode and Ielm blank lines already do separate paragraphs,
since the value of paragraph separate is "[ \t\f]*$".  In view of the
above excerpt of the docstring, paragraph-start should match empty
lines too, which the current value does not.

I can install if desired.

Sincerely,

Luc.

Re: paragraph-start in Shell mode and Ielm.

[Luc Teirlinck]

>From my previous message:

   I propose to change paragraph-start in Shell mode and Ielm from:

   (setq paragraph-start comint-prompt-regexp)

   to:

   (setq paragraph-start (concat "[ \t\f]*$\\|" comint-prompt-regexp))

Maybe (concat paragraph-separate "\\|" comint-prompt-regexp)
might be better, because it is more robust against changes in
paragraph-separate.

Sincerely,

Luc.

Re: paragraph-start in Shell mode and Ielm.

[Stefan Monnier]

> I propose to change paragraph-start in Shell mode and Ielm from:
> (setq paragraph-start comint-prompt-regexp)
> to:
> (setq paragraph-start (concat "[ \t\f]*$\\|" comint-prompt-regexp))

I believe it's a bad idea.

> This will make no user visible difference, for reasons explained
> below.  But the current value is misleading given the paragraph-start
> docstring:

>   *Regexp for beginning of a line that starts OR separates paragraphs.
>   This regexp should match lines that separate paragraphs
>   and should also match lines that start a paragraph
>   (and are part of that paragraph).

The docstring should be updated: it was necessary for paragraph-start to
also match entries matched by paragraph-separate, but it's not the case
any more.


        Stefan

Re: paragraph-start in Shell mode and Ielm.

[Luc Teirlinck]

Stefan Monnier wrote:

   The docstring should be updated: it was necessary for paragraph-start to
   also match entries matched by paragraph-separate, but it's not the case
   any more.

I do not know when this change occurred, but I do not see any mention
in the NEWS.  The docstring, the Emacs manual and the Elisp manual all
are still very emphatic about the fact that paragraph-start "is the
regular expression for recognizing the beginning of a line that starts
_or_ separates paragraphs".  (From `(elisp)Standard Regexps'.)

Sincerely,

Luc.

Re: paragraph-start in Shell mode and Ielm.

[Stefan Monnier]

>    The docstring should be updated: it was necessary for paragraph-start to
>    also match entries matched by paragraph-separate, but it's not the case
>    any more.

> I do not know when this change occurred, but I do not see any mention
> in the NEWS.

I think the change happened as a side effect of bug-fixes and cleanups.

> The docstring, the Emacs manual and the Elisp manual all
> are still very emphatic about the fact that paragraph-start "is the
> regular expression for recognizing the beginning of a line that starts
> _or_ separates paragraphs".  (From `(elisp)Standard Regexps'.)

These should maybe be fixed as well, then.


        Stefan

Re: paragraph-start in Shell mode and Ielm.

[Richard Stallman]

    >    The docstring should be updated: it was necessary for paragraph-start to
    >    also match entries matched by paragraph-separate, but it's not the case
    >    any more.

    > I do not know when this change occurred, but I do not see any mention
    > in the NEWS.

    I think the change happened as a side effect of bug-fixes and cleanups.

If this change has been made, it is worth mentioning in NEWS.
However, if you didn't set out intentionally to make this change,
are you sure it has been made completely?  Have you checked all
the uses of paragraph-start to make sure they really work
when paragraph-start does not match separator lines?

Re: paragraph-start in Shell mode and Ielm.

[Stefan Monnier]

>> The docstring should be updated: it was necessary for paragraph-start to
>> also match entries matched by paragraph-separate, but it's not the case
>> any more.

>> I do not know when this change occurred, but I do not see any mention
>> in the NEWS.

>     I think the change happened as a side effect of bug-fixes and cleanups.

> If this change has been made, it is worth mentioning in NEWS.
> However, if you didn't set out intentionally to make this change,
> are you sure it has been made completely?  Have you checked all
> the uses of paragraph-start to make sure they really work
> when paragraph-start does not match separator lines?

I did check w.r.t to paragraph.el, refill.el, and fill.el.
Don't know about the rest.


        Stefan

Re: paragraph-start in Shell mode and Ielm.

[Luc Teirlinck]

Richard Stallman wrote:

   Have you checked all the uses of paragraph-start to make sure they
   really work when paragraph-start does not match separator lines?

Actually, another problem is that some modes apparently _relied_ on the
fact that paragraph-separate used to be meaningless if it did not also
match paragraph-start.  The recent 22.0.50 changes in paragraph
behavior have broken paragraphs at least in Shell mode and Ielm.
Although I do not know any further examples, it is likely that
paragraphs in other modes are now broken too for the same reason.

In my proposed changes to misc.texi, I documented the fact that blank
lines in Shell mode also separated paragraphs.  But I now realize that
this was documenting a bug.  To be useful in shell mode, paragraphs
should cover prompt, input and output.  That was the behavior in 21.3
and earlier and we should revert to that correct behavior.  If we do
not want to revert the changes Stefan referred to then we should set
paragraph-separate to something that does not match anything (what is
the standard regexp for that? "\\'"?), in Shell mode and Ielm and
possibly in some other modes.

Below is the proposed new last paragraph of the new "Shell Prompts"
node.  It assumes that the above bug will be fixed.  The description
of the behavior of paragraph commands if comint-use-prompt-regexp is
nil (the default) is deliberately vague, as it is messy (for people
not familiar with fields, it could easily appear to be buggy,
especially in as far as numeric arguments and such are concerned).
But that behavior is already present in 21.3, so people could actually
be used to it.

    In Shell mode, only shell prompts start new paragraphs.  Thus, a
  paragraph consists of a prompt and the input and output that follow
  it.  However, if @code{comint-use-prompt-regexp} is @code{nil}, the
  default, most paragraph commands do not cross field boundaries.  Thus,
  with this setting, prompt, input and output behave mostly as if they
  were separate paragraphs and most numeric arguments to paragraph
  commands are ignored.  For the purpose of finding paragraph
  boundaries, Shell mode uses @code{shell-prompt-pattern}, regardless of
  @code{comint-use-prompt-regexp}.

Re: paragraph-start in Shell mode and Ielm.

[Luc Teirlinck]

>From my previous reply:

  Thus, with this setting, prompt, input and output behave mostly as
  if they were separate paragraphs and most numeric arguments to
  paragraph commands are ignored.

Should have been:

  Thus, with this setting, prompt, input and output behave mostly as
  if they were separate paragraphs and most numeric arguments to
  paragraph commands are ignored, except for their sign.

Sincerely,

Luc.

Re: paragraph-start in Shell mode and Ielm.

[Richard Stallman]

    > If this change has been made, it is worth mentioning in NEWS.
    > However, if you didn't set out intentionally to make this change,
    > are you sure it has been made completely?  Have you checked all
    > the uses of paragraph-start to make sure they really work
    > when paragraph-start does not match separator lines?

    I did check w.r.t to paragraph.el, refill.el, and fill.el.
    Don't know about the rest.

Could you please do a grep and check any other uses?

Re: paragraph-start in Shell mode and Ielm.

[Richard Stallman]

    Actually, another problem is that some modes apparently _relied_ on the
    fact that paragraph-separate used to be meaningless if it did not also
    match paragraph-start.  The recent 22.0.50 changes in paragraph
    behavior have broken paragraphs at least in Shell mode and Ielm.

This is very disturbing news.

There are only two ways out of this squeeze: to back up to the
previous consistent design, or move forward to a new one.  One
advantages in moving back is that we know where the changes would have
to be made.  We don't know all the places that would have to be
changed to move forward.

Since the change was not made intentionally, and since there
is no great advantage in this alteration in the specifications
of paragraph-start, I think it is best if we move back.

Stefan, can you work on that?

Re: paragraph-start in Shell mode and Ielm.

[Stefan Monnier]

> Stefan, can you work on that?

I just got a baby girl and am kind of swamped,


        Stefan

Re: paragraph-start in Shell mode and Ielm.

[Richard Stallman]

    > Stefan, can you work on that?

    I just got a baby girl and am kind of swamped,

I am sorry to hear it.  Unless someone else can figure these things
out, I guess the release has to wait until you have time.

Re: paragraph-start in Shell mode and Ielm.

[Nick Roberts]

 >     > Stefan, can you work on that?
 > 
 >     I just got a baby girl and am kind of swamped,
 > 
 > I am sorry to hear it.  Unless someone else can figure these things
 > out, I guess the release has to wait until you have time.

Congratulations, Stefan! I suggest that you spend any spare time with your
daughter as she will grow up before you know it. Emacs, on the other hand,
will still be around after she has left home.

Nick

Re: paragraph-start in Shell mode and Ielm.

[Richard Stallman]

It doesn't take special talents to reproduce--even plants can do it.
On the other hand, contributing to a program like Emacs takes real
skill.  That is really something to be proud of.

It helps more people, too.