bug#66050: Making perl-mode.el obsolete

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Severity: wishist

I don't think it makes sense for us to spend our meager resources
maintaining two major modes for Perl.  I would like to gauge what people
think about obsoleting perl-mode.el.

Harald Jörg wrote previously on emacs-devel:

> Finally, perl-mode comes with its own list of ancient open bugs, many of
> those don't occur in cperl-mode.  I wonder whether the authors of these
> bugs would accept "use cperl-mode instead" as a workaround?
>
> At some time, it might make sense to merge those two modes into one.
> Perl continues to evolve, and upgrading two modes to support that
> doesn't seem to be an economic use of time.

    https://lists.gnu.org/r/emacs-devel/2020-10/msg01492.html

Here are some additional observations:

- cperl-mode.el sees more maintenance than perl-mode.el, in large part
  thanks to the efforts of Harald Jörg.

- The Perl community tends to favor cperl-mode over perl-mode.
  perl-mode is seen as lacking in features compared to cperl-mode, and
  no significant development has taken place to bridge the gap.

- cperl-mode.el used to be maintained outside of Emacs, but this is no
  longer the case.  All relevant development has been merged into and
  takes place in emacs.git.

- Perl, while historically important to hacker culture and still widely
  used in some quarters (e.g. Debian), is seeing much less use today
  than it used to.  This will negatively affect the amount of help we
  can expect with maintaining these modes from others.

- Instead of maintaining perl-mode.el, I'd rather see that people worked
  on a new perl-ts-mode.el.  From a web search, more than one treesitter
  grammar exist; I have no idea which one is the most promising or how
  mature any of them are.

bug#66050: Making perl-mode.el obsolete

[Jens Schmidt via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas <stefankangas@gmail.com> writes:

> I don't think it makes sense for us to spend our meager resources
> maintaining two major modes for Perl.  I would like to gauge what people
> think about obsoleting perl-mode.el.

There has been a thread on emacs-devel recently on that:

  https://yhetil.org/emacs-devel/16da6ae7-66d8-fc43-cb84-6d104d3a2ef8@mavit.org.uk/

plus my opinion:

  https://yhetil.org/emacs-devel/4d18a051-07d5-fba7-1c36-ae2eb72bf71c@vodafonemail.de/

which still holds.

But I can see your points, of course.

cperl-mode already has some big-config-sweep-knobs, like it seems
(`cperl-hairy').  It would certainly be nice to also have some
"make-things-work-like-in-perl-mode-but-sans-its-bugs" knob if it came
to obsoleting perl-mode.  For me, the most important things to cover
here would be a nearly-identical look-and-feel for indentation and
syntax highlighting.

One could even imagine to flip that knob automatically when `cperl-mode'
gets started as `perl-mode'.

> - Instead of maintaining perl-mode.el, I'd rather see that people worked
>   on a new perl-ts-mode.el.  From a web search, more than one treesitter
>   grammar exist; I have no idea which one is the most promising or how
>   mature any of them are.

+1, with the same restrictions on indentation and syntax highlighting.
After 25+ years of using one mode you just get so used to its
look-and-feel.

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes:

> cperl-mode already has some big-config-sweep-knobs, like it seems
> (`cperl-hairy').  It would certainly be nice to also have some
> "make-things-work-like-in-perl-mode-but-sans-its-bugs" knob if it came
> to obsoleting perl-mode.  For me, the most important things to cover
> here would be a nearly-identical look-and-feel for indentation and
> syntax highlighting.

Which features would be covered by `like-perl-mode-sans-bugs', that are
not already covered by `cperl-hairy'?  Is it just a matter of setting
some already existing options or is "more work" needed?

> One could even imagine to flip that knob automatically when `cperl-mode'
> gets started as `perl-mode'.

Maybe, but then again some users might want to continue using the
original perl-mode even if it's obsolete.

bug#66050: Making perl-mode.el obsolete

[Jens Schmidt via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas <stefankangas@gmail.com> writes:

> Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes:
>
>> cperl-mode already has some big-config-sweep-knobs, like it seems
>> (`cperl-hairy').  It would certainly be nice to also have some
>> "make-things-work-like-in-perl-mode-but-sans-its-bugs" knob if it came
>> to obsoleting perl-mode.  For me, the most important things to cover
>> here would be a nearly-identical look-and-feel for indentation and
>> syntax highlighting.
>
> Which features would be covered by `like-perl-mode-sans-bugs', that are
> not already covered by `cperl-hairy'?

Just to avoid misunderstandings: I have tried so far everything with
`cperl-hairy' unchanged from default nil.  I guess that non-nil makes
cperl-mode even more electric, etc., which is actually the opposite
direction of what I'm heading for.

> Is it just a matter of setting some already existing options or is
> "more work" needed?

That's more a question for Harald, I think, since I'm not very well
acquainted with cperl-mode.

I can state only what I have noticed this afternoon, after some very
cursory comparison and review.  Hoping this is the right place for such
a wish list:

1. All scalar references should be hightlighted with
   font-lock-variable-name-face.

Seems that `cperl-highlight-variables-indiscriminately' provides that.

2. Arrays and hashes should be highlighted just like scalars, in
   font-lock-variable-name-face.

3. Variable sigils ("$", "%", "@") should not be highlighted at all.

4. Builtins ("shift", "ref", "defined") should not be highlighted at
   all.

5. Package names should be highlighted as font-lock-constant-face rather
   than font-lock-function-face.

6. Unquoted hash keys before "=>" should not be highlighted at all.

7. Regexps should be highlighted as font-lock-string-face.

When I switch on cperl-mode, there is little left which is *not*
highlighted ...

8. Re-indentation of blocks should not merge else/elsif keywords with
   preceeding curlies.

9. Re-indenting a region should not adjust positions of after-code
   comments.

Ah, cperl-style "PBP" covers 8. and 9. above!

10. Some parts of cperl-mode look, well, over-engineered.  Function
    `cperl-find-pods-heres' is a stunning 1000+ lines long and obviously
    features regexps with more than 20 subres.

    BTW, on Image/ExifTool.pm as of version 12.16 I get errors in "emacs
    -Q" when switching on cperl-mode, I think because of that function.

11. Last not least, these pseudo-variables on the page labeled "Short
    extra-docs."  always bug me.  Packing arbitrary documentation in
    variables seems rather strange and rather non-standard.

While that does not sound much:

- I have the impression that, despite so many options in cperl-mode
  being available, just the right ones to "turn cperl-mode into
  perl-mode" are missing.  But that could also be because I haven't
  found them yet ...

- There's probably more to it than that.  Having some
  `like-perl-mode-sans-bugs' umbrella would be not only helpful to make
  this more easily customizable, but also to have something to "attach"
  future development to.  Not sure how to express that better.

bug#66050: Making perl-mode.el obsolete

[Corwin Brust]

On Sun, Sep 17, 2023 at 7:47 AM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> Severity: wishist
>
> I don't think it makes sense for us to spend our meager resources
> maintaining two major modes for Perl.  I would like to gauge what people
> think about obsoleting perl-mode.el.
>

+1

I've been aliasing perl-mode to cperl-mode for well over a decade.
Every few years I try removing this but I haven't been very happy with
"vanilla" perl-mode: it just isn't as pretty, and I press more keys to
do the same things.  (Probably not enough specificity here to be very
helpful, alas.)

Before I knew any elisp I'd memorized these lines use when creating a
new .emacs file to Emacs on new system:

(defalias 'perl-mode 'cperl-mode "Prefer cperl-mode.")
(setq cperl-hairy t)
(setq cperl-indent-level 2)
(require 'cperl-mode)

IMO, given I already need to (and, in fact, do) configure Emacs to
turn on all the bells and whistles, it would make sense to add
configuration to (probably by default) turn off whatever is needed to
make cperl-mode less intrusive/off putting to those preferring
perl-mode now.  As a route to removing the need for two perl major
modes in core, I think it makes more sense than trying to making
perl-mode do what cperl-mode can while retaining it's more "bland"
defaults: that seems futile.


>
> - Instead of maintaining perl-mode.el, I'd rather see that people worked
>   on a new perl-ts-mode.el.  From a web search, more than one treesitter
>   grammar exist; I have no idea which one is the most promising or how
>   mature any of them are.
>

I believe this project from Paul Evans (author of Corrina, the new OO
system added in Perl 5.38) is the better TS grammar for perl:

https://github.com/tree-sitter-perl/tree-sitter-perl

A fairly current DLL for Windows uses should usually be available from here:

https://corwin.bru.st/emacs-tree-sitter/

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I don't think it makes sense for us to spend our meager resources
> maintaining two major modes for Perl.  I would like to gauge what people
> think about obsoleting perl-mode.el.

I think I sadly agree.

I've been a supporter of `perl-mode` because when I had to edit Perl
files, I always found `cperl-mode` not to be to my taste (too different
from other Emacs major modes).  So I've worked to improve `perl-mode`
based on my own needs, which were rather limited.

I considered improving `cperl-mode` instead, but found the code of
`perl-mode` easier to deal with (especially at the beginning, where
I found collaborating with Ilya was sometimes difficult).

Luckily I haven't needed to edit Perl files for the last several years.

I think the best option is to obsolete `perl-mode` and to encourage
people to report bugs against `cperl-mode` when they find it doesn't
work as well as their beloved `perl-mode` :-)


        Stefan

bug#66050: Making perl-mode.el obsolete

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Many years ago, I thought it would be an improvement on general
principles to have only one Perl mode.  I do not know Perl so I have
no opinion about either one.

Of course, I first asked what Perl users thought of this.  I found
that many preferred CPerl mode and many preferred Perl mode.  And the
ones who preferred each one were very attached to it, and did not want
to switch.

That may have changed, or maybe not.  I think we had better ask the
users now what they feel.  If there are still many users who strongly
prefer Perl mode, even if they are only 10% as many as strongly prefer
CPerl mode, we definitely should not mark Perl mode obsolete.

If it is feasible to put conditionals in CPerl mode to make it behave
like Perl mode -- or close enough that almost everyone is happy with
it -- maybe then we could obsolete the current Perl mode.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> If it is feasible to put conditionals in CPerl mode to make it behave
> like Perl mode -- or close enough that almost everyone is happy with
> it -- maybe then we could obsolete the current Perl mode.

Agreed.  Let's start with adding such options to cperl-mode.

bug#66050: Making perl-mode.el obsolete

[Corwin Brust]

On Wed, Sep 20, 2023 at 6:26 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> Richard Stallman <rms@gnu.org> writes:
>
> > If it is feasible to put conditionals in CPerl mode to make it behave
> > like Perl mode -- or close enough that almost everyone is happy with
> > it -- maybe then we could obsolete the current Perl mode.
>
> Agreed.  Let's start with adding such options to cperl-mode.
>

Does there already exist a list of cperl features/behaviors which
perl-mode users find objectionable?

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Corwin Brust <corwin@bru.st> writes:

> On Wed, Sep 20, 2023 at 6:26 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>>
>> Richard Stallman <rms@gnu.org> writes:
>>
>> > If it is feasible to put conditionals in CPerl mode to make it behave
>> > like Perl mode -- or close enough that almost everyone is happy with
>> > it -- maybe then we could obsolete the current Perl mode.
>>
>> Agreed.  Let's start with adding such options to cperl-mode.
>
> Does there already exist a list of cperl features/behaviors which
> perl-mode users find objectionable?

I'm only aware of the list provided by Jens Schmidt upthread:

    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=66050#16

bug#66050: Making perl-mode.el obsolete

[Corwin Brust]

On Wed, Sep 20, 2023 at 7:16 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> Corwin Brust <corwin@bru.st> writes:
> >
> > Does there already exist a list of cperl features/behaviors which
> > perl-mode users find objectionable?
>
> I'm only aware of the list provided by Jens Schmidt upthread:
>
>     https://debbugs.gnu.org/cgi/bugreport.cgi?bug=66050#16
>

I missed that message completely; thank you for calling my attention
to it.  I will see if there are any of these I'm able to suggest fixes
for.  I hope my saying so will not prevent other (more experienced)
folk from working on this, if so inclined.

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Corwin Brust <corwin@bru.st> writes:

> I missed that message completely; thank you for calling my attention
> to it.  I will see if there are any of these I'm able to suggest fixes
> for.

Thank you for working on this.

> I hope my saying so will not prevent other (more experienced) folk
> from working on this, if so inclined.

Your offer to volunteer won't prevent anyone else from jumping in, so I
wouldn't worry about it.

bug#66050: Making perl-mode.el obsolete

[Mauro Aranda]

 > Corwin Brust <corwin@bru.st> writes:
 >
 >> On Wed, Sep 20, 2023 at 6:26 PM Stefan Kangas 
<stefankangas@gmail.com> wrote:
 >>>
 >>> Richard Stallman <rms@gnu.org> writes:
 >>>
 >>>> If it is feasible to put conditionals in CPerl mode to make it behave
 >>>> like Perl mode -- or close enough that almost everyone is happy with
 >>>> it -- maybe then we could obsolete the current Perl mode.
 >>>
 >>> Agreed.  Let's start with adding such options to cperl-mode.
 >>
 >> Does there already exist a list of cperl features/behaviors which
 >> perl-mode users find objectionable?
 >
 > I'm only aware of the list provided by Jens Schmidt upthread:
 >
 >      https://debbugs.gnu.org/cgi/bugreport.cgi?bug=66050#16

What would be a good way of collecting this information?

In addition to what Jens Schmidt said, I can add that:

1. If I have something like:
my $some_code = "";
$some_code.= q(
my $counter = 0;
);

If I put point at column 0 of the line "my $counter", and hit TAB, I get
indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
options for making this work but I couldn't find anything.

2. While I'm typing the above string, I get messages about string/RE not
found:
End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced 
parentheses 1092 1874)
End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced 
parentheses 1092 1918) [2 times]
End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced 
parentheses 1092 1962) [2 times]

That's annoying.


So far, my settings for getting a perl-mode experience in cperl-mode,
with emacs -Q: (taken from a custom file):
'(cperl-highlight-variables-indiscriminately t)
'(cperl-indent-level 4)
'(cperl-indent-parens-as-block t)
'(cperl-invalid-face 'default)

'(cperl-array-face ((t (:inherit cperl-hash-face))))
'(cperl-hash-face ((t (:underline t :inherit 
font-lock-variable-name-face))))
'(cperl-nonoverridable-face ((t (:inherit default))))

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> What would be a good way of collecting this information?

I'd say: use `cperl-mode` and report regressions compared to what
`perl-mode` used to do via `M-x report-emacs-bug`.

> In addition to what Jens Schmidt said, I can add that:
>
> 1. If I have something like:
> my $some_code = "";
> $some_code.= q(
> my $counter = 0;
> );
>
> If I put point at column 0 of the line "my $counter", and hit TAB, I get
> indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
> options for making this work but I couldn't find anything.
>
> 2. While I'm typing the above string, I get messages about string/RE not
> found:
> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced parentheses
> 1092 1874)
> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced parentheses
> 1092 1918) [2 times]
> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced parentheses
> 1092 1962) [2 times]
>
> That's annoying.

Sounds like a bug, eh?


        Stefan

bug#66050: Making perl-mode.el obsolete

[Mauro Aranda]

On 21/9/23 11:20, Stefan Monnier wrote:
 >> What would be a good way of collecting this information?
 >
 > I'd say: use `cperl-mode` and report regressions compared to what
 > `perl-mode` used to do via `M-x report-emacs-bug`.
 >

OK, I'll do that.  But Someone will have to keep track of the bug
reports that need to be handled before calling perl-mode obsolete.

 >> 2. While I'm typing the above string, I get messages about string/RE not
 >> found:
 >> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced 
parentheses
 >> 1092 1874)
 >> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced 
parentheses
 >> 1092 1918) [2 times]
 >> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced 
parentheses
 >> 1092 1962) [2 times]
 >>
 >> That's annoying.
 >
 > Sounds like a bug, eh?

I think so.  I had some time to look into the archived bugs and found
this: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=37127

I don't have time right now to read and understand the whole
conversation, but I think it's related to what I see.

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes:

>     BTW, on Image/ExifTool.pm as of version 12.16 I get errors in "emacs
>     -Q" when switching on cperl-mode, I think because of that function.

Please report this bug separately.

> 11. Last not least, these pseudo-variables on the page labeled "Short
>     extra-docs."  always bug me.  Packing arbitrary documentation in
>     variables seems rather strange and rather non-standard.

Suggestions for how to improve the situation are welcome, preferably in
a new bug report.

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Stefan Kangas <stefankangas@gmail.com> writes:

> I don't think it makes sense for us to spend our meager resources
> maintaining two major modes for Perl.  I would like to gauge what people
> think about obsoleting perl-mode.el.

I'm not the right person to talk about obsoleting, since I have never
used perl-mode.el.  When I started using Emacs for Perl, usenet was
still a thing.  Ilya Zakharevich (the long-time maintainer of
cperl-mode.el outside of Emacs) was active in the Perl newsgroups and
very helpful, so I started with cperl-mode.el and never looked back.

But I think that making cperl-mode.el acceptable for users of
perl-mode.el is doable and might be worth the one-time effort.

> [...]
> Here are some additional observations:
>
> - cperl-mode.el sees more maintenance than perl-mode.el, in large part
>   thanks to the efforts of Harald Jörg.

I intend to continue maintaining cperl-mode.el, though I've been a bit
distracted this year.

> - The Perl community tends to favor cperl-mode over perl-mode.
>   perl-mode is seen as lacking in features compared to cperl-mode, and
>   no significant development has taken place to bridge the gap.

The extra features of cperl-mode are also a source of criticism... and
they come at a cost of some arcane elisp code.  Yet, it should be easier
to add options to switch *off* unwanted features in cperl-mode than to
add them to perl-mode.  So collecting such unwanted features, as
suggested later in this thread, is a good start!

> - cperl-mode.el used to be maintained outside of Emacs, but this is no
>   longer the case.  All relevant development has been merged into and
>   takes place in emacs.git.

In my opinion, this makes sense.  Several Emacs developers have
continually worked on cperl-mode.el to care for obsoletions or
modernizations.  I don't think a place outside of emacs.git could
provide this.

> - Perl, while historically important to hacker culture and still widely
>   used in some quarters (e.g. Debian), is seeing much less use today
>   than it used to.  This will negatively affect the amount of help we
>   can expect with maintaining these modes from others.

I guess that's true.  Also, neither Perl mode systematically tracked the
development of Perl for a decade or so, it will take some time to
re-build confidence.

> - Instead of maintaining perl-mode.el, I'd rather see that people worked
>   on a new perl-ts-mode.el.  From a web search, more than one treesitter
>   grammar exist; I have no idea which one is the most promising or how
>   mature any of them are.

https://github.com/tree-sitter-perl/tree-sitter-perl is very actively
developed, also following the "actual" Perl grammar in Perl's perly.y
source.  I consider it promising.  However, we've already identified a
shortcoming of the tree-sitter approach regarding Perl: Many Perl
extensions from CPAN _extend_ the syntax with new keywords and
constructs.  It is possible to accound for those extensions in Elisp,
even dynamically on a per-buffer basis, but very difficult to do for
tree-sitter grammars which need to be "built" from C-code and
"installed" as dynamic libraries.

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes:

> [...]
> Just to avoid misunderstandings: I have tried so far everything with
> `cperl-hairy' unchanged from default nil.  I guess that non-nil makes
> cperl-mode even more electric, etc., which is actually the opposite
> direction of what I'm heading for.

Indeed.  I always use `cperl-hairy' and am not really aware how
cperl-mode looks without it, but I know that it is still more colourful
than perl-mode.

>> Is it just a matter of setting some already existing options or is
>> "more work" needed?
>
> That's more a question for Harald, I think, since I'm not very well
> acquainted with cperl-mode.

It would need more work.  An important step is to identify the features
we should be able to switch off per customization.  I am somewhat
reluctant to add more options to cperl-mode if they can't be bundled
(like `cperl-hairy' does):  The possible combinations of options are a
nightmare to document, to test, and also to learn for new users.

> I can state only what I have noticed this afternoon, after some very
> cursory comparison and review.  Hoping this is the right place for such
> a wish list:
>
> 1. All scalar references should be hightlighted with
>    font-lock-variable-name-face.
>
> Seems that `cperl-highlight-variables-indiscriminately' provides that.

Yes, that should be covered.  The option name is somewhat ... weird, but
I didn't find enough motivation to change it (or to fiddle with
font-lock-level 3, which would be more in line with other modes).

> 2. Arrays and hashes should be highlighted just like scalars, in
>    font-lock-variable-name-face.

This seems doable.  The easy way is to make cperl-hash-face and
cperl-array-face customizable so that they can be "downgraded" to
font-lock-variable-name-face.

cperl-mode.el goes to great lengths to identify usage of array and hash
variables in situations where they don't look like arrays (for example,
$var is a scalar, but $#var belongs to an array), and this
distinction is helpful for Perl beginners (it was for me).

But I see that these two extra faces of cperl-mode are somewhat
nonstandard, and they don't play well with all of the Emacs themes (nor
can we expect the developers of themes to take these faces into
account).

> 3. Variable sigils ("$", "%", "@") should not be highlighted at all.

I doubt that this is worth the effort in cperl-mode... and guess it
should be tolerable.  The sigil *is* part of the variable, after all.

> 4. Builtins ("shift", "ref", "defined") should not be highlighted at
>    all.

This is an area where cperl-mode is a bit untidy.  It has two different
faces for builtins, depending on whether they can be overridden by user
functions with the same name.  Many occur in two lists for
fontification, only the first one ever applies.  This has *some*
justification because builtins allow sloppy syntax (omitting
parentheses).  When we clean up this mess, adding a way to not
highlighting them at all should not be too difficult.

> 5. Package names should be highlighted as font-lock-constant-face rather
>    than font-lock-function-face.

perl-mode uses font-lock-function-name-face in the package declaration
but font-lock-constant-face in 'use' statements.  I can't say whether
that is intentional.

Someone once suggested to use font-lock-type-face (because packages
usually are classes, and classes are sort-of types).  This is one of the
cases where I doubt that adding more options will actually improve
things.

> 6. Unquoted hash keys before "=>" should not be highlighted at all.

They are highlighted as strings because the *are* strings.  This is not
implemented in perl-mode, and it causes misinterpretations like in
Bug#45083: Not highlighting barewords but also preventing them to be
picked up by some other highlighting rule might be tricky.

> 7. Regexps should be highlighted as font-lock-string-face.

This can be added as an option with some effort.  Regexps aren't
strings, but alas, almost no syntax highlighter takes the same effort as
cperl-mode to display them.

> When I switch on cperl-mode, there is little left which is *not*
> highlighted ...
>
> 8. Re-indentation of blocks should not merge else/elsif keywords with
>    preceeding curlies.
>
> 9. Re-indenting a region should not adjust positions of after-code
>    comments.
>
> Ah, cperl-style "PBP" covers 8. and 9. above!

You are welcome :)

> 10. Some parts of cperl-mode look, well, over-engineered.  Function
>     `cperl-find-pods-heres' is a stunning 1000+ lines long and obviously
>     features regexps with more than 20 subres.

I agree that this function isn't a piece of beauty.  It was written
before non-capturing groups were added to Emacs and never thoroughly
reengineered.  This makes maintenance of that part difficult, but should
not affect users.

>     BTW, on Image/ExifTool.pm as of version 12.16 I get errors in "emacs
>     -Q" when switching on cperl-mode, I think because of that function.

It would be an interesting exercise to find out which options cause
this.  I can reproduce it (it spits out an unjustified error message,
but apparently without any consequences).  It does not happen with my
settings, and it does not happen if I explicitly call
cperl-fund-pods-heres with any settings.

> 11. Last not least, these pseudo-variables on the page labeled "Short
>     extra-docs."  always bug me.  Packing arbitrary documentation in
>     variables seems rather strange and rather non-standard.

Strange and non-standard for sure, but harmless.

> While that does not sound much:
>
> - I have the impression that, despite so many options in cperl-mode
>   being available, just the right ones to "turn cperl-mode into
>   perl-mode" are missing.  But that could also be because I haven't
>   found them yet ...

Correct, such an option does not exist.

> - There's probably more to it than that.  Having some
>   `like-perl-mode-sans-bugs' umbrella would be not only helpful to make
>   this more easily customizable, but also to have something to "attach"
>   future development to.  Not sure how to express that better.

I am not sure I understand what this means.  perl-mode does not
understand new syntax like the "class" feature - but that's not a bug,
it just has never been implemented.  Should these things be treated
"like in cperl-mode" (because in cperl-mode they are implemented), left
as they are in perl-mode today, or would perl-mode users want to decide
how support should look like for new language features?

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I don't think it makes sense for us to spend our meager resources
>> maintaining two major modes for Perl.  I would like to gauge what people
>> think about obsoleting perl-mode.el.
>
> I think I sadly agree.
>
> I've been a supporter of `perl-mode` because when I had to edit Perl
> files, I always found `cperl-mode` not to be to my taste (too different
> from other Emacs major modes). ...

Are there conventions or general expectations for major modes?  I would
love to make cperl-mode "less exotic", but Perl *is* a bit exotic, and
the "Major Mode Conventions" seem to be more or less obeyed (they cover
developer conventions, not user expectations).  Both modes derive from
prog-mode, and both have no built-in completion nor ElDoc support... but
Perl language servers can add these to some extent.

Another convention is documented in the "Levels of Font Lock", but neither
of the Perl modes makes use of the three levels as recommended.

> [...]
> I think the best option is to obsolete `perl-mode` and to encourage
> people to report bugs against `cperl-mode` when they find it doesn't
> work as well as their beloved `perl-mode` :-)

+1
-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Mauro Aranda <maurooaranda@gmail.com> writes:

> On 21/9/23 11:20, Stefan Monnier wrote:
>  >> What would be a good way of collecting this information?
>  >
>  > I'd say: use `cperl-mode` and report regressions compared to what
>  > `perl-mode` used to do via `M-x report-emacs-bug`.
>  >
>
> OK, I'll do that.  But Someone will have to keep track of the bug
> reports that need to be handled before calling perl-mode obsolete.

If the bugs include the word "perl" somewhere in Subject, tracking them
for this purpose will not take much effort.

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Harald Jörg <haj@posteo.de> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> I think the best option is to obsolete `perl-mode` and to encourage
>> people to report bugs against `cperl-mode` when they find it doesn't
>> work as well as their beloved `perl-mode` :-)
>
> +1

How about this basic plan:

First, we make perl-mode.el obsolete on master.

Second, we announce that people should report bugs against cperl-mode
where they want it to be more like perl-mode, and that we are looking
for interested volunteers to help work on implementing this.

Third, we close all bugs against perl-mode as wontfix, and recommend the
reporter to see if it is fixed by using cperl-mode instead.

Fourth, if we don't have enough tangible progress in cperl-mode, we can
unobsolete perl-mode.el on the Emacs 30 release branch, right after it
is cut, while keeping it in obsolete/ on master.  That would give us
additional time to make sure cperl-mode.el is up to scratch.

My idea here is to motivate people to work on cperl-mode, avoid
unnecessary bikeshedding, and to take a firm stance on the direction we
want to take as a project here.

Does anyone object to or see any problems with the above?

bug#66050: Making perl-mode.el obsolete

[Stefan Kangas]

Harald Jörg <haj@posteo.de> writes:

> But I think that making cperl-mode.el acceptable for users of
> perl-mode.el is doable and might be worth the one-time effort.

Great!  It sounds like a plan, then.

> I intend to continue maintaining cperl-mode.el, though I've been a bit
> distracted this year.

BTW, feel free to mark yourself as a maintainer in the cperl-mode.el
header.  I think it'd be useful to provide clarity on the fact that it
is, in fact, actively maintained.

>> - cperl-mode.el used to be maintained outside of Emacs, but this is no
>>   longer the case.  All relevant development has been merged into and
>>   takes place in emacs.git.
>
> In my opinion, this makes sense.  Several Emacs developers have
> continually worked on cperl-mode.el to care for obsoletions or
> modernizations.  I don't think a place outside of emacs.git could
> provide this.

Fully agreed.

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> It would need more work.  An important step is to identify the features
> we should be able to switch off per customization.  I am somewhat
> reluctant to add more options to cperl-mode if they can't be bundled
> (like `cperl-hairy' does):  The possible combinations of options are a
> nightmare to document, to test, and also to learn for new users.

BTW, "bundling" is what Custom themes are for.
At least to solve the "learn for new users" side of the problem.
Not to say we should add more options (or more fine-grained options),
but it can be helpful to simplify the code (a.g. avoid having to test both
`cperl-dont-be-hairy` and `cperl-highlight-variables-indiscriminately`).

> Yes, that should be covered.  The option name is somewhat ... weird, but
> I didn't find enough motivation to change it (or to fiddle with
> font-lock-level 3, which would be more in line with other modes).

That's a downvote for font-lock levels from me.

> This seems doable.  The easy way is to make cperl-hash-face and
> cperl-array-face customizable so that they can be "downgraded" to
> font-lock-variable-name-face.

They're faces, so they are already customizable, e.g. via Custom themes.

>> 3. Variable sigils ("$", "%", "@") should not be highlighted at all.
> I doubt that this is worth the effort in cperl-mode... and guess it
> should be tolerable.  The sigil *is* part of the variable, after all.

FWIW, I agree.  We don't have to satisfy all the wishlist items of
previous `perl-mode` users.

>> 4. Builtins ("shift", "ref", "defined") should not be highlighted at
>>    all.
> This is an area where cperl-mode is a bit untidy.  It has two different
> faces for builtins, depending on whether they can be overridden by user
> functions with the same name.  Many occur in two lists for
> fontification, only the first one ever applies.  This has *some*
> justification because builtins allow sloppy syntax (omitting
> parentheses).

What part of this is "untidy" or "a mess"?  I can see why you'd say it
w.r.t Perl having those fine distinctions, but the corresponding
features in `cperl-mode` seem to just reflect Perl's syntax&semantics.
Or is it the implementation part to distinguish those two kinds of
builtins messy?

> When we clean up this mess, adding a way to not
> highlighting them at all should not be too difficult.

AFAIK that can also be solved by changing the faces' appearance, hence
without changing the code.

>> 5. Package names should be highlighted as font-lock-constant-face rather
>>    than font-lock-function-face.
>
> perl-mode uses font-lock-function-name-face in the package declaration
> but font-lock-constant-face in 'use' statements.  I can't say whether
> that is intentional.
>
> Someone once suggested to use font-lock-type-face (because packages
> usually are classes, and classes are sort-of types).  This is one of the
> cases where I doubt that adding more options will actually improve
> things.

I think it's more important here to choose a "meaningful/consistent"
behavior than to reproduce what was done historically in `perl-mode` or
`cperl-mode`.

>> 7. Regexps should be highlighted as font-lock-string-face.
> This can be added as an option with some effort.  Regexps aren't
> strings, but alas, almost no syntax highlighter takes the same effort as
> cperl-mode to display them.

Here again, I believe it's a small matter of changing faces.


        Stefan

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Does anyone object to or see any problems with the above?

You can put two votes under "Stefan".


        The first

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> It would need more work.  An important step is to identify the features
>> we should be able to switch off per customization.  I am somewhat
>> reluctant to add more options to cperl-mode if they can't be bundled
>> (like `cperl-hairy' does):  The possible combinations of options are a
>> nightmare to document, to test, and also to learn for new users.
>
> BTW, "bundling" is what Custom themes are for.
> At least to solve the "learn for new users" side of the problem.
> Not to say we should add more options (or more fine-grained options),
> but it can be helpful to simplify the code (a.g. avoid having to test both
> `cperl-dont-be-hairy` and `cperl-highlight-variables-indiscriminately`).

Agreed.

cperl-mode has many customization options I've never tried (nor fully
understood).  These days, "Perl Best Practices" (the book, almost 20
years old by now) and perltidy (the tool) could be taken as guidelines
for new users.  Unfortunately, Perl has lots of very knowledgeable old
users, some of which are rather religious in their opinions - and
perltidy lets all of them have it their way.

I *guess* that for new users a manual or tutorial would be helpful, but
don't know whether I can spend the time to write it (nor whether I'm the
right person to do it).

>> Yes, that should be covered.  The option name is somewhat ... weird, but
>> I didn't find enough motivation to change it (or to fiddle with
>> font-lock-level 3, which would be more in line with other modes).
>
> That's a downvote for font-lock levels from me.

Oh - I guessed that ignoring font-lock-levels was one of the things you
meant when you wrote that cperl-mode is different than other major modes...

>> This seems doable.  The easy way is to make cperl-hash-face and
>> cperl-array-face customizable so that they can be "downgraded" to
>> font-lock-variable-name-face.
>
> They're faces, so they are already customizable, e.g. via Custom themes.

Sure.  I think that creating custom themes might be a bit beyond what
*users* of c?perl-mode might want to do (I myself never did that).
Emacs comes with themes which look better than anything I could create -
and adding four extra faces which fit these palettes needs experience I
don't have.

>>> 3. Variable sigils ("$", "%", "@") should not be highlighted at all.
>> I doubt that this is worth the effort in cperl-mode... and guess it
>> should be tolerable.  The sigil *is* part of the variable, after all.
>
> FWIW, I agree.  We don't have to satisfy all the wishlist items of
> previous `perl-mode` users.

>>> 4. Builtins ("shift", "ref", "defined") should not be highlighted at
>>>    all.
>> This is an area where cperl-mode is a bit untidy.  It has two different
>> faces for builtins, depending on whether they can be overridden by user
>> functions with the same name.  Many occur in two lists for
>> fontification, only the first one ever applies.  This has *some*
>> justification because builtins allow sloppy syntax (omitting
>> parentheses).
>
> What part of this is "untidy" or "a mess"?  I can see why you'd say it
> w.r.t Perl having those fine distinctions, but the corresponding
> features in `cperl-mode` seem to just reflect Perl's syntax&semantics.
> Or is it the implementation part to distinguish those two kinds of
> builtins messy?

It is the implementation.

There's some overlap in the "regex-opt" lists of keywords
(e.g. "require", "for").  Some of the "builtin functions" are actually binary
operands ("eq" and friends).  "use" is fontified as a keyword, but "no"
(which is sort of "do not use") is fontified as a non-overridable
function.  The builtins use font-lock-type-face, which isn't
appropriate.  "shift" is fontified as non-overridable, but it can be
overridden in recent Perl versions.

An overhaul of all of this would take some time, and probably a thick
skin to weather the storm caused by cperl-mode users who are accustomed
to the current colors.

>> When we clean up this mess, adding a way to not
>> highlighting them at all should not be too difficult.
>
> AFAIK that can also be solved by changing the faces' appearance, hence
> without changing the code.

Agreed.

>>> 5. Package names should be highlighted as font-lock-constant-face rather
>>>    than font-lock-function-face.
>>
>> perl-mode uses font-lock-function-name-face in the package declaration
>> but font-lock-constant-face in 'use' statements.  I can't say whether
>> that is intentional.
>>
>> Someone once suggested to use font-lock-type-face (because packages
>> usually are classes, and classes are sort-of types).  This is one of the
>> cases where I doubt that adding more options will actually improve
>> things.
>
> I think it's more important here to choose a "meaningful/consistent"
> behavior than to reproduce what was done historically in `perl-mode` or
> `cperl-mode`.

Agreed!

>>> 7. Regexps should be highlighted as font-lock-string-face.
>> This can be added as an option with some effort.  Regexps aren't
>> strings, but alas, almost no syntax highlighter takes the same effort as
>> cperl-mode to display them.
>
> Here again, I believe it's a small matter of changing faces.

It is a bit more than _changing_ faces.  In the match part, cperl-mode
highlights metacharacters (|) as keywords, [] as functions, and
character classes (\d \D \s and friends) as types.  The substitution
part can be actual Perl code, and cperl-mode will fontify it as Perl
code.

For example, the following substitution is rather colorful in cperl-mode
but (sorry) dull in perl-mode:

   s/\d+|[IVXLCDM]+(\D)/sqrt(2) . "_$1_"/eri;

Making these highlightings optional would, as far as I know, require to
define an extra set of faces, which could then be mapped either to their
cperl-mode values (cperl-mode style) or all to font-lock-string-face
(perl-mode style).

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>>> Yes, that should be covered.  The option name is somewhat ... weird, but
>>> I didn't find enough motivation to change it (or to fiddle with
>>> font-lock-level 3, which would be more in line with other modes).
>> That's a downvote for font-lock levels from me.
> Oh - I guessed that ignoring font-lock-levels was one of the things you
> meant when you wrote that cperl-mode is different than other major modes...

Levels are just not the right concept, because there are many ways to
"measure" and hence order by levels.

E.g. Most of the time, "level" is associated with the amount of effort
to figure out where to highlight, which is related to how much
understanding of the language is needed.  So it ends up providing
"lexical-level" information only at the lowest level and then as you
move up the level it gets more fancy and start accounting for grammar
and then static semantics (e.g. whether an identifier is a reference to
a local or global var).

Personally I want very little highlighting, but I'm usually more
interested in the "higher-level" highlighting, so I end up having to
choose the highest-level and then configuring most faces to be
indistinguishable from `default` :-(

> Sure.  I think that creating custom themes might be a bit beyond what
> *users* of c?perl-mode might want to do (I myself never did that).

I'm not talking about users creating a custom theme.  I'm talking about
cperl-mode providing a custom theme that gets it closer to something
like `perl-mode`.

> Emacs comes with themes which look better than anything I could create -

A cutom theme is nothing more than a set of Custom settings (vars and
faces), and users can enable *several* themes at the same time.  So we
can provide a `perl-mode` theme which just changes `cperl-mode` to look
and behave somewhat like `perl-mode` and that doesn't prevent users from
*also* using `modus-vivendi` or whichever "global color theme" they like.

A "custom theme" is like a minor mode, except that it's more declarative
so it better interacts with other user settings (or other themes which
may partly overlap).

> An overhaul of all of this would take some time, and probably a thick
> skin to weather the storm caused by cperl-mode users who are
> accustomed to the current colors.

I think you're overly pessimistic.  A few tweaks here and there
shouldn't cause too much noise, if they make things more
regular/consistent.

> It is a bit more than _changing_ faces.  In the match part, cperl-mode
> highlights metacharacters (|) as keywords, [] as functions, and
> character classes (\d \D \s and friends) as types.

I guess using `font-lock-function-name-face` for [] does sound quite
wrong, since it plays a very different role from "the place where an
identifier is defined as a function-like thingy", which is what this
face is typically used for elsewhere.
I'd recommend to highlight them as something like keywords as well.
[ BTW, ELisp mode uses `font-lock-regexp-grouping-construct` for some
  of that.  ]

"Types" also sounds odd for \d and friends but the harm is probably
a bit more mild.

> The substitution part can be actual Perl code, and cperl-mode will
> fontify it as Perl code.

IIRC `perl-mode` also tries to fontify the replacement part as Perl code
(when it does contain Perl code).  I can't remember if I managed to make
it work for all the relevant cases, but I haven't heard any complaint
from `perl-mode` users when I made this change, so if `cperl-mode` does
it in more cases, I don't think `perl-mode` users will complain when
switching to `cperl-mode`.


        Stefan

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Mauro Aranda <maurooaranda@gmail.com> writes:

> In addition to what Jens Schmidt said, I can add that:
>
> 1. If I have something like:
> my $some_code = "";
> $some_code.= q(
> my $counter = 0;
> );
>
> If I put point at column 0 of the line "my $counter", and hit TAB, I get
> indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
> options for making this work but I couldn't find anything.

I consider the behavior of perl-mode to be a bug.

Whatever is within the parens of  q(...) is a string, and will be
assigned to the variable $some_code.  By "indenting", perl-mode changes
the value of $some_code by adding spaces.  In my opinion, indenting
should change the optical layout, but not the code!

> 2. While I'm typing the above string, I get messages about string/RE not
> found:
> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
> parentheses 1092 1874)
> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
> parentheses 1092 1918) [2 times]
> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
> parentheses 1092 1962) [2 times]
>
> That's annoying.

The message is technically correct, and generally I consider the ability
of cperl-mode to locate syntax errors useful.  But I understand that it
can be annoying while you're typing (I myself don't see these messages
because I use paredit-mode, but I understand that not everyone wants
this electricity).  I guess that a way to optionally suppress these
messages can be found.

The frequency of this error message exploded with the introduction of
jit-lock-mode - before that cperl-mode didn't scan the code for every
character you type.  I think that backing out of jit-lock-mode would not
be wise.

> So far, my settings for getting a perl-mode experience in cperl-mode,
> with emacs -Q: (taken from a custom file):
> '(cperl-highlight-variables-indiscriminately t)
> '(cperl-indent-level 4)
> '(cperl-indent-parens-as-block t)
> '(cperl-invalid-face 'default)
,>
> '(cperl-array-face ((t (:inherit cperl-hash-face))))
> '(cperl-hash-face ((t (:underline t :inherit
> font-lock-variable-name-face))))
> '(cperl-nonoverridable-face ((t (:inherit default))))

Thanks for collecting this!

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Mauro Aranda]

On 24/9/23 19:21, Harald Jörg wrote:
 > Mauro Aranda <maurooaranda@gmail.com> writes:
 >
 >> In addition to what Jens Schmidt said, I can add that:
 >>
 >> 1. If I have something like:
 >> my $some_code = "";
 >> $some_code.= q(
 >> my $counter = 0;
 >> );
 >>
 >> If I put point at column 0 of the line "my $counter", and hit TAB, I get
 >> indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
 >> options for making this work but I couldn't find anything.
 >
 > I consider the behavior of perl-mode to be a bug.
 >
 > Whatever is within the parens of  q(...) is a string, and will be
 > assigned to the variable $some_code.  By "indenting", perl-mode changes
 > the value of $some_code by adding spaces.  In my opinion, indenting
 > should change the optical layout, but not the code!

I might be completely wrong about this, but I have the feeling that
TAB behaves differently because in perl-mode I see TAB is bound to
indent-for-tab-command (this is not in emacs -Q, just in case it has
something to do with that).

So, TAB is not really indenting (as I said originally, sorry), but
adding a TAB character upon request.  And yes, in this case I want to
change the code, so TAB is not doing anything incorrect here, I think.

 >> 2. While I'm typing the above string, I get messages about string/RE not
 >> found:
 >> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
 >> parentheses 1092 1874)
 >> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
 >> parentheses 1092 1918) [2 times]
 >> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
 >> parentheses 1092 1962) [2 times]
 >>
 >> That's annoying.
 >
 > The message is technically correct, and generally I consider the ability
 > of cperl-mode to locate syntax errors useful.  But I understand that it
 > can be annoying while you're typing (I myself don't see these messages
 > because I use paredit-mode, but I understand that not everyone wants
 > this electricity).  I guess that a way to optionally suppress these
 > messages can be found.

Sure is correct, but I'm used to see messages like those when I have
really screwed up (typically editing an elisp file and making
forward-sexp fail).  Not while I'm typing.  To me, it feels better to
have kind of a visual indication, like when you are entering a string
and everything fontifies as a string so you are reminded you better
close it.

 >> So far, my settings for getting a perl-mode experience in cperl-mode,
 >> with emacs -Q: (taken from a custom file):
 >> '(cperl-highlight-variables-indiscriminately t)
 >> '(cperl-indent-level 4)
 >> '(cperl-indent-parens-as-block t)
 >> '(cperl-invalid-face 'default)
 > ,>
 >> '(cperl-array-face ((t (:inherit cperl-hash-face))))
 >> '(cperl-hash-face ((t (:underline t :inherit
 >> font-lock-variable-name-face))))
 >> '(cperl-nonoverridable-face ((t (:inherit default))))
 >
 > Thanks for collecting this!
 >

You're welcome.  I'm still trying to figure out if there are more
settings to tweak.  Together with the settings that need to be added for
a smooth perl-mode -> cperl-mode transition, they could be placed in a
cperl-perl-mode-users-asylum-theme ;-)

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> 1. If I have something like:
>> my $some_code = "";
>> $some_code.= q(
>> my $counter = 0;
>> );
>>
>> If I put point at column 0 of the line "my $counter", and hit TAB, I get
>> indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
>> options for making this work but I couldn't find anything.
>
> I consider the behavior of perl-mode to be a bug.

The behavior looks right to me, tho I haven't checked the code if it's
by accident (i.e. because of a bug in `perl-mode`) or not.

> Whatever is within the parens of  q(...) is a string, and will be
> assigned to the variable $some_code.  By "indenting", perl-mode changes
> the value of $some_code by adding spaces.  In my opinion, indenting
> should change the optical layout, but not the code!

Agreed in the sense that `indent-according-to-mode` (and
`indent-region`) should not touch this line.
But TAB should, using this part of `indent-line-function`:

    indent-line-function is a variable defined in ‘indent.el’.
    [...]
    If it is called somewhere where it cannot auto-indent, the function
    should return ‘noindent’ to signal that it didn’t.
    [...]

> The message is technically correct, and generally I consider the ability
> of cperl-mode to locate syntax errors useful.  But I understand that it
> can be annoying while you're typing (I myself don't see these messages
> because I use paredit-mode, but I understand that not everyone wants
> this electricity).  I guess that a way to optionally suppress these
> messages can be found.

I think the way we usually solve these tensions is to replace those
error messages with some kind of highlighting (and if necessary
`help-echo` properties to help the user figure out what this
highlighting is for).

E.g. in ELisp mode, a non-terminated string results in a string-highlight
that bleeds into the subsequent code.  And an argument wrongly placed on
the same line as something already indented within a subexpression as
in:

      (foo arg1
           (some sub
                 expression) badarg2)

gets a `font-lock-warning-face` together with a `help-echo`.


        Stefan

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Mauro Aranda <maurooaranda@gmail.com> writes:

> On 24/9/23 19:21, Harald Jörg wrote:
>> Mauro Aranda <maurooaranda@gmail.com> writes:
>>
>>> In addition to what Jens Schmidt said, I can add that:
>>>
>>> 1. If I have something like:
>>> my $some_code = "";
>>> $some_code.= q(
>>> my $counter = 0;
>>> );
>>>
>>> If I put point at column 0 of the line "my $counter", and hit TAB, I get
>>> indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
>>> options for making this work but I couldn't find anything.
>>
>> I consider the behavior of perl-mode to be a bug.
>>
>> Whatever is within the parens of  q(...) is a string, and will be
>> assigned to the variable $some_code.  By "indenting", perl-mode changes
>> the value of $some_code by adding spaces.  In my opinion, indenting
>> should change the optical layout, but not the code!
>
> I might be completely wrong about this, but I have the feeling that
> TAB behaves differently because in perl-mode I see TAB is bound to
> indent-for-tab-command (this is not in emacs -Q, just in case it has
> something to do with that).
>
> So, TAB is not really indenting (as I said originally, sorry), but
> adding a TAB character upon request.  And yes, in this case I want to
> change the code, so TAB is not doing anything incorrect here, I think.

I apologize!  I made a mistake here: I am so addicted to
cperl-tab-always-indent (t by default) that it didn't occur to me to
even *try* to insert a tab character!

And indeed, cperl-mode does not insert a tab character.  Even if
cperl-tab-always-indent is nil, it doesn't insert a character if point
is in the left margin.  This is documented in the docstring of
cperl-indent-command.

This behavior of cperl-mode is weird and it makes sense to find a way to
make <tab> insert a tab character.

(A workaround is <C-q> <tab>, but that isn't convincing)

>>> 2. While I'm typing the above string, I get messages about string/RE not
>>> found:
>>> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
>>> parentheses 1092 1874)
>>> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
>>> parentheses 1092 1918) [2 times]
>>> End of ‘q( ... )’ string/RE not found: (scan-error Unbalanced
>>> parentheses 1092 1962) [2 times]
>>>
>>> That's annoying.
>>
>> The message is technically correct, and generally I consider the ability
>> of cperl-mode to locate syntax errors useful.  But I understand that it
>> can be annoying while you're typing (I myself don't see these messages
>> because I use paredit-mode, but I understand that not everyone wants
>> this electricity).  I guess that a way to optionally suppress these
>> messages can be found.
>
> Sure is correct, but I'm used to see messages like those when I have
> really screwed up (typically editing an elisp file and making
> forward-sexp fail).  Not while I'm typing.  To me, it feels better to
> have kind of a visual indication, like when you are entering a string
> and everything fontifies as a string so you are reminded you better
> close it.

I agree.

Stefan Monnier has suggested how this could be handled in his reply, and
I think this would be an improvement to cperl-mode in general
(independent of using it as a replacement for perl-mode).

>>> So far, my settings for getting a perl-mode experience in cperl-mode,
>>> with emacs -Q: (taken from a custom file):
>>> '(cperl-highlight-variables-indiscriminately t)
>>> '(cperl-indent-level 4)
>>> '(cperl-indent-parens-as-block t)
>>> '(cperl-invalid-face 'default)
>> ,>
>>> '(cperl-array-face ((t (:inherit cperl-hash-face))))
>>> '(cperl-hash-face ((t (:underline t :inherit
>>> font-lock-variable-name-face))))
>>> '(cperl-nonoverridable-face ((t (:inherit default))))
>>
>> Thanks for collecting this!
>
> You're welcome.  I'm still trying to figure out if there are more
> settings to tweak.  Together with the settings that need to be added for
> a smooth perl-mode -> cperl-mode transition, they could be placed in a
> cperl-perl-mode-users-asylum-theme ;-)

Yes, a custom theme seems to be the way to go.
-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> 1. If I have something like:
>>> my $some_code = "";
>>> $some_code.= q(
>>> my $counter = 0;
>>> );
>>>
>>> If I put point at column 0 of the line "my $counter", and hit TAB, I get
>>> indentation in perl-mode.  I don't in cperl-mode.  I tried to look into
>>> options for making this work but I couldn't find anything.
>>
>> I consider the behavior of perl-mode to be a bug.
>
> The behavior looks right to me, tho I haven't checked the code if it's
> by accident (i.e. because of a bug in `perl-mode`) or not.

I was wrong (as explained in my reply to Mauro).  Sorry for the
confusion!

> [...]
>> The message is technically correct, and generally I consider the ability
>> of cperl-mode to locate syntax errors useful.  But I understand that it
>> can be annoying while you're typing (I myself don't see these messages
>> because I use paredit-mode, but I understand that not everyone wants
>> this electricity).  I guess that a way to optionally suppress these
>> messages can be found.
>
> I think the way we usually solve these tensions is to replace those
> error messages with some kind of highlighting (and if necessary
> `help-echo` properties to help the user figure out what this
> highlighting is for).
>
> E.g. in ELisp mode, a non-terminated string results in a string-highlight
> that bleeds into the subsequent code.  And an argument wrongly placed on
> the same line as something already indented within a subexpression as
> in:
>
>       (foo arg1
>            (some sub
>                  expression) badarg2)
>
> gets a `font-lock-warning-face` together with a `help-echo`.

Thanks for this suggestion!  This would be an improvement to cperl-mode
in general.  I didn't know about the help-echo property!

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> Yes, that should be covered.  The option name is somewhat ... weird, but
>>>> I didn't find enough motivation to change it (or to fiddle with
>>>> font-lock-level 3, which would be more in line with other modes).
>>> That's a downvote for font-lock levels from me.
>> Oh - I guessed that ignoring font-lock-levels was one of the things you
>> meant when you wrote that cperl-mode is different than other major modes...
>
> Levels are just not the right concept, because there are many ways to
> "measure" and hence order by levels.

Oh, i see.  And I agree.

> [...]
>> Sure.  I think that creating custom themes might be a bit beyond what
>> *users* of c?perl-mode might want to do (I myself never did that).
>
> I'm not talking about users creating a custom theme.  I'm talking about
> cperl-mode providing a custom theme that gets it closer to something
> like `perl-mode`.

Ah - I understand.  Indeed, this looks like the way to go!

> [...]
>> An overhaul of all of this would take some time, and probably a thick
>> skin to weather the storm caused by cperl-mode users who are
>> accustomed to the current colors.
>
> I think you're overly pessimistic.  A few tweaks here and there
> shouldn't cause too much noise, if they make things more
> regular/consistent.

...and also I have a thick skin, so this won't stop me :)

>> It is a bit more than _changing_ faces.  In the match part, cperl-mode
>> highlights metacharacters (|) as keywords, [] as functions, and
>> character classes (\d \D \s and friends) as types.
>
> I guess using `font-lock-function-name-face` for [] does sound quite
> wrong, since it plays a very different role from "the place where an
> identifier is defined as a function-like thingy", which is what this
> face is typically used for elsewhere.

Yes, that's not a "correct" use of `font-lock-function-name-face` - but
it has been like this for ages.  I think it makes sense to highlight
*all* characters which aren't taken literally in the search pattern
since this is a frequent source of beginner problems.

> I'd recommend to highlight them as something like keywords as well.
> [ BTW, ELisp mode uses `font-lock-regexp-grouping-construct` for some
>   of that.  ]

So this would be a general improvement for cperl-mode!

> "Types" also sounds odd for \d and friends but the harm is probably
> a bit more mild.

I find it actually defendable.  Within the realm of the regular
expressions language, "digits" or "spaces" are sort of "types" (and
there's no dedicated font-lock-regexp-* face for those).

>> The substitution part can be actual Perl code, and cperl-mode will
>> fontify it as Perl code.
>
> IIRC `perl-mode` also tries to fontify the replacement part as Perl code
> (when it does contain Perl code). ...

Oh - I couldn't see it with my example.  Perhaps I miss a setting.

> ... I can't remember if I managed to make it work for all the relevant
> cases, but I haven't heard any complaint from `perl-mode` users when I
> made this change, so if `cperl-mode` does it in more cases, I don't
> think `perl-mode` users will complain when switching to `cperl-mode`.

I agree.

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Mauro Aranda]

On 25/9/23 06:18, Harald Jörg wrote:
 > Stefan Monnier <monnier@iro.umontreal.ca> writes:
 >>> The substitution part can be actual Perl code, and cperl-mode will
 >>> fontify it as Perl code.
 >>
 >> IIRC `perl-mode` also tries to fontify the replacement part as Perl code
 >> (when it does contain Perl code). ...
 >
 > Oh - I couldn't see it with my example.  Perhaps I miss a setting.

Try it with: s{}{}e
Doesn't work with: s///e

bug#66050: Making perl-mode.el obsolete

[Harald Jörg]

Mauro Aranda <maurooaranda@gmail.com> writes:

> On 25/9/23 06:18, Harald Jörg wrote:
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>>> The substitution part can be actual Perl code, and cperl-mode will
>>>> fontify it as Perl code.
>>>
>>> IIRC `perl-mode` also tries to fontify the replacement part as Perl code
>>> (when it does contain Perl code). ...
>>
>> Oh - I couldn't see it with my example.  Perhaps I miss a setting.
>
> Try it with: s{}{}e
> Doesn't work with: s///e

Oh, thanks for the hint!

So, cperl-mode should just keep that feature, even if perl-mode themed.

-- 
Cheers,
haj

bug#66050: Making perl-mode.el obsolete

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> is in the left margin.  This is documented in the docstring of
> cperl-indent-command.

BTW, we should aim to make `cperl-indent-command` obsolete (and keep
the TAB binding untouched).  If there's a difficulty in doing that,
please `M-x report-emacs-bug` requesting enhancements in
`indent-for-tab-command` :-)


        Stefan

bug#66050: Making perl-mode.el obsolete

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > How about this basic plan:

  > First, we make perl-mode.el obsolete on master.

It seems to me that that should be the last step in the plan -- IF we
succeed in making CPerl mode satisfactory for those users (assuming
there still are a significa t number) who prefer Perl mode.

If the plan is to make sure that CPerl satisfies them, we shouldn't
pressure them through hasty degrading of its status.

Also, shouldn't we discuss this on emacs-devel?  It is an
important enough design decision, and NOT a bug fix.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)