bug#43558: [PATCH]: Fix (forward-comment 1) when end delimiter is escaped.

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> > As already said, this is a(n ugly) workaround.  syntax.c should handle
>> > comments in all their generality.  With a bit of consideration, the
>> > method to do this is clear:
>> In my world, it's quite normal for a specific language's lexical rules
>> not to line up 100% with syntax tables (whether for strings, comments,
>> younameit).  I don't see anything very special here.
> Normally when there's a mismatch, it's because a character is
> syntactically ambiguous.  There's nothing syntax.c can do about this.

Oh, no, there are many more situations than just "a character is
syntactically ambiguous" (or alternatively you could argue that all
cases are "a character is syntactically ambiguous", including your cases
of escaped newline and escaped */).

>> A `syntax-propertize` rule for "\*/" should be very easy to implement
>> and fairly cheap since the regexp is simple and will almost never match.
> Well, the rule would actually be for escaped newlines,

It doesn't have to be if you set `comment-end-can-be-escaped` to non-nil,
in which case you only need to tweak the \*/ case, AFAICT.

> but this would be quite expensive (compared with a syntax.c solution)
> since every comment near a change region would need scanning at
> each change.

I don't know what you mean by scanning, but yes you'd need to search for
all "\\\\\n" or "\\\\\\*/" (depending on how you set
`comment-end-can-be-escaped) and mark the second char accordingly.
Seems pretty cheap in either case.

> I've hacked up a working, but as yet unsatisfactory, change to syntax.c.
> It is surely better, where possible, to fix bugs at their point of
> causation rather than by workarounds elsewhere.

I don't think it's a bug in `syntax.c`.  `syntax.c` is not defined to
support the syntax of C, it's only defined to handle a particular set of
comment and string styles, which correspond to a common subset of what
is in use in most languages, but IME most languages need some extra
tweaks handled via the `syntax-table` text property.  It's only
a question of time until we add a `syntax-propertize-function` for Elisp
mode to properly handle some corner cases, for example.

>> I do think it would be good to handle this without `syntax-table`
>> text-property hacks, but I think that should come with an overhaul of
>> syntax.c based on a major-mode provided DFA (or something like that) so
>> it can accommodate all the various oddball cases without even the need
>> to introduce the notion of escaping comment markers.
> That sounds almost more like a rewrite than an overhaul.

Tomato tomahto.

> We would need to make sure that this wouldn't run more slowly than the
> current syntax.c/Lisp combination.

I don't think that would be required, as long as it runs fast enough.
In any case, the resulting performance is probably not the main worry
(I suspect it will/would be easy to make it fast enough).


        Stefan