Re: 2 character comment starter bug

Re: 2 character comment starter bug

[Stefan Monnier]

[ Please keep the discussions on the mailing-list. ]

>> >     (modify-syntax-entry ?\= "_ b12" st) ; comment start == 
>> 
>> Yes, it seems the problem is that your 2-char comment  sequence is made
>> of symbol-chars, so there are cases where the code does things  like "oh,
>> here's a symbol, let's skip it" without checking whether some of the
>> chars that compose the symbol happen to also be a comment-marker.
>> 
>> Does your = char really need to have "symbol" syntax (i.e. "_") or
>> could it have punctuation syntax instead (i.e. ".") ?

> Punctuation syntax seems to cause all kinds of problems.  The =
> character is commonly used as the first character in in
> filenames.

But does it matter in such a case whether it has punctuation syntax or
symbol syntax?  Do you also give symbol syntax to the / directory separator
as well?

> It's also part of several operators such as := and ':=' and '=:' which
> would behave quite oddly without proper syntax.

Traditionally, punctuation syntax has been used specifically for things like
the above.  So, I'd say that punctuation *is* the proper syntax.  If you use
symbol syntax for those chars, things like M-C-f risk skipping over
"foo:=bar" in "foo:=bar + 1", rather than just skipping over "foo".

> It sounds like you are saying it might be a problem to fix the code.

Yes: it might take a bit of work; it risks slowing down syntax-based
operationd in all buffers; and it could introduce bugs in other languages
where the current behavior is closer to what we want (after all, if you
define your language using lex and you say that a symbol can be [a-z=_]+
and a comment starter is ==, your lexer will take `foo==' to be a symbol
and won't see the comment starter in it).

The current behavior is buggy (it doesn't behave consistently between
things like forward-sexp, backward-sexp, and parse-partial-sexp).

But before someone can convince me to try and fix these bugs, they should
first make a good case that the way they setup their syntax-tables is well
thought out.


        Stefan

RE: 2 character comment starter bug

[Bielawski, Richard G.]

> >> >     (modify-syntax-entry ?\= "_ b12" st) ; comment start == 
> >> 
> >> Yes, it seems the problem is that your 2-char comment  
> sequence is made
> >> of symbol-chars, so there are cases where the code does 
> things  like "oh,
> >> here's a symbol, let's skip it" without checking whether 
> some of the
> >> chars that compose the symbol happen to also be a comment-marker.
> >> 
> >> Does your = char really need to have "symbol" syntax (i.e. "_") or
> >> could it have punctuation syntax instead (i.e. ".") ?
> 
> > Punctuation syntax seems to cause all kinds of problems.  The =
> > character is commonly used as the first character in in
> > filenames.
> 
> But does it matter in such a case whether it has punctuation syntax or
> symbol syntax?  Do you also give symbol syntax to the / 
> directory separator as well?

A / is an illegal character in any part of a filename on this system.
In this language / / are most often seen as paired delimiters.

> > It's also part of several operators such as := and ':=' and 
> '=:' which
> > would behave quite oddly without proper syntax.
> 
> Traditionally, punctuation syntax has been used specifically 
> for things like
> the above.  So, I'd say that punctuation *is* the proper 
> syntax.  If you use
> symbol syntax for those chars, things like M-C-f risk skipping over
> "foo:=bar" in "foo:=bar + 1", rather than just skipping over "foo".

[#set foo [bar]] and [#set foo [#compute bar + 1]] would be required
to accomplish the operations your examples seem to describe. 
If I see ':=' in this language I would almost certainly want it 
treated as a word.

> 
> > It sounds like you are saying it might be a problem to fix the code.
> 
> Yes: it might take a bit of work; it risks slowing down syntax-based
> operations in all buffers; and it could introduce bugs in 
> other languages
> where the current behavior is closer to what we want (after 
> all, if you
> define your language using lex and you say that a symbol can 
> be [a-z=_]+
> and a comment starter is ==, your lexer will take `foo==' to 
> be a symbol
> and won't see the comment starter in it).

In this language == always begins a comment no matter where it is
found unless a ~ precedes it.  There are no exceptions.

> 
> The current behavior is buggy (it doesn't behave consistently between
> things like forward-sexp, backward-sexp, and parse-partial-sexp).
> 
> But before someone can convince me to try and fix these bugs, 
> they should
> first make a good case that the way they setup their 
> syntax-tables is well
> thought out.
> 

If the fact that the problem exists reflects the standard by
which `well thought out' is measured you're being a pushover.

I suppose, since it's possible for me to work around this problem
by using syntactic-keywords to handle the comments I would prefer 
problems for which there are no workarounds be addressed first.  
If it will remain this way for any length of time though, I do
think it should be documented somewhere.  Even if only on a known
problems list.

Re: 2 character comment starter bug

[Stefan]

>> But does it matter in such a case whether it has punctuation syntax or
>> symbol syntax?  Do you also give symbol syntax to the / 
>> directory separator as well?

> A / is an illegal character in any part of a filename on this system.
> In this language / / are most often seen as paired delimiters.

Then, what about !, @, #, $, %, ^, &, *, _, +, ,, <, > ., ?, :, ;, |, ... ?

> [#set foo [bar]] and [#set foo [#compute bar + 1]] would be required
> to accomplish the operations your examples seem to describe.
> If I see ':=' in this language I would almost certainly want it
> treated as a word.

Let me ask the question more directly, then: is "foo=bar" one token
or three?

> In this language == always begins a comment no matter where it is
> found unless a ~ precedes it.  There are no exceptions.

Yes, you made that clear in your original message.

>> The current behavior is buggy (it doesn't behave consistently between
>> things like forward-sexp, backward-sexp, and parse-partial-sexp).
>> 
>> But before someone can convince me to try and fix these bugs,  they
>> should first make a good case that the way they setup their
>> syntax-tables is well thought out.

> If the fact that the problem exists reflects the standard by
> which `well thought out' is measured you're being a pushover.

I don't understand this paragraph, sorry.  But just to make it clear: if you
read my above text, you'll note that I do not claim these are not bugs
and/or shouldn't be fixed.

> If it will remain this way for any length of time though, I do
> think it should be documented somewhere.  Even if only on a known
> problems list.

Patches welcome,


        Stefan