XKCD/541 compliance, anyone?

XKCD/541 compliance, anyone?

[Marcin Borkowski]

Hi there,

it is so frustrating to be unable to use paren matching (like in
show-paren mode, or C-M-f & friends) when I use smileys.  And this
*does* happen; comments in, say, elisp-mode seem to be safe, but in
LaTeX smileys do break my workflow.  Try entering this in AUCTeX:

\textbf{Hello :-)}

then put the point at the EOL and press C-M-b.

I have a dream: to have the s-exp functions ignore closing parens,
brackets etc. /after/, say, things matching

[;:B]-?

(this regex should probably cover the most basic cases).

AFAIK, these functions are deeply hardcoded in Emacs; its behavior here
seems to be inborn and instinctive rather than learned and reflective,
so to speak.  (I.e., it is based on the `scan-sexp' function, which is
written in C rather than Elisp.)  I imagine that redefining scan-sexp in
Elisp is possible, but there might be problems with performance.

Did anyone try that?  Am I the only one with this problem, actually?

-- 
Marcin Borkowski               This email was proudly sent
http://mbork.pl                from my Emacs.

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

> I have a dream: to have the s-exp functions ignore closing parens,
> brackets etc. /after/, say, things matching

> [;:B]-?

You can use syntax-propertize-function to change the syntax class of
those parens.


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2014-12-04, at 23:34, Stefan Monnier wrote:

>> I have a dream: to have the s-exp functions ignore closing parens,
>> brackets etc. /after/, say, things matching
>
>> [;:B]-?
>
> You can use syntax-propertize-function to change the syntax class of
> those parens.

Ah.  Do I get it correctly that functions like forward-sexp look at
syntax class of the character, and that you can change the syntax class
of some (but not all) parens in the buffer?  If this is the case, it
would be really cool.

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2014-12-04, at 23:34, Stefan Monnier wrote:

>> I have a dream: to have the s-exp functions ignore closing parens,
>> brackets etc. /after/, say, things matching
>
>> [;:B]-?
>
> You can use syntax-propertize-function to change the syntax class of
> those parens.

Not working.  :-(

I decided to try this out manually.  I put this into my buffer:

\emph{Hello world! ;-)}

changed the mode to LaTeX-mode (from AUCTeX), put the point at the ")"
character, and did

M-: (put-text-property (point) (1+ (point)) 'syntax-table '(1))

After inspection by C-u C-x =, it turned out that nothing like a
"syntax-table" property got its way into the buffer.

Strangeness: if I change the name of the property to, e.g., my-prop,
putting (and getting) the property value works as expected.

Note: I also set parse-sexp-lookup-properties to t; unsurprisingly, it
didn't help.

Looks like either a bug in Emacs, or a bug in my mental model of how
this should work.

The same happens with emacs -Q (though without AUCTeX, just in the stock
Emacs latex-mode).

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Artur Malabarba]

> I decided to try this out manually.  I put this into my buffer:
>
> \emph{Hello world! ;-)}
>
> changed the mode to LaTeX-mode (from AUCTeX), put the point at the ")"
> character, and did
>
> M-: (put-text-property (point) (1+ (point)) 'syntax-table '(1))
>
> After inspection by C-u C-x =, it turned out that nothing like a
> "syntax-table" property got its way into the buffer.
>
> Strangeness: if I change the name of the property to, e.g., my-prop,
> putting (and getting) the property value works as expected.
>

Try that again with font-lock-mode disabled.
Major modes can specify other text properties (besides 'face) for
font-lock-mode to set and reset. It's possible AucTex has font-lock-mode
resetting the' syntax-table property.

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2014-12-06, at 00:41, Artur Malabarba wrote:

>> I decided to try this out manually.  I put this into my buffer:
>>
>> \emph{Hello world! ;-)}
>>
>> changed the mode to LaTeX-mode (from AUCTeX), put the point at the ")"
>> character, and did
>>
>> M-: (put-text-property (point) (1+ (point)) 'syntax-table '(1))
>>
>> After inspection by C-u C-x =, it turned out that nothing like a
>> "syntax-table" property got its way into the buffer.
>>
>> Strangeness: if I change the name of the property to, e.g., my-prop,
>> putting (and getting) the property value works as expected.
>>
>
> Try that again with font-lock-mode disabled.
> Major modes can specify other text properties (besides 'face) for
> font-lock-mode to set and reset. It's possible AucTex has font-lock-mode
> resetting the' syntax-table property.

Thank.  You.  So.  Much, Sir!  Now it works (the manual fiddling, that
is).

I assume that if (=when) I hook into `syntax-propertize-function', it
will work, too.  (I hope so at least!)  It's 00:46 (AM) here in Poland
and I have still more than an hour of work before I go to sleep, so I
will not try that out today, but I will definitely investigate this (and
post about it if it works).

Thanks again!

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2014-12-04, at 23:34, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> I have a dream: to have the s-exp functions ignore closing parens,
>> brackets etc. /after/, say, things matching
>
>> [;:B]-?
>
> You can use syntax-propertize-function to change the syntax class of
> those parens.

OK, so I did try.  This is what I came up with:

(defvar smiley-regex "[:;x>B][,']?-?[]()PD]"
  "This regex should match smileys.")

(defun make-smileys-punctuation (beg end)
  "Look for smileys between BEG and END position in the buffer, and
  change their syntax property to punctuation."
  (goto-char beg)
  (while (re-search-forward smiley-regex end t)
    (put-text-property (match-beginning 0) (match-end 0) 'syntax-table '(1))))

(defun enable-smileys-punctuation ()
  (setq syntax-propertize-function #'make-smileys-punctuation)
  (setq parse-sexp-lookup-properties t))

(add-hook 'text-mode-hook #'enable-smileys-punctuation)

However, it did not work (in text mode); my make-smileys-punctuation
seems not even to get called.  Curiously enough, it seems to work in
e.g. Elisp mode - but then again, not in message mode, for instance.

What might be going on here?

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

> (defvar smiley-regex "[:;x>B][,']?-?[]()PD]"
>   "This regex should match smileys.")

> (defun make-smileys-punctuation (beg end)
>   "Look for smileys between BEG and END position in the buffer, and
>   change their syntax property to punctuation."
>   (goto-char beg)
>   (while (re-search-forward smiley-regex end t)
>     (put-text-property (match-beginning 0) (match-end 0) 'syntax-table '(1))))

You can also use

   (syntax-propertize-rules (smiley-regex (0 ".")))

> (defun enable-smileys-punctuation ()
>   (setq syntax-propertize-function #'make-smileys-punctuation)

Beware: syntax-propertize-function might already be in use, in which
case you should probably use add-function to combine the two.

>   (setq parse-sexp-lookup-properties t))

This is not necessary, it will be set by the syntax-propertize function.

> However, it did not work (in text mode); my make-smileys-punctuation
> seems not even to get called.

Right, syntax-propertization is done lazily, so if nothing calls
syntax-propertize, then that's that.  Usually the main triggers for
syntax-propertize are syntax-ppss and font-lock, but neither is likely
to be used in text-mode.  So you'll probably need to arrange for font-lock to
be enabled *and* for font-lock-keywords-only not to be set to t.

> but then again, not in message mode, for instance.

Probably because font-lock-keywords-only is set to t, so font-lock
doesn't end up calling syntax-ppss.


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-01, at 18:07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> You can also use
>
>    (syntax-propertize-rules (smiley-regex (0 ".")))

Thanks, I didn't know about this function.  Very handy.

>> (defun enable-smileys-punctuation ()
>>   (setq syntax-propertize-function #'make-smileys-punctuation)
>
> Beware: syntax-propertize-function might already be in use, in which
> case you should probably use add-function to combine the two.

Hm.  My Emacs (24.3) doesn't have anything called add-function.  I
checked on the interwebs, and it seems it's part of the new advice
system.  I'll have to upgrade finally.

My question: wouldn't it be reasonable to change
syntax-propertize-function into a /list/ of functions?  Right now the
manual says that it should be set up by major mode to achieve some
special effects.  In my case, however, it would be more appropriate to
use a minor mode (possibly even a global minor mode).  Wouldn't
syntax-propertize-functions be more appropriate here?

OTOH, one your answer on Emacs.SE says explicitly that the trend is to
go away from "<foo>-functions" and use "<foo>-function" instead.  What
is the rationale?  Do I guess correctly that it will be easier to
/remove/ things added this way?

>>   (setq parse-sexp-lookup-properties t))
>
> This is not necessary, it will be set by the syntax-propertize function.

I checked it, and I see it's even better, it will make it buffer-local.
Again: very handy.  BTW: do I guess correctly that the reason that
parse-sexp-lookup-properties is nil by default (and the reason for its
existence in the first place) is performance issues?  If yes, is the
difference between having it nil and t substantial on modern hardware?

>> However, it did not work (in text mode); my make-smileys-punctuation
>> seems not even to get called.
>
> Right, syntax-propertization is done lazily, so if nothing calls
> syntax-propertize, then that's that.  Usually the main triggers for
> syntax-propertize are syntax-ppss and font-lock, but neither is likely
> to be used in text-mode.  So you'll probably need to arrange for font-lock to
> be enabled *and* for font-lock-keywords-only not to be set to t.

Well, I did not understand everything you wrote here.  I guess I will
just have to RTFM; I vaguely remember reading about "lazy font-lock" 15
years ago, when I was reading (almost) the whole Emacs manual.  I guess
this is something similar.  I'll write back to the list if I don't get
it from the manual.

>> but then again, not in message mode, for instance.
>
> Probably because font-lock-keywords-only is set to t, so font-lock
> doesn't end up calling syntax-ppss.

Yes it is, though I don't (yet) understand what this means.  See above.

>         Stefan

PS. I guess that I will have to turn the knowledge from this exchange to
a lengthy blog post.  As I said in some other thread: I've been using
Emacs for about 15 years now (which is not much by Emacs standards, but
/a lot/ by so-called /modern/ standards), and still learning /a lot/...

Thank you so much!

-- 
Marcin Borkowski               This email was proudly sent
http://mbork.pl                from my Emacs.

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

>> Beware: syntax-propertize-function might already be in use, in which
>> case you should probably use add-function to combine the two.
> Hm.  My Emacs (24.3) doesn't have anything called add-function.  I
> checked on the interwebs, and it seems it's part of the new advice
> system.  I'll have to upgrade finally.

Tho you probably only need this syntax-propertize-smileys thingy for
modes which don't use syntax-propertize-function yet, so maybe
add-function is not really necessary.

> My question: wouldn't it be reasonable to change
> syntax-propertize-function into a /list/ of functions?

At this point, it wouldn't be reasonable, no: add-function works as well
already.  Also, combining various functions there doesn't work quite as
easily as it sounds.  For example, in your case, it works OK to put your
function before the major mode's function, but doing it the other way
around wouldn't always work right.  For more general
syntax-propertize-functions, you can't just run them in turn.

> I checked it, and I see it's even better, it will make it buffer-local.
> Again: very handy.  BTW: do I guess correctly that the reason that
> parse-sexp-lookup-properties is nil by default (and the reason for its
> existence in the first place) is performance issues?  If yes, is the
> difference between having it nil and t substantial on modern hardware?

No, the difference is negligible.  It's a historical left-over.

>>> However, it did not work (in text mode); my make-smileys-punctuation
>>> seems not even to get called.
>> Right, syntax-propertization is done lazily, so if nothing calls
>> syntax-propertize, then that's that.  Usually the main triggers for
>> syntax-propertize are syntax-ppss and font-lock, but neither is likely
>> to be used in text-mode.  So you'll probably need to arrange for font-lock
>> to be enabled *and* for font-lock-keywords-only not to be set to t.
> Well, I did not understand everything you wrote here.  I guess I will
> just have to RTFM; I vaguely remember reading about "lazy font-lock" 15
> years ago,

This is unrelated.  I used "lazy" in the general sense of "do things
as late as possible".

>> Probably because font-lock-keywords-only is set to t, so font-lock
>> doesn't end up calling syntax-ppss.
> Yes it is, though I don't (yet) understand what this means.  See above.

This has to do with the way font-lock handles strings and comments
separately from other highlighting.  In message-mode, this special
strings-and-comments highlighting is disabled, yet, that's the one that
normally triggers syntax-propertize.

You could also use something like

   (defun my-call-syntax-propertize (limit)
     (syntax-propertize limit)
     (goto-char limit)
     nil)
   (font-lock-add-keywords nil '((my-call-syntax-propertize)))

> Thank you so much!

My pleasure,


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-02, at 05:07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>>> Beware: syntax-propertize-function might already be in use, in which
>>> case you should probably use add-function to combine the two.
>> Hm.  My Emacs (24.3) doesn't have anything called add-function.  I
>> checked on the interwebs, and it seems it's part of the new advice
>> system.  I'll have to upgrade finally.
>
> Tho you probably only need this syntax-propertize-smileys thingy for
> modes which don't use syntax-propertize-function yet, so maybe
> add-function is not really necessary.

Not sure if I understand.

I don't need my smileys thing for programming modes – here, smileys
should appear only in strings/comments, and they don't do any harm
there.  Where I need it is mainly text, org, message and (la)tex modes.

>> My question: wouldn't it be reasonable to change
>> syntax-propertize-function into a /list/ of functions?
>
> At this point, it wouldn't be reasonable, no: add-function works as well
> already.  Also, combining various functions there doesn't work quite as
> easily as it sounds.  For example, in your case, it works OK to put your
> function before the major mode's function, but doing it the other way
> around wouldn't always work right.  For more general
> syntax-propertize-functions, you can't just run them in turn.

OK.

>> I checked it, and I see it's even better, it will make it buffer-local.
>> Again: very handy.  BTW: do I guess correctly that the reason that
>> parse-sexp-lookup-properties is nil by default (and the reason for its
>> existence in the first place) is performance issues?  If yes, is the
>> difference between having it nil and t substantial on modern hardware?
>
> No, the difference is negligible.  It's a historical left-over.

That's what I thought, more or less.

>>>> However, it did not work (in text mode); my make-smileys-punctuation
>>>> seems not even to get called.
>>> Right, syntax-propertization is done lazily, so if nothing calls
>>> syntax-propertize, then that's that.  Usually the main triggers for
>>> syntax-propertize are syntax-ppss and font-lock, but neither is likely
>>> to be used in text-mode.  So you'll probably need to arrange for font-lock
>>> to be enabled *and* for font-lock-keywords-only not to be set to t.
>> Well, I did not understand everything you wrote here.  I guess I will
>> just have to RTFM; I vaguely remember reading about "lazy font-lock" 15
>> years ago,
>
> This is unrelated.  I used "lazy" in the general sense of "do things
> as late as possible".

I see.

>>> Probably because font-lock-keywords-only is set to t, so font-lock
>>> doesn't end up calling syntax-ppss.
>> Yes it is, though I don't (yet) understand what this means.  See above.
>
> This has to do with the way font-lock handles strings and comments
> separately from other highlighting.  In message-mode, this special
> strings-and-comments highlighting is disabled, yet, that's the one that
> normally triggers syntax-propertize.

Again: I'll have to read the manual.

> You could also use something like
>
>    (defun my-call-syntax-propertize (limit)
>      (syntax-propertize limit)
>      (goto-char limit)
>      nil)
>    (font-lock-add-keywords nil '((my-call-syntax-propertize)))

This idea might come in handy, since the next step is to mark smileys
with gray;-).

>> Thank you so much!
>
> My pleasure,
>
>         Stefan

Thanks again!

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-01, at 18:07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> (defvar smiley-regex "[:;x>B][,']?-?[]()PD]"
>>   "This regex should match smileys.")
>
>> (defun make-smileys-punctuation (beg end)
>>   "Look for smileys between BEG and END position in the buffer, and
>>   change their syntax property to punctuation."
>>   (goto-char beg)
>>   (while (re-search-forward smiley-regex end t)
>>     (put-text-property (match-beginning 0) (match-end 0) 'syntax-table '(1))))
>
> You can also use
>
>    (syntax-propertize-rules (smiley-regex (0 ".")))

OK, so I started reading about this.

I can see that syntax-propertize-rules is a huuuge macro I'm afraid even
to look at from distance;-).  Do I get it correctly that it "outputs" a
function, so that I can write

(setq syntax-propertize-function (syntax-propertize-rules (...)))

?

>> (defun enable-smileys-punctuation ()
>>   (setq syntax-propertize-function #'make-smileys-punctuation)
>
> Beware: syntax-propertize-function might already be in use, in which
> case you should probably use add-function to combine the two.

Yes, I know that; I wanted just to experiment a bit to start with.

> Right, syntax-propertization is done lazily, so if nothing calls
> syntax-propertize, then that's that.  Usually the main triggers for
> syntax-propertize are syntax-ppss and font-lock, but neither is likely
> to be used in text-mode.  So you'll probably need to arrange for font-lock to
> be enabled *and* for font-lock-keywords-only not to be set to t.

I have global font lock mode, so font locking is on everywhere.

>> but then again, not in message mode, for instance.
>
> Probably because font-lock-keywords-only is set to t, so font-lock
> doesn't end up calling syntax-ppss.

Now that is strange: it seems that font-lock-keywords-only is set to t
in mu4e-message mode (where I'm now), but grepping through mu4e sources
didn't give any results (nor did isearching message.el).  I'll have to
look into this further (check other modes etc.)

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

>>>> Beware: syntax-propertize-function might already be in use, in which
>>>> case you should probably use add-function to combine the two.
>>> Hm.  My Emacs (24.3) doesn't have anything called add-function.  I
>>> checked on the interwebs, and it seems it's part of the new advice
>>> system.  I'll have to upgrade finally.
>> Tho you probably only need this syntax-propertize-smileys thingy for
>> modes which don't use syntax-propertize-function yet, so maybe
>> add-function is not really necessary.
> Not sure if I understand.
> I don't need my smileys thing for programming modes

Exactly: and most of the major modes which set
syntax-propertize-function are programming modes.

> Where I need it is mainly text, org, message and (la)tex modes.

Of those 4 modes, I think only latex-mode sets syntax-propertize-function.


        Stefan

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

> Do I get it correctly that it "outputs" a function, so that I can
> write
> (setq syntax-propertize-function (syntax-propertize-rules (...)))
> ?

That's right.

> I have global font lock mode, so font locking is on everywhere.

Nope: global-font-lock-mode only enabled font-lock in those modes where
there are font-lock keywords defined.

>>> but then again, not in message mode, for instance.
>> Probably because font-lock-keywords-only is set to t, so font-lock
>> doesn't end up calling syntax-ppss.
> Now that is strange: it seems that font-lock-keywords-only is set to t
> in mu4e-message mode (where I'm now), but grepping through mu4e sources
> didn't give any results (nor did isearching message.el).  I'll have to
> look into this further (check other modes etc.)

That's because font-lock-keywords-only is set indirectly via the 2nd
element of `font-lock-defaults'.


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-03, at 02:47, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> Where I need it is mainly text, org, message and (la)tex modes.
>
> Of those 4 modes, I think only latex-mode sets syntax-propertize-function.

Not even one of them!  (I didn't check the built-in LaTeX mode, only
AUCTeX's one.)

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-01, at 18:07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Beware: syntax-propertize-function might already be in use, in which
> case you should probably use add-function to combine the two.

I think I'll just use the easiest solution: signal an error or warning
if syntax-propertize-function is non-nil.  As I wrote, in modes this is
going to be useful it’s probably nil anyway.

>> However, it did not work (in text mode); my make-smileys-punctuation
>> seems not even to get called.
>
> Right, syntax-propertization is done lazily, so if nothing calls
> syntax-propertize, then that's that.  Usually the main triggers for
> syntax-propertize are syntax-ppss and font-lock, but neither is likely
> to be used in text-mode.  So you'll probably need to arrange for font-lock to
> be enabled *and* for font-lock-keywords-only not to be set to t.

Now this question remains: how to ensure that syntax-propertize is
getting called everywhere?  Is (syntax-propertize (point-max)) the best
idea?  (In fact, probably rather something like (save-restriction
(widen) (syntax-propertize (point-max))) would be better.)  This way,
I wouldn’t even have to ensure that font-lock is enabled or
font-lock-keywords-only is set to nil.

The reason I'm asking is that I do not want to assume that font-lock is
in use; otherwise I could also e.g. call (font-lock-fontify-buffer).

Curiously enough, doing M-: (syntax-ppss) RET did the trick, but only
for a few lines.  I guess this is because of the “laziness” you
mentioned.

Now one problem remains.  I’d like to mark all the smileys with a greyed
font.  Are “font lock keywords” the right tool for that?  (Also, it
might be interesting to use the “prettifying” mode to display them as
Unicode smileys.)

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

> Now this question remains: how to ensure that syntax-propertize is
> getting called everywhere?

Depends what you mean by "everywhere".  It "should" be called internally
by things like forward-sexp, but currently isn't.
What you can do is something like

   (jit-lock-register (lambda (from to) (syntax-propertize to)) t)

[ This needs to be called individually in each buffer.  ]
It will make sure that all the visible text has been syntax-propertized,
regardless of font-lock-mode.


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-07, at 23:37, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> Now this question remains: how to ensure that syntax-propertize is
>> getting called everywhere?
>
> Depends what you mean by "everywhere".  It "should" be called internally

Everywhere in the buffer.

> by things like forward-sexp, but currently isn't.
> What you can do is something like
>
>    (jit-lock-register (lambda (from to) (syntax-propertize to)) t)
>
> [ This needs to be called individually in each buffer.  ]
> It will make sure that all the visible text has been syntax-propertized,
> regardless of font-lock-mode.

Do I get it correctly that it's better than my solution (basically,
(syntax-propertize (point-max))) because it is faster for large buffer
("lazy smilification";-))?

And "calling in each buffer" will probably mean installing it in the
hook for modes I want it to work in.

(BTW, I love the fact that in the above sentence the closing paren of
the smiley is /not/ treated as the counterpart of the opening paren at
the beginning of the line.  Now that smileys start to "work" for me, I'm
going to use them more often :-) :-) ;-P).

>         Stefan

Thanks a lot

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

> Do I get it correctly that it's better than my solution (basically,
> (syntax-propertize (point-max)))

The problem with your solution is "when?".


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-08, at 02:49, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> Do I get it correctly that it's better than my solution (basically,
>> (syntax-propertize (point-max)))
>
> The problem with your solution is "when?".

Say, in text-mode-hook (or similar)?

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

>>> Do I get it correctly that it's better than my solution (basically,
>>> (syntax-propertize (point-max)))
>> The problem with your solution is "when?".
> Say, in text-mode-hook (or similar)?

No, that would be "where".  Then "when" is something like "after opening
the file", "after every command", "after a buffer modification", ...


        Stefan

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

On 2015-01-08, at 04:11, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>>>> Do I get it correctly that it's better than my solution (basically,
>>>> (syntax-propertize (point-max)))
>>> The problem with your solution is "when?".
>> Say, in text-mode-hook (or similar)?
>
> No, that would be "where".  Then "when" is something like "after opening
> the file", "after every command", "after a buffer modification", ...

Ah.  I thought that if I set up this syntax-propertize-function thing,
things will happen after buffer modification automatically.  (My –
admittedly limited – tests showed that this was indeed the case.)

I have to say that I'm more and more embarassed to ask somewhat stupid,
RTFM-style questions here.  However, the part of the manual about syntax
parsing is not very verbose.  Should I read the parts about font-lock to
understand better what's going on here?

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Marcin Borkowski]

Hi all,

so finally I sat down and finished my smiley code.  I have still three
questions left, though.  But first – the (deceptively simple) code.

--8<---------------cut here---------------start------------->8---
(defvar smiley-regex "[:;>B][,']?-?[][()]"
  "This regex should match smileys, and only them.")

(defface smiley '((t :foreground "gray"))
  "Face for displaying smileys.")

(setq parse-sexp-lookup-properties t)
(add-to-list 'font-lock-extra-managed-props 'syntax-table)

(defconst smiley-props `(face smiley syntax-table ,(string-to-syntax ".")))

(define-minor-mode xkcd-541-mode
  "Toggle XKCD/541 mode.  In this mode, smileys (as defined by
smiley-regex) are treated as punctuation by Emacs movement
commands etc.  In particular, parens in smileys are skipped by
paren-matching commands."
  :lighter " :)"
  (if xkcd-541-mode
      (font-lock-add-keywords nil `((,smiley-regex . smiley-props)))
    (font-lock-remove-keywords nil `((,smiley-regex . smiley-props)))))
--8<---------------cut here---------------end--------------->8---

And now the questions.

1. In this post:
http://lists.gnu.org/archive/html/help-gnu-emacs/2015-01/msg00005.html
I noticed that syntax-propertize makes parse-sexp-lookup-properties
buffer-local.  OTOH, in the next post
(http://lists.gnu.org/archive/html/help-gnu-emacs/2015-01/msg00008.html),
Stefan Monnier confirmed my suspicion that the performance hit from
setting it globally to t is negligible.  Since I ended up not using
syntax-propertize – I think font-lock is better, especially that I also
want my smileys to be greyed out – I have to set
parse-sexp-lookup-properties manually.  Should I do it in my mode toggle
function, remembering the previous value and resetting it after
disabling the mode?  (More or less the same question concerns
font-lock-extra-managed-props.)

2. After enabling xkcd-541-mode, I have

,----
| (t
|  (("[:;>B][,']?-?[][()]" . smiley-props))
|  ("[:;>B][,']?-?[][()]"
|   (0 smiley-props)))
`----

as the value of font-lock-keywords.  Why does that look that strange?
Moreover, after /disabling/ the mode, the value of font-lock-keywords is
(t nil).  Strange, isn’t it?  Shouldn’t it be just nil?

3. The mode works – sort of, since there is one (a bit annoying) thing.
When I turn it on, and type this:

(hello world;-))

sometimes the opening paren is highlighted when typing the smiley (this
happens e.g. in emacs -Q after loading that mode), and sometimes not
(e.g. while I’m writing this message).  Why might that be so?

Best,
Marcin




On 2014-12-04, at 22:16, Marcin Borkowski <mbork@wmi.amu.edu.pl> wrote:

> Hi there,
>
> it is so frustrating to be unable to use paren matching (like in
> show-paren mode, or C-M-f & friends) when I use smileys.  And this
> *does* happen; comments in, say, elisp-mode seem to be safe, but in
> LaTeX smileys do break my workflow.  Try entering this in AUCTeX:
>
> \textbf{Hello :-)}
>
> then put the point at the EOL and press C-M-b.
>
> I have a dream: to have the s-exp functions ignore closing parens,
> brackets etc. /after/, say, things matching
>
> [;:B]-?
>
> (this regex should probably cover the most basic cases).
>
> AFAIK, these functions are deeply hardcoded in Emacs; its behavior here
> seems to be inborn and instinctive rather than learned and reflective,
> so to speak.  (I.e., it is based on the `scan-sexp' function, which is
> written in C rather than Elisp.)  I imagine that redefining scan-sexp in
> Elisp is possible, but there might be problems with performance.
>
> Did anyone try that?  Am I the only one with this problem, actually?


-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: XKCD/541 compliance, anyone?

[Stefan Monnier]

> Since I ended up not using syntax-propertize – I think font-lock is
> better, especially that I also want my smileys to be greyed out –
> I have to set parse-sexp-lookup-properties manually.

This is OK as long as you don't use it in a mode that uses
syntax-propertize (otherwise your code and the mode's syntax-propertize
will step on each other's toes).


        Stefan