Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss)

Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss)

[Daniel Colascione]

On Jul 27, 2008, at 10:50 AM, Alan Mackenzie wrote:
> What I think really needs doing is to make this function  
> bulletproof: It
> should work on narrowed buffers, it should give reliable elements 2  
> and
> 6, its cache should be cleared when functions like `modify-syntax- 
> entry'
> are called or parse-sexp-lookup-properties is changed, and the cache
> should be bound to nil on `with-syntax-table'.  I actually think it
> could be useful to maintain several parallel caches, each for a
> different syntax-table (or an equivalence class of syntax tables).   
> And
> so on.  Basically, I would like `(syntax-ppss)' to tell me with 100%
> reliability, no ifs, no buts, whether I am at top-level, in a comment,
> or in a string.

Such a thing would have to live on the C side of things, right? (With  
the proliferation of with-this and inhibit-that options available to  
Lisp, I don't see how one can easily and robustly catch all buffer  
modification. Not to mention that no matter which of before-change- 
functions and after-change-functions you used, you could still race  
against other functions using the same facility.)

If this perfectly caching parse-partial-sexp lives in C anyway, why  
not just call it parse-partial-sexp? Optimize parse-partial-sexp for  
the case of start being 1 or (point-min). syntax-ppss becomes a simple  
wrapper. Not only would it be possible to robustly catch all buffer  
and context modification, but by optimizing the existing function, all  
existing users would automatically win. I'd offer to write a patch,  
but I don't know the core well enough to know how to "easily and  
robustly catch all buffer modification".

> Also, Lennart is asking for it to work nicely with multiple major  
> modes.
> Surely this would be a Good Thing.  Files containing several major  
> modes
> are commonplace (awk or sed embedded within a shell script, html
> embedded within php, ....).

After several attempts at using and understanding multiple major mode  
facilities, I'm convinced the only way forward is core support for the  
concept. Lennart's done a fine job with what's in Emacs currently. But  
anything involving multiple major modes today is a quivering mound of  
hacks. All the work Lennart's had to do to get modes playing nice with  
each other is a testament to that.

Maybe a core solution could be something like this: in a given buffer,  
each character has a chunk-name character property. You'd buffer- 
locally map chunk names to major modes. For each chunk name, create a  
buffer containing just the text assigned to that chunk. Make the major- 
mode the major mode for the chunk buffer, and let that major-mode  
handle fontification, keybindings, and so on. In the main buffer,  
assemble the various bits from the chunk-buffers and allow the user to  
navigate the combined buffer normally.

Keybindings with point at a particular character would just be the  
keybindings present in that character's chunk-buffer. If you need  
special keybindings common across all chunk buffers, just bind the key  
in all the chunk buffers. If a given chunk needs placeholder text to  
represent text of some other chunk, it should be possible add it to  
that chunk buffer without affecting any of the others.

Anyway, this scheme is:

1) Robust - no messing around with variables, no tweaking fontification
2) Backwards compatible - a major-mode doesn't need to know it's being  
used this way
3) Versatile - you can compose arbitrary modes this way, even  
recursively
4) Conceptually simple (I hope)

Any thoughts?

Re: Better parse-partial-sexp; multiple major modes

[Lennart Borgman (gmail)]

Daniel Colascione wrote:
>> Also, Lennart is asking for it to work nicely with multiple major modes.
>> Surely this would be a Good Thing.  Files containing several major modes
>> are commonplace (awk or sed embedded within a shell script, html
>> embedded within php, ....).
> 
> After several attempts at using and understanding multiple major mode
> facilities, I'm convinced the only way forward is core support for the
> concept. Lennart's done a fine job with what's in Emacs currently. But
> anything involving multiple major modes today is a quivering mound of
> hacks. All the work Lennart's had to do to get modes playing nice with
> each other is a testament to that.

Are you suggesting that you have problems using MuMaMo today? If so then
please report it as a bug.

> Maybe a core solution could be something like this: in a given buffer,
> each character has a chunk-name character property. You'd buffer-locally
> map chunk names to major modes. For each chunk name, create a buffer
> containing just the text assigned to that chunk. Make the major-mode the
> major mode for the chunk buffer, and let that major-mode handle
> fontification, keybindings, and so on. In the main buffer, assemble the
> various bits from the chunk-buffers and allow the user to navigate the
> combined buffer normally.

This was an idea I played with at the beginning when I wrote mumamo.el.

I am afraid I think the concepts involved (like buffer local etc) must
be considered quite a bit more first before going in this direction. It
is unclear to me now whether chunk-buffers really would be of any help.
They might be, but I am not sure.

> Keybindings with point at a particular character would just be the
> keybindings present in that character's chunk-buffer. If you need
> special keybindings common across all chunk buffers, just bind the key
> in all the chunk buffers. If a given chunk needs placeholder text to
> represent text of some other chunk, it should be possible add it to that
> chunk buffer without affecting any of the others.
> 
> Anyway, this scheme is:
> 
> 1) Robust - no messing around with variables, no tweaking fontification

Unfortunately I do not think that will hold.

I think the way to go is "interface style". font-lock has some good
support for making multiple major modes possible.

> 2) Backwards compatible - a major-mode doesn't need to know it's being
> used this way

Are you aware of that a major mode does not need to know anything when
used in MuMaMo?

> 3) Versatile - you can compose arbitrary modes this way, even recursively

The main difficulty with sub-modes in sub-modes is stability. How do you
find the sub-modes in sub-modes after buffer changes?

I think I know how to implement this in MuMaMo, but I do not have time
right now. (And it is not very much needed.)

> 4) Conceptually simple (I hope)
> 
> Any thoughts?
> 
> 
> 

Re: Better parse-partial-sexp; multiple major modes

[Stefan Monnier]

> Such a thing would have to live on the C side of things, right? (With the
> proliferation of with-this and inhibit-that options available to Lisp,
> I don't see how one can easily and robustly catch all buffer
> modification.  Not to mention that no matter which of before-change-
> functions and after-change-functions you used, you could still race against
> other functions using the same facility.)

In theory, that might be true, but it's not been that big of a deal
for now.  And of course moving it to C would help some but it won't
solve all those problems magically.

> Anyway, this scheme is:

> 1) Robust - no messing around with variables, no tweaking fontification
> 2) Backwards compatible - a major-mode doesn't need to know it's being used
> this way
> 3) Versatile - you can compose arbitrary modes this way, even recursively
> 4) Conceptually simple (I hope)

I don't think it's possible to get 1 and 2 together, really.


        Stefan