* Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss)
@ 2008-08-31 9:39 Daniel Colascione
2008-08-31 18:17 ` Better parse-partial-sexp; multiple major modes Stefan Monnier
0 siblings, 1 reply; 3+ messages in thread
From: Daniel Colascione @ 2008-08-31 9:39 UTC (permalink / raw)
To: emacs-devel
On Jul 27, 2008, at 10:50 AM, Alan Mackenzie wrote:
> What I think really needs doing is to make this function
> bulletproof: It
> should work on narrowed buffers, it should give reliable elements 2
> and
> 6, its cache should be cleared when functions like `modify-syntax-
> entry'
> are called or parse-sexp-lookup-properties is changed, and the cache
> should be bound to nil on `with-syntax-table'. I actually think it
> could be useful to maintain several parallel caches, each for a
> different syntax-table (or an equivalence class of syntax tables).
> And
> so on. Basically, I would like `(syntax-ppss)' to tell me with 100%
> reliability, no ifs, no buts, whether I am at top-level, in a comment,
> or in a string.
Such a thing would have to live on the C side of things, right? (With
the proliferation of with-this and inhibit-that options available to
Lisp, I don't see how one can easily and robustly catch all buffer
modification. Not to mention that no matter which of before-change-
functions and after-change-functions you used, you could still race
against other functions using the same facility.)
If this perfectly caching parse-partial-sexp lives in C anyway, why
not just call it parse-partial-sexp? Optimize parse-partial-sexp for
the case of start being 1 or (point-min). syntax-ppss becomes a simple
wrapper. Not only would it be possible to robustly catch all buffer
and context modification, but by optimizing the existing function, all
existing users would automatically win. I'd offer to write a patch,
but I don't know the core well enough to know how to "easily and
robustly catch all buffer modification".
> Also, Lennart is asking for it to work nicely with multiple major
> modes.
> Surely this would be a Good Thing. Files containing several major
> modes
> are commonplace (awk or sed embedded within a shell script, html
> embedded within php, ....).
After several attempts at using and understanding multiple major mode
facilities, I'm convinced the only way forward is core support for the
concept. Lennart's done a fine job with what's in Emacs currently. But
anything involving multiple major modes today is a quivering mound of
hacks. All the work Lennart's had to do to get modes playing nice with
each other is a testament to that.
Maybe a core solution could be something like this: in a given buffer,
each character has a chunk-name character property. You'd buffer-
locally map chunk names to major modes. For each chunk name, create a
buffer containing just the text assigned to that chunk. Make the major-
mode the major mode for the chunk buffer, and let that major-mode
handle fontification, keybindings, and so on. In the main buffer,
assemble the various bits from the chunk-buffers and allow the user to
navigate the combined buffer normally.
Keybindings with point at a particular character would just be the
keybindings present in that character's chunk-buffer. If you need
special keybindings common across all chunk buffers, just bind the key
in all the chunk buffers. If a given chunk needs placeholder text to
represent text of some other chunk, it should be possible add it to
that chunk buffer without affecting any of the others.
Anyway, this scheme is:
1) Robust - no messing around with variables, no tweaking fontification
2) Backwards compatible - a major-mode doesn't need to know it's being
used this way
3) Versatile - you can compose arbitrary modes this way, even
recursively
4) Conceptually simple (I hope)
Any thoughts?
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Better parse-partial-sexp; multiple major modes
2008-08-31 9:39 Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss) Daniel Colascione
@ 2008-08-31 18:17 ` Stefan Monnier
0 siblings, 0 replies; 3+ messages in thread
From: Stefan Monnier @ 2008-08-31 18:17 UTC (permalink / raw)
To: Daniel Colascione; +Cc: emacs-devel
> Such a thing would have to live on the C side of things, right? (With the
> proliferation of with-this and inhibit-that options available to Lisp,
> I don't see how one can easily and robustly catch all buffer
> modification. Not to mention that no matter which of before-change-
> functions and after-change-functions you used, you could still race against
> other functions using the same facility.)
In theory, that might be true, but it's not been that big of a deal
for now. And of course moving it to C would help some but it won't
solve all those problems magically.
> Anyway, this scheme is:
> 1) Robust - no messing around with variables, no tweaking fontification
> 2) Backwards compatible - a major-mode doesn't need to know it's being used
> this way
> 3) Versatile - you can compose arbitrary modes this way, even recursively
> 4) Conceptually simple (I hope)
I don't think it's possible to get 1 and 2 together, really.
Stefan
^ permalink raw reply [flat|nested] 3+ messages in thread
* Idea for syntax-ppss. Is it new? Could it be any good?
@ 2008-07-26 21:44 Alan Mackenzie
2008-07-27 1:34 ` Stefan Monnier
0 siblings, 1 reply; 3+ messages in thread
From: Alan Mackenzie @ 2008-07-26 21:44 UTC (permalink / raw)
To: emacs-devel
Hi, Emacs,
Looking at the doc string for syntax-ppss, it seems this could be _very_
useful in a certain body of code I'm responsible for. That body of code
has a lot of heuristics that determine whether point is within a
string/comment, and some of these are not watertight (such as hard-coded
limits on comment sizes to achieve speed). Basically, they're a PITA.
syntax-ppss, if it was guaranteed watertight, could remove the gnawing
uncertainty from much of the code.
However, the manual documents limitations on syntax-ppss's functionality.
How about reimplementing it thusly?: The current syntax would be cached
for positions at every N bytes (where N would be, perhaps 1024, possibly
8192). A call to syntax-ppss would simply call parse-partial-sexp from
the latest valid cached position, filling out the cache as it goes. Any
buffer change would invalidate cached values for N > POS.
I envisage coding this in C rather than Lisp. There would be some
complications to do with making sure the syntax table isn't tampered
with, and so on. This code would surely be fast and reliable.
Obviously I'm not proposing this for the pending release, but what do
people think about the idea?
--
Alan Mackenzie (Nuremberg, Germany)
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Idea for syntax-ppss. Is it new? Could it be any good?
2008-07-26 21:44 Idea for syntax-ppss. Is it new? Could it be any good? Alan Mackenzie
@ 2008-07-27 1:34 ` Stefan Monnier
2008-07-27 14:50 ` Alan Mackenzie
0 siblings, 1 reply; 3+ messages in thread
From: Stefan Monnier @ 2008-07-27 1:34 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: emacs-devel
> How about reimplementing it thusly?: The current syntax would be cached
> for positions at every N bytes (where N would be, perhaps 1024, possibly
> 8192). A call to syntax-ppss would simply call parse-partial-sexp from
> the latest valid cached position, filling out the cache as it goes. Any
> buffer change would invalidate cached values for N > POS.
Isn't that what syntax-ppss does?
Stefan
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Idea for syntax-ppss. Is it new? Could it be any good?
2008-07-27 1:34 ` Stefan Monnier
@ 2008-07-27 14:50 ` Alan Mackenzie
2008-08-31 8:37 ` Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss) Daniel Colascione
0 siblings, 1 reply; 3+ messages in thread
From: Alan Mackenzie @ 2008-07-27 14:50 UTC (permalink / raw)
To: Stefan Monnier; +Cc: emacs-devel
Hi, Stefan!
On Sat, Jul 26, 2008 at 09:34:45PM -0400, Stefan Monnier wrote:
> > How about reimplementing it thusly?: The current syntax would be cached
> > for positions at every N bytes (where N would be, perhaps 1024, possibly
> > 8192). A call to syntax-ppss would simply call parse-partial-sexp from
> > the latest valid cached position, filling out the cache as it goes. Any
> > buffer change would invalidate cached values for N > POS.
> Isn't that what syntax-ppss does?
It caches the state for several positions, but I don't think they're at
regular positions. I don't understand the detailed workings of the
routine at the moment. I suspect that the slowness of all the lisp
manipulation will outweigh the benefit of the caching, but I would
confirm or refute that with the profiler before doing anything serious.
partial-parse-sexp is blindingly fast. To scan an entire 3Mb C buffer
on my elderly 1.2 GHz Athlon takes 0.27s. That is why I suspect that
the lisping in syntax-ppss might need severe optimisation. But again,
it's only a hunch.
What I think really needs doing is to make this function bulletproof: It
should work on narrowed buffers, it should give reliable elements 2 and
6, its cache should be cleared when functions like `modify-syntax-entry'
are called or parse-sexp-lookup-properties is changed, and the cache
should be bound to nil on `with-syntax-table'. I actually think it
could be useful to maintain several parallel caches, each for a
different syntax-table (or an equivalence class of syntax tables). And
so on. Basically, I would like `(syntax-ppss)' to tell me with 100%
reliability, no ifs, no buts, whether I am at top-level, in a comment,
or in a string.
Also, Lennart is asking for it to work nicely with multiple major modes.
Surely this would be a Good Thing. Files containing several major modes
are commonplace (awk or sed embedded within a shell script, html
embedded within php, ....).
At the moment, CC Mode applies a heuristic maximum size of strings and
comments, for performance reasons. Checking for strings and comments is
done so frequently that the mode uses elaborate internal caches. It
would be nice if this cacheing could move to the Emacs core.
Again, this isn't something which can be implemented in a weekend, but I
think it would be worthwhile for Emacs 24.
> Stefan
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 3+ messages in thread
* Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss)
2008-07-27 14:50 ` Alan Mackenzie
@ 2008-08-31 8:37 ` Daniel Colascione
2008-08-31 15:02 ` Better parse-partial-sexp; multiple major modes Lennart Borgman (gmail)
0 siblings, 1 reply; 3+ messages in thread
From: Daniel Colascione @ 2008-08-31 8:37 UTC (permalink / raw)
To: emacs-devel
On Jul 27, 2008, at 10:50 AM, Alan Mackenzie wrote:
> What I think really needs doing is to make this function
> bulletproof: It
> should work on narrowed buffers, it should give reliable elements 2
> and
> 6, its cache should be cleared when functions like `modify-syntax-
> entry'
> are called or parse-sexp-lookup-properties is changed, and the cache
> should be bound to nil on `with-syntax-table'. I actually think it
> could be useful to maintain several parallel caches, each for a
> different syntax-table (or an equivalence class of syntax tables).
> And
> so on. Basically, I would like `(syntax-ppss)' to tell me with 100%
> reliability, no ifs, no buts, whether I am at top-level, in a comment,
> or in a string.
Surely, such a creature would have to live on the C side of things, if
only for practical reasons. (With the proliferation of with-this and
inhibit-that options available to Lisp, I don't see how one can easily
and robustly catch all buffer modification. Not to mention that no
matter which of before-change-functions and after-change-functions you
used, you could still race against other functions using the same
facility.)
If this perfectly caching parse-partial-sexp lives in C anyway, why
not just call it parse-partial-sexp? Optimize parse-partial-sexp for
the case of start being 1 or (point-min). syntax-ppss becomes a simple
wrapper. Not only would it be possible to robustly catch all buffer
and context modification, but by optimizing the existing function, all
existing users would automatically win. I'd offer to write a patch,
but I don't know the core well enough to know how to "easily and
robustly catch all buffer modification".
>
> Also, Lennart is asking for it to work nicely with multiple major
> modes.
> Surely this would be a Good Thing. Files containing several major
> modes
> are commonplace (awk or sed embedded within a shell script, html
> embedded within php, ....).
After several attempts at using and understanding multiple major mode
facilities, I'm convinced the only way forward is core support for the
concept. Lennart's done a fine job with what's in Emacs currently. But
anything involving multiple major modes today is a quivering mound of
hacks. All the work Lennart's had to do to get modes playing nice with
each other is a testament to that.
Maybe a core solution could be something like this: in a given buffer,
each character has a chunk-name character property. You'd buffer-
locally map chunk names to major modes. For each chunk name, create a
buffer containing just the text assigned to that chunk. Make the major-
mode the major mode for the chunk buffer, and let that major-mode
handle fontification, keybindings, and so on. In the main buffer,
assemble the various bits from the chunk-buffers and allow the user to
navigate the combined buffer normally.
Keybindings with point at a particular character would just be the
keybindings present in that character's chunk-buffer. If you need
special keybindings common across all chunk buffers, just bind the key
in all the chunk buffers. If a given chunk needs placeholder text to
represent text of some other chunk, it should be possible add it to
that chunk buffer without affecting any of the others.
Anyway, this scheme is:
1) Robust - no messing around with variables, no tweaking fontification
2) Backwards compatible - a major-mode doesn't need to know it's being
used this way
3) Versatile - you can compose arbitrary modes this way, even
recursively
4) Conceptually simple (I hope)
Any thoughts?
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Better parse-partial-sexp; multiple major modes
2008-08-31 8:37 ` Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss) Daniel Colascione
@ 2008-08-31 15:02 ` Lennart Borgman (gmail)
0 siblings, 0 replies; 3+ messages in thread
From: Lennart Borgman (gmail) @ 2008-08-31 15:02 UTC (permalink / raw)
To: Daniel Colascione; +Cc: emacs-devel
Daniel Colascione wrote:
>> Also, Lennart is asking for it to work nicely with multiple major modes.
>> Surely this would be a Good Thing. Files containing several major modes
>> are commonplace (awk or sed embedded within a shell script, html
>> embedded within php, ....).
>
> After several attempts at using and understanding multiple major mode
> facilities, I'm convinced the only way forward is core support for the
> concept. Lennart's done a fine job with what's in Emacs currently. But
> anything involving multiple major modes today is a quivering mound of
> hacks. All the work Lennart's had to do to get modes playing nice with
> each other is a testament to that.
Are you suggesting that you have problems using MuMaMo today? If so then
please report it as a bug.
> Maybe a core solution could be something like this: in a given buffer,
> each character has a chunk-name character property. You'd buffer-locally
> map chunk names to major modes. For each chunk name, create a buffer
> containing just the text assigned to that chunk. Make the major-mode the
> major mode for the chunk buffer, and let that major-mode handle
> fontification, keybindings, and so on. In the main buffer, assemble the
> various bits from the chunk-buffers and allow the user to navigate the
> combined buffer normally.
This was an idea I played with at the beginning when I wrote mumamo.el.
I am afraid I think the concepts involved (like buffer local etc) must
be considered quite a bit more first before going in this direction. It
is unclear to me now whether chunk-buffers really would be of any help.
They might be, but I am not sure.
> Keybindings with point at a particular character would just be the
> keybindings present in that character's chunk-buffer. If you need
> special keybindings common across all chunk buffers, just bind the key
> in all the chunk buffers. If a given chunk needs placeholder text to
> represent text of some other chunk, it should be possible add it to that
> chunk buffer without affecting any of the others.
>
> Anyway, this scheme is:
>
> 1) Robust - no messing around with variables, no tweaking fontification
Unfortunately I do not think that will hold.
I think the way to go is "interface style". font-lock has some good
support for making multiple major modes possible.
> 2) Backwards compatible - a major-mode doesn't need to know it's being
> used this way
Are you aware of that a major mode does not need to know anything when
used in MuMaMo?
> 3) Versatile - you can compose arbitrary modes this way, even recursively
The main difficulty with sub-modes in sub-modes is stability. How do you
find the sub-modes in sub-modes after buffer changes?
I think I know how to implement this in MuMaMo, but I do not have time
right now. (And it is not very much needed.)
> 4) Conceptually simple (I hope)
>
> Any thoughts?
>
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2008-08-31 18:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-08-31 9:39 Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss) Daniel Colascione
2008-08-31 18:17 ` Better parse-partial-sexp; multiple major modes Stefan Monnier
-- strict thread matches above, loose matches on Subject: below --
2008-07-26 21:44 Idea for syntax-ppss. Is it new? Could it be any good? Alan Mackenzie
2008-07-27 1:34 ` Stefan Monnier
2008-07-27 14:50 ` Alan Mackenzie
2008-08-31 8:37 ` Better parse-partial-sexp; multiple major modes (was: Idea for syntax-ppss) Daniel Colascione
2008-08-31 15:02 ` Better parse-partial-sexp; multiple major modes Lennart Borgman (gmail)
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).