Should nXML be included

Should nXML be included

[Leo]

Dear all,

I wonder if nXML¹ can be included.

Footnotes: 
¹  http://www.thaiopensource.com/nxml-mode/
-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Should nXML be included

[Lennart Borgman (gmail)]

Leo wrote:
> Dear all,
> 
> I wonder if nXML¹ can be included.
> 
> Footnotes: 
> ¹  http://www.thaiopensource.com/nxml-mode/

nXml is very good and I hope it will be included in Emacs. However there 
are reasons to break it up. I think the parsing code somehow should be 
broken up so that it can work for cases where the buffer is divided into 
multiple major modes.

This is however not a simple job.

For a comment on this see

http://sourceforge.net/mailarchive/forum.php?thread_name=4638A428.9010408%40pareto.nl&forum_name=cedet-devel

Re: Should nXML be included

[Jason Rumney]

Lennart Borgman (gmail) wrote:
> nXml is very good and I hope it will be included in Emacs. However
> there are reasons to break it up. I think the parsing code somehow
> should be broken up so that it can work for cases where the buffer is
> divided into multiple major modes.

I think what you really mean is that you want to use nxml-mode to edit
non-xml files. Is that right? The problem I see with this is that the
power of nxml-mode compared to psgml and sgml modes is that it deals
with strict xml, and deals with it well. As soon as you start ripping
out features to support non-XML documents, you're back to a general SGML
mode and the compromises that brings.

If you really want to create PHP in an XML file, you need to use CDATA
blocks to tell the XML parser to ignore the invalid tags in that block.

Re: Should nXML be included

[Lennart Borgman (gmail)]

Jason Rumney wrote:
> Lennart Borgman (gmail) wrote:
>> nXml is very good and I hope it will be included in Emacs. However
>> there are reasons to break it up. I think the parsing code somehow
>> should be broken up so that it can work for cases where the buffer is
>> divided into multiple major modes.
> 
> I think what you really mean is that you want to use nxml-mode to edit
> non-xml files. Is that right? The problem I see with this is that the
> power of nxml-mode compared to psgml and sgml modes is that it deals
> with strict xml, and deals with it well. As soon as you start ripping
> out features to support non-XML documents, you're back to a general SGML
> mode and the compromises that brings.

Yes, you are right, I want to edit non-xml files.

But I think the power of nxml-mode can still be used. The parser 
provides to things: Validation and completion.

Validation of course loose most of its meanings, but not all. Completion 
can still be used. In fact that is what I do in nXhtml with nxhtml-mode 
+ mumamo-mode. To be able to make it more cleanly than currently the 
parser must be broken out.

> If you really want to create PHP in an XML file, you need to use CDATA
> blocks to tell the XML parser to ignore the invalid tags in that block.
> 
> 

Re: Should nXML be included

[Leo]

----- Peter Heslin (2007-06-12) wrote:-----

> Some years ago, James Clark posted a message to the nxml mailing list
> to say that he would not have time to maintain it.  If it were brought
> into Emacs, that would help to avoid the potential for future bitrot
> -- and for me nxml is one of Emacs' killer apps.

This is one of the reasons behind my proposal.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Should nXML be included

[Peter Heslin]

Leo <sdl.web@gmail.com> writes:

> I wonder if nXML¹ can be included.

Some years ago, James Clark posted a message to the nxml mailing list to
say that he would not have time to maintain it.  If it were brought into
Emacs, that would help to avoid the potential for future bitrot -- and
for me nxml is one of Emacs' killer apps.

Peter

-- 
Peter Heslin (http://www.dur.ac.uk/p.j.heslin)

Re: Should nXML be included

[Stefan Monnier]

> nXml is very good and I hope it will be included in Emacs.

Good.

> However there are reasons to break it up. I think the parsing code somehow
> should be broken up so that it can work for cases where the buffer is
> divided into multiple major modes.

Irrelevant to the discussion at hand.


        Stefan

Re: Should nXML be included

[Peter Heslin]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:

> Validation of course loose most of its meanings, but not
> all. Completion can still be used. In fact that is what I do in nXhtml
> with nxhtml-mode + mumamo-mode. To be able to make it more cleanly
> than currently the parser must be broken out.

As you know, another possibility is to put the php code into CDATA
sections so that nxml ignores it; and when you want to edit that code,
narrow the buffer temporarily to that section and switch to php-mode.

In any case, the viability of multiple major modes in a single buffer is
a much larger issue which does not really impact the question of whether
nxml should be included with Emacs.

Peter

-- 
Peter Heslin (http://www.dur.ac.uk/p.j.heslin)

Re: Should nXML be included

[joakim]

Leo <sdl.web@gmail.com> writes:

> ----- Peter Heslin (2007-06-12) wrote:-----
>
>> Some years ago, James Clark posted a message to the nxml mailing list
>> to say that he would not have time to maintain it.  If it were brought
>> into Emacs, that would help to avoid the potential for future bitrot
>> -- and for me nxml is one of Emacs' killer apps.
>
> This is one of the reasons behind my proposal.

nxml is definitively one of Emacs killer apps.

James Clark appears to be difficult to contact these days, however,
which might prove an obstacle for inclusion.

> -- 
> Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

-- 
Joakim Verona

Re: Should nXML be included

[David Kastrup]

Peter Heslin <p.j.heslin@dur.ac.uk> writes:

> joakim@verona.se writes:
>
>> James Clark appears to be difficult to contact these days, however,
>> which might prove an obstacle for inclusion.
>
> Not necessarily.  The files in question say that they are copyright FSF,
> and I assume that James has papers on file for the other parts of Emacs
> that he has contributed.  I hope these would cover nxml too.
>
> An example header:
>
> ;;; nxml-mode.el --- a new XML mode
>
> ;; Copyright (C) 2003, 2004 Free Software Foundation, Inc.
>
> ;; Author: James Clark
> ;; Keywords: XML

Copyright headers are not necessarily correct.  That's why one needs
the written assignment on file for contributions.  If James has an
assignment that only covers his (explicit) contributions to Emacs, he
most certainly can't be held responsible for the accuracy of some
headers in a file he never contributed to Emacs.

-- 
David Kastrup

Re: Should nXML be included

[Peter Heslin]

joakim@verona.se writes:

> James Clark appears to be difficult to contact these days, however,
> which might prove an obstacle for inclusion.

Not necessarily.  The files in question say that they are copyright FSF,
and I assume that James has papers on file for the other parts of Emacs
that he has contributed.  I hope these would cover nxml too.

An example header:

;;; nxml-mode.el --- a new XML mode

;; Copyright (C) 2003, 2004 Free Software Foundation, Inc.

;; Author: James Clark
;; Keywords: XML



-- 
Peter Heslin (http://www.dur.ac.uk/p.j.heslin)

Re: Should nXML be included

[Jason Rumney]

Lennart Borgman (gmail) wrote:
> Validation of course loose most of its meanings, but not all.
> Completion can still be used.

The completion in nxml-mode is only as good as it is because the
validation tells it what is valid in that context. You can't have one
without the other. Like I said, as soon as you start trying to make nxml
a general SGML mode, you introduce compromises.

Multiple major mode support is a separate issue from nxml, and something
that should be addressed without imposing restrictions on specialized
major modes.

Multiple major modes (was: Should nXML be included)

[Stefan Monnier]

> Multiple major mode support is a separate issue from nxml,

100% agreement, obviously.

> and something that should be addressed without imposing restrictions on
> specialized major modes.

Actually, I think that in order to address it well, we will need to impose
restrictions on major modes (though only on the ones involved in
multiple-major-mode buffers) and maybe also on minor modes.


        Stefan

Re: Should nXML be included

[Stefan Monnier]

> Copyright headers are not necessarily correct.  That's why one needs
> the written assignment on file for contributions.  If James has an
> assignment that only covers his (explicit) contributions to Emacs, he
> most certainly can't be held responsible for the accuracy of some
> headers in a file he never contributed to Emacs.

He does have papers on file that cover Emacs as well as nXML-mode (and then
some).  But before being able to include it, we need to check the
contributions by others, which is usually the trickier part.
The distribution does not come with any ChangeLog, so we will very much the
help of James to figure out what was contributed by whom.


        Stefan

Re: Multiple major modes

[Leo]

----- Stefan Monnier (2007-06-12) wrote:-----

>> and something that should be addressed without imposing restrictions
>> on specialized major modes.
>
> Actually, I think that in order to address it well, we will need to
> impose restrictions on major modes (though only on the ones involved
> in multiple-major-mode buffers) and maybe also on minor modes.

There seems to be a few multi mode implementation, what is the
preference from Emacs developers?

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Multiple major modes

[Lennart Borgman (gmail)]

Stefan Monnier wrote:
>> Multiple major mode support is a separate issue from nxml,
> 
> 100% agreement, obviously.

Of course. I just thought it was a good time to mention it.

>> and something that should be addressed without imposing restrictions on
>> specialized major modes.
> 
> Actually, I think that in order to address it well, we will need to impose
> restrictions on major modes (though only on the ones involved in
> multiple-major-mode buffers) and maybe also on minor modes.


I think so to. I have seen some areas where some form of coherence is 
necessary:

- fontification
- indentation
- parsing

I think nxml-mode is a very good candidate for those that can be used in 
a multiple-major-mode buffers, since a mix of for example XHTML and PHP 
is common.

This might seem impossible, since such code could not be valid XHTML. 
Actually it is not, the rng parser used in nxhtml-mode is quite good on 
guessing an apropriate state for completion. (But do not ask me what the 
rng parser does, I do not understand it.)

Re[2]: Multiple major modes

[Eric M. Ludlam]

  [ ... ]
>>> "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> seems to think that:
>Stefan Monnier wrote:
>>> and something that should be addressed without imposing restrictions on
>>> specialized major modes.
>> 
>> Actually, I think that in order to address it well, we will need to impose
>> restrictions on major modes (though only on the ones involved in
>> multiple-major-mode buffers) and maybe also on minor modes.

My parsing tool (semantic, referenced earlier) solved some problems w/
lots of mode-specific configurations via 'mode-local' variables and
methods.  David Ponce wrote this, and suggested it here once before.
I don't recall what the end resolution was on it.

A multi-mode style thing would likely be simplified if key behaviors
were all defined via mode-local configurations, as that would allow
the multi-mode manager to get a complete query list of all
configuration differences without running the major-mode function.

http://cedet.cvs.sourceforge.net/cedet/cedet/common/mode-local.el?view=log

This tools has a spiff macro `with-mode-local', which lets you run a
bunch of code as if some other mode were active.  This is used in our
grammar file so we can operate on grammar syntax w/ lisp syntax mixed
in.  We can also operate on tags from other buffers by momentarily
using features of the originating major-mode.

Enjoy
Eric


>I think so to. I have seen some areas where some form of coherence is 
>necessary:
>
>- fontification
>- indentation
>- parsing
>
>I think nxml-mode is a very good candidate for those that can be used in 
>a multiple-major-mode buffers, since a mix of for example XHTML and PHP 
>is common.
>
>This might seem impossible, since such code could not be valid XHTML. 
>Actually it is not, the rng parser used in nxhtml-mode is quite good on 
>guessing an apropriate state for completion. (But do not ask me what the 
>rng parser does, I do not understand it.)
>

-- 
          Eric Ludlam:                 zappo@gnu.org, eric@siege-engine.com
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org

Re: Multiple major modes

[Lennart Borgman (gmail)]

Eric M. Ludlam wrote:
>   [ ... ]
>>>> "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> seems to think that:
>> Stefan Monnier wrote:
>>>> and something that should be addressed without imposing restrictions on
>>>> specialized major modes.
>>> Actually, I think that in order to address it well, we will need to impose
>>> restrictions on major modes (though only on the ones involved in
>>> multiple-major-mode buffers) and maybe also on minor modes.
> 
> My parsing tool (semantic, referenced earlier) solved some problems w/
> lots of mode-specific configurations via 'mode-local' variables and
> methods.  David Ponce wrote this, and suggested it here once before.
> I don't recall what the end resolution was on it.
> 
> A multi-mode style thing would likely be simplified if key behaviors
> were all defined via mode-local configurations, as that would allow
> the multi-mode manager to get a complete query list of all
> configuration differences without running the major-mode function.
> 
> http://cedet.cvs.sourceforge.net/cedet/cedet/common/mode-local.el?view=log

Would it be possible to catch all local bindings by setting the major 
mode in a temporary buffer and then save them for later use? This would 
avoid having to change major modes, perhaps.

> This tools has a spiff macro `with-mode-local', which lets you run a
> bunch of code as if some other mode were active.  This is used in our
> grammar file so we can operate on grammar syntax w/ lisp syntax mixed
> in.  We can also operate on tags from other buffers by momentarily
> using features of the originating major-mode.
> 
> Enjoy
> Eric

Re: Multiple major modes

[Stefan Monnier]

> Would it be possible to catch all local bindings by setting the major mode
> in a temporary buffer and then save them for later use? This would avoid
> having to change major modes, perhaps.

This could be a default heuristic to be compatible with major modes which
have not been adapted, but better provide a more robust solution as well, so
that this hack can be phased out little by little.


        Stefan

Re: Should nXML be included

[Richard Stallman]

    Copyright headers are not necessarily correct.  That's why one needs
    the written assignment on file for contributions.  If James has an
    assignment that only covers his (explicit) contributions to Emacs, he
    most certainly can't be held responsible for the accuracy of some
    headers in a file he never contributed to Emacs.

He didn't sign a general assignment for Emacs changes.  He signed
assignments for a few specific files.  They cannot apply to this file.

So we need to contact him and ask for another assignment for this
file.

Re: Multiple major modes

[Richard Stallman]

    This tools has a spiff macro `with-mode-local', which lets you run a
    bunch of code as if some other mode were active.

Can you explain, in a nutshell, how the mechanism works?

Re: Multiple major modes

[Richard Stallman]

MuMaMo, which was discussed here a few months ago, seems to be basically
a good way of handling multiple major modes.  If people are discussing
other ways, I suggest comparing them with MuMaMo.

Re: Should nXML be included

[Claus]

+1 that nxml-mode is a killer app and a must-have for every Emacs user
editing xml-files (and who is not these days :)

Well, James Clark recently joined the blogsphere and his email address
can be found on his web page[1]. Should be worth trying to contact
him, he used to be very responsive and helpful on the nxml-mailing
list.

Claus

[1] http://www.jclark.com/

On 6/13/07, Richard Stallman <rms@gnu.org> wrote:
>     Copyright headers are not necessarily correct.  That's why one needs
>     the written assignment on file for contributions.  If James has an
>     assignment that only covers his (explicit) contributions to Emacs, he
>     most certainly can't be held responsible for the accuracy of some
>     headers in a file he never contributed to Emacs.
>
> He didn't sign a general assignment for Emacs changes.  He signed
> assignments for a few specific files.  They cannot apply to this file.
>
> So we need to contact him and ask for another assignment for this
> file.
>
>
> _______________________________________________
> Emacs-devel mailing list
> Emacs-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-devel
>


-- 

Mit freundlichem Gruß,
Claus Klingberg

Re: Should nXML be included

[Richard Stallman]

    Well, James Clark recently joined the blogsphere and his email address
    can be found on his web page[1].

It isn't feasible for me to browse a web site--could you
tell me the email address?

Re: Should nXML be included

[Leo]

Dear rms,

----- Richard Stallman (2007-06-14) wrote:-----

>     Well, James Clark recently joined the blogsphere and his email address
>     can be found on his web page[1].
>
> It isn't feasible for me to browse a web site--could you
> tell me the email address?

James Clark
jjc@jclark.com

regards,
-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Should nXML be included

[Eric Hanchrow]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

    Richard>     Well, James Clark recently joined the blogsphere and
    Richard>     his email address can be found on his web page[1].

    Richard> It isn't feasible for me to browse a web site--could you
    Richard> tell me the email address?

jjc@jclark.com appears to be him.

-- 
I ... don't want programming to be "really interesting" any more than
I want my toaster to be "really interesting".  I want my toaster to
get the job done and I do not want to hear about exciting new
tungesten alloys in the heating wires nor do I want to subscribe to
Toaster Today magazine.
        -- Daniel Weinreb <DLWeinreb@attbi.com>, ll1 mailing list

Re: Should nXML be included

[Richard Stallman]

Thanks.  I wrote to him.

Re[2]: Multiple major modes

[Eric M. Ludlam]

>>> Richard Stallman <rms@gnu.org> seems to think that:
>    This tools has a spiff macro `with-mode-local', which lets you run a
>    bunch of code as if some other mode were active.
>
>Can you explain, in a nutshell, how the mechanism works?
>

The mode-local system ignores all variables set for the mode outside
the scope of mode-local.

Step 1:

  Define all your variables with `defvar-mode-local'.
  If changing a default value, use `setq-mode-local'

Step 2:

  For a program, use `define-overload' to specify a function that
  can have a mode specific override.  Supply a default body for modes
  that accept the default.


Step 3:

  The mode-local system will automatically assign buffer-local values
  to mode-local defined variables.

  If a function is called created with `define-overload', then a mode
  specific behavior will be run.


Ok, now if you use `with-mode-local', it will just substitute the old
local variables defined in the current mode with those from the new
mode, deleting any that don't exist in the new mode.

When exiting body (from with-mode-local macro) then it will just
reverse the process.


This is a key piece of abstraction for my semantic tool which provides
language independent access to tagging information, which needs
mode-specific values for parser information and many other functions.

http://cedet.sourceforge.net/semantic.shtml


Enjoy
Eric

-- 
          Eric Ludlam:                 zappo@gnu.org, eric@siege-engine.com
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org

Re: Multiple major modes

[Richard Stallman]

      Define all your variables with `defvar-mode-local'.
      If changing a default value, use `setq-mode-local'

Does this mean we would have to go thru all of Emacs and put in
such definitions for all the variables that major modes can change?

Currently, the idea is that major modes can make any global variable
buffer-local.  Are you suggesting we divide all variables into two
classes, those that can be and those that can't be?  If so, what is
the benefit of that?

Would it make sense for `defvar' itself to do whatever you want
`defvar-mode-local' to do?

      For a program, use `define-overload' to specify a function that
      can have a mode specific override.  Supply a default body for modes
      that accept the default.

Since multiple major modes work (in different buffers) without this
feature, why would we need to add this feature merely to support
multiple major modes in one buffer?

      The mode-local system will automatically assign buffer-local values
      to mode-local defined variables.

I don't understand.  Does this mean that everything defined with
`defvar-mode-local' gets buffer-local in all buffers?  If so,
how is that different from `make-variable-buffer-local'?

I guess there must be parts of this explanation that are missing.

Re[2]: Multiple major modes

[Eric M. Ludlam]

Hi,

>>> Richard Stallman <rms@gnu.org> seems to think that:
>      Define all your variables with `defvar-mode-local'.
>      If changing a default value, use `setq-mode-local'
>
>Does this mean we would have to go thru all of Emacs and put in
>such definitions for all the variables that major modes can change?

First, I'd like to be clear that I did not concoct the mode-local
program so that I could write a cheap multi-major-mode type program.
For semantic, which is a parser-generator, and has many minor modes,
it allows someone to write mode-specific code and variable values in a
way that is declarative, instead of in a giant hook function.  It
makes it very easy to write a tool (like semantic) that can work in
many major modes, and where the differences between major modes is
very small.

Another advantage is that it allows each mode to add documentation
specific to its needs.

As for adopting it for all modes, I expect it is optional.  It would
replace the big block in most mode functions that create and set
buffer-local values for tools that are not specific to that
major-mode.

>Currently, the idea is that major modes can make any global variable
>buffer-local.  Are you suggesting we divide all variables into two
>classes, those that can be and those that can't be?  If so, what is
>the benefit of that?

There would be two classes.  There would be values that are specific
to the major mode, and variables that are local to just some buffer.
The buffer local value would trump the mode-specific value, and any
mode-specific value would trump the global value.

>Would it make sense for `defvar' itself to do whatever you want
>`defvar-mode-local' to do?

That could simplify things greatly, though I haven't though too hard
on this idea.

>      For a program, use `define-overload' to specify a function that
>      can have a mode specific override.  Supply a default body for modes
>      that accept the default.
>
>Since multiple major modes work (in different buffers) without this
>feature, why would we need to add this feature merely to support
>multiple major modes in one buffer?

The function overload mechanism is also a feature I use in semantic.
Most features that work in multiple major modes today provide a
variable where you can put a symbol that is a function that would then
provide some mode-specific functionality.

My semantic tool has hundreds of these functions, so I abstracted the
concept up so that the implementations could be declarative, instead
of programmatic.  It also makes it easy to make most functions
overridable, which helps avoid forcing users to use advice when
customizing my tool.

As for the multi-major-mode case, if the technique of a variable w/ a
symbol in it is used, then you are correct, this already works.


>      The mode-local system will automatically assign buffer-local values
>      to mode-local defined variables.
>
>I don't understand.  Does this mean that everything defined with
>`defvar-mode-local' gets buffer-local in all buffers?  If so,
>how is that different from `make-variable-buffer-local'?
>
>I guess there must be parts of this explanation that are missing.

The mode-local feature I wrote (with a lot of help from David Ponce)
works in all versions of Emacs commonly used today.  It does this work
by modifying the buffer-local values for different major-modes.  It
provides the concept of what a mode-local variable might be.  Ideally
it would not operate this way, and instead have built-in support for
real mode-local values and method tables.

In summary, the mode-local tool is useful (to me at least) because it
lets me write more declarative code while setting up values for a
major mode.

For the multi-major-mode case, it would allow a multi-major-mode tool
to operate on predefined lists of variables and it would know the
difference between a variable that is specific to that major mode (and
thus must be swapped in and out) and a variable that is buffer local
for some other reason, such as a minor-mode.  A side effect is that
you could have minor modes that operate across both major modes in a
single buffer, and that minor mode won't get tromped.

I used this technique for semantic's grammar mode which swaps between
the grammar syntax, and Emacs Lisp which is the grammar
implementation.


It is obvious that if a mode-local type tool or syntax were adopted
into Emacs, that it would be a big effort to transition everything to
it.  I also don't recommend that David's mode-local implementation be
taken as a sole implementation.  The real advantage would come if the
concepts were built into Emacs and the help system which could then
show chains of values (For C-h v) or chains of function doc (for C-h f)
based on mode.  The readability of my code increased greatly when I
transitioned to a more declarative way of setting up deltas to
different major-modes.  It also made the grammar mode easy to
implement, and that's why I suggest it here.

Thanks
Eric

Re: Multiple major modes

[Richard Stallman]

Please forgive my delay in studying your message and responding.

    >Currently, the idea is that major modes can make any global variable
    >buffer-local.  Are you suggesting we divide all variables into two
    >classes, those that can be and those that can't be?  If so, what is
    >the benefit of that?

    There would be two classes.

I don't entirely follow.  Two classes of what?  Two classes of variables?

If so, could you describe each of the two classes?

				 There would be values that are specific
    to the major mode, and variables that are local to just some buffer.
    The buffer local value would trump the mode-specific value, and any
    mode-specific value would trump the global value.

That part makes sense to me; but can't we make all kinds of bindings
possible for every variable?

    The mode-local feature I wrote (with a lot of help from David Ponce)
    works in all versions of Emacs commonly used today.  It does this work
    by modifying the buffer-local values for different major-modes.  It
    provides the concept of what a mode-local variable might be.  Ideally
    it would not operate this way, and instead have built-in support for
    real mode-local values and method tables.

That makes sense.  We could implement mode-local bindings at the low
level if that is what we want.

What should their semantics be?

      A side effect is that
    you could have minor modes that operate across both major modes in a
    single buffer, and that minor mode won't get tromped.

How would this work?

    It is obvious that if a mode-local type tool or syntax were adopted
    into Emacs, that it would be a big effort to transition everything to
    it.

I hope we can design the mechanism to avoid the need for pervasive
changes in the code of major modes.  Perhaps we would need to change
some of the current calls to make-local-variable to call some new
primitive, make-mode-variable, instead.  But even that would be a pain
if every major mode HAD to do it.

      The real advantage would come if the
    concepts were built into Emacs and the help system which could then
    show chains of values (For C-h v) or chains of function doc (for C-h f)
    based on mode.

I agree, that would be good to do.

Re: Multiple major modes

[Richard Stallman]

    The function overload mechanism is also a feature I use in semantic.
    Most features that work in multiple major modes today provide a
    variable where you can put a symbol that is a function that would then
    provide some mode-specific functionality.

    My semantic tool has hundreds of these functions, so I abstracted the
    concept up so that the implementations could be declarative, instead
    of programmatic. 

I really don't like the idea of function overloads.  This mechanism
shares the drawbacks of advice: that a function doesn't do what its
definition says.

		      It also makes it easy to make most functions
    overridable, which helps avoid forcing users to use advice when
    customizing my tool.

It is easy to replace advising with another similar mechanism,
but it doesn't solve the problem.

It seems to me that there is no need for this.  Calling a variable
with funcall should do the same job.  That way, the call _shows_
that the function isn't fixed.

Re: Multiple major modes

[T. V. Raman]

For the record, advice does update the function documentation,
assuming one documents the advice.


>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:
    Richard>     The function overload mechanism is also a
    Richard> feature I use in semantic.  Most features that work
    Richard> in multiple major modes today provide a variable
    Richard> where you can put a symbol that is a function that
    Richard> would then provide some mode-specific functionality.
    Richard> 
    Richard>     My semantic tool has hundreds of these
    Richard> functions, so I abstracted the concept up so that
    Richard> the implementations could be declarative, instead of
    Richard> programmatic.
    Richard> 
    Richard> I really don't like the idea of function overloads.
    Richard> This mechanism shares the drawbacks of advice: that
    Richard> a function doesn't do what its definition says.
    Richard> 
    Richard> 		      It also makes it easy to make most
    Richard> functions overridable, which helps avoid forcing
    Richard> users to use advice when customizing my tool.
    Richard> 
    Richard> It is easy to replace advising with another similar
    Richard> mechanism, but it doesn't solve the problem.
    Richard> 
    Richard> It seems to me that there is no need for this.
    Richard> Calling a variable with funcall should do the same
    Richard> job.  That way, the call _shows_ that the function
    Richard> isn't fixed.
    Richard> 
    Richard> 
    Richard> _______________________________________________
    Richard> Emacs-devel mailing list Emacs-devel@gnu.org
    Richard> http://lists.gnu.org/mailman/listinfo/emacs-devel

-- 
Best Regards,
--raman

      
Email:  raman@users.sf.net
WWW:    http://emacspeak.sf.net/raman/
AIM:    emacspeak       GTalk: tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc
Google: tv+raman 
IRC:    irc://irc.freenode.net/#emacs

Re: Multiple major modes

[David Kastrup]

"T. V. Raman" <raman@users.sf.net> writes:

> For the record, advice does update the function documentation,
> assuming one documents the advice.

Once the advice is loaded and applied.

Now it just needs to update the more extensive documentation in the
manual, and we are all set.

Except that it is hard to guess the cumulative effects of multiple
pieces of advice.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Multiple major modes

[Richard Stallman]

    For the record, advice does update the function documentation,
    assuming one documents the advice.

It doesn't update the source code of the function definition,
which is the main thing that someone debugging will look at.

This suggests an idea to me.  Maybe Emacs could highlight the function
definition somehow, to indicate that the function has been advised.

That might solve the problem that makes advising bad for debugging.
Would someone like to try it?

To be sure whether it solves that problem, we would have to try it for
a while, and see the results.  We can't assume this a priori.

Re[2]: Multiple major modes

[Eric M. Ludlam]

>>> Richard Stallman <rms@gnu.org> seems to think that:
>    The function overload mechanism is also a feature I use in semantic.
>    Most features that work in multiple major modes today provide a
>    variable where you can put a symbol that is a function that would then
>    provide some mode-specific functionality.
>
>    My semantic tool has hundreds of these functions, so I abstracted the
>    concept up so that the implementations could be declarative, instead
>    of programmatic. 
>
>I really don't like the idea of function overloads.  This mechanism
>shares the drawbacks of advice: that a function doesn't do what its
>definition says.
>
>		      It also makes it easy to make most functions
>    overridable, which helps avoid forcing users to use advice when
>    customizing my tool.
>
>It is easy to replace advising with another similar mechanism,
>but it doesn't solve the problem.
>
>It seems to me that there is no need for this.  Calling a variable
>with funcall should do the same job.  That way, the call _shows_
>that the function isn't fixed.

The use syntax of an overloadable function is similar to programs that
call `indent-for-tab-command', or `indent-according-to-mode', but
there is no restriction forcing `according-to-mode' as part of the
function name.

The difference between writing an `indent-according-to-mode'
function and an overload function is that you don't have to write the
body of the function or, in the specific case of
`indent-according-to-mode', you could write a body that wraps up an
:overload token similar to the way `indent-according-to-mode' wraps
the `funcall'.

Anyway, I think these overload functions are more like methods which
dispatch on major mode instead of advice.  You can advise any
function, but you can only overload something created with
`define-overload', so it is known via syntactic sugar instead of a
funcall call.

When it comes right down to it though, this is not needed for multiple
major modes where one of the concerns is buffer-local vs mode-local
variables, and I don't want to imply that I think it is needed.  It is
only a feature which I feel has enhanced the maintainability of my
code, and has helped others understand and discover how to extend my
tools for different modes.

Eric

-- 
          Eric Ludlam:                 zappo@gnu.org, eric@siege-engine.com
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org

Re[2]: Multiple major modes

[Eric M. Ludlam]

>>> Richard Stallman <rms@gnu.org> seems to think that:
>Please forgive my delay in studying your message and responding.
>
>    >Currently, the idea is that major modes can make any global variable
>    >buffer-local.  Are you suggesting we divide all variables into two
>    >classes, those that can be and those that can't be?  If so, what is
>    >the benefit of that?
>
>    There would be two classes.
>
>I don't entirely follow.  Two classes of what?  Two classes of variables?
>
>If so, could you describe each of the two classes?

Sorry for being vague.  There would be "buffer-local variables", and
"mode-local variables".  I was distinguishing those two from "global
variables". In reading the "Elisp" manual to check my phrases, I see
there are also "frame-local variables", and many other ways to bind
variables, so I am just wrong in saying there are only two classes of
variables.

>				 There would be values that are specific
>    to the major mode, and variables that are local to just some buffer.
>    The buffer local value would trump the mode-specific value, and any
>    mode-specific value would trump the global value.
>
>That part makes sense to me; but can't we make all kinds of bindings
>possible for every variable?

Correct.  When trying to resolve the value of some symbol, mode-local
values would fit into the current scheme of determining the value in
some way, and I am suggesting that buffer local values would be used
before mode-local values.

>    The mode-local feature I wrote (with a lot of help from David Ponce)
>    works in all versions of Emacs commonly used today.  It does this work
>    by modifying the buffer-local values for different major-modes.  It
>    provides the concept of what a mode-local variable might be.  Ideally
>    it would not operate this way, and instead have built-in support for
>    real mode-local values and method tables.
>
>That makes sense.  We could implement mode-local bindings at the low
>level if that is what we want.
>
>What should their semantics be?

If you have a variable defined:

(defvar my-var nil)

and then set a mode-local value for a mojor mode:

;; programmatic
(setq-mode-local 'c-mode my-var t)
;; or declarative
(defvar-mode-local c-mode my-var t "Description about C value")
;; or a constant for that mode, if `defconst' was used instead of `defvar'.
(defconst-mode-local c-mode my-var t "Description here...")

then all buffers in c-mode would get the new value of my-var.

If any of those buffers have a buffer-local version of my-var, then
that value would be used instead.  I'm not sure about frame-local
variables.

If a c-mode buffer switched to fundamental-mode, then the value of
my-var would go back to nil, (the default value).

>      A side effect is that
>    you could have minor modes that operate across both major modes in a
>    single buffer, and that minor mode won't get tromped.
>
>How would this work?

A minor mode which works in an HTML buffer with embedded php in it
could use `mode-local-symbol-value' to get the value of some HTML
variable when inside a PHP block for, perhaps, fill-column, or other
variable even though the effective major mode is PHP.

Basically, any code could be written to be aware of the combination of
two modes for a specific case, and have access to both sets of
mode-local variable bindings, regardless of cursor position.

>    It is obvious that if a mode-local type tool or syntax were adopted
>    into Emacs, that it would be a big effort to transition everything to
>    it.
>
>I hope we can design the mechanism to avoid the need for pervasive
>changes in the code of major modes.  Perhaps we would need to change
>some of the current calls to make-local-variable to call some new
>primitive, make-mode-variable, instead.  But even that would be a pain
>if every major mode HAD to do it.

I think it can be completely optional.  Whichever multiple major-mode
tool that is chosen could take advantage of mode-local variables, and
major modes would adopt the use of them because of performance and
usage enhancement.

If a multi-major-mode tool knows a particular major-mode doesn't use
buffer-local variables, then it would not need to run the major-mode
function (which could call `kill-all-local-variables') for every
section switch and lots of work needed to save and restore
buffer-local values that may or may not be mode-specific would be
saved.


I like providing the mode-local values in a declarative way outside of
the mode function.  It provides a place to add documentation and talk
about the change.  That alone, I think, is worthwhile.

Eric

-- 
          Eric Ludlam:                 zappo@gnu.org, eric@siege-engine.com
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org

Re: Multiple major modes

[Stefan Monnier]

> That might solve the problem that makes advising bad for debugging.
> Would someone like to try it?

I don't find advising difficult for debugging.

Actually, the problem with defadvice (for me) is not when the advice is
enabled, but when it's not because then there's no trace anywhere that the
function you're looking at has a special relationship to some other piece of
code elsewhere.

So whenever I change a function (i.e. whenever I change some piece of code
in Emacs) I may potentially break an advice.  So if I don't want to break
other advices (or should I say "pieces of advice") I'd have to constantly
grep the code to see if the function I'm changing happens to be
advised somewhere.

When the advice is outside Emacs, I don't care so much about breaking it,
but if it's inside Emacs it's a more serious problem,


        Stefan

Re: Multiple major modes

[Richard Stallman]

    Sorry for being vague.  There would be "buffer-local variables", and
    "mode-local variables".

I think you really mean two classes of bindings, right?

    ;; programmatic
    (setq-mode-local 'c-mode my-var t)

    then all buffers in c-mode would get the new value of my-var.

Ok.  What happens if you do (setq my-var 69) in a buffer in C mode?
Does it change the value to 69 for all C mode buffers?

    If a multi-major-mode tool knows a particular major-mode doesn't use
    buffer-local variables, then it would not need to run the major-mode
    function (which could call `kill-all-local-variables') for every
    section switch and lots of work needed to save and restore
    buffer-local values that may or may not be mode-specific would be
    saved.

That's true, if we only consider local variables.  But major modes
set other things too: local keymap, syntax table, abbrev table.
We would need a new way to handle them too, right?

Here is another idea: `run-mode-hooks' could record all local bindings
made by a certain major mode, as mode-local bindings.  This way,
nearly all major modes would "just work" with the new mechanism; we
would not have to change their code.  Of course, `delay-mode-hooks'
would need to cooperate.

Re: Multiple major modes

[Richard Stallman]

    I don't find advising difficult for debugging.

I am surprised.  Why not?

    Actually, the problem with defadvice (for me) is not when the advice is
    enabled, but when it's not because then there's no trace anywhere that the
    function you're looking at has a special relationship to some other piece of
    code elsewhere.

They are two different problems.  Enabled advice can cause confusion
in debugging, because the function may not do what its source code
says; that can cause confusion in debugging.  Separately, changing the
function can break code in advice of that function.

Both of them can be real.  I'm surprised that you think that only the
latter one is important.

When you change a function, often you would search for its callers
to make sure you don't break them.  That search would find advice
for the function, as well as callers.  Thus, I tend to think that
the second problem won't be so bad.

However, when debugging you are likely to need to look
at lots of functions, and it would be a pain to have to search
for advice for each of these functions.

Re: Multiple major modes

[Stephen J. Turnbull]

Richard Stallman writes:

 >     I don't find advising difficult for debugging.
 > 
 > I am surprised.  Why not?

I can't speak for Stefan, but I can say why I rarely have a problem
with it.  Once I've localized to a function (usually via a backtrace),
the first thing I normally do is `describe-function', and then jump to
the source via find-function from the help buffer.

Re: Multiple major modes

[Stefan Monnier]

>     I don't find advising difficult for debugging.
> I am surprised.  Why not?

No idea, but I just can't remember a single case where it has bothered me.

> When you change a function, often you would search for its callers
> to make sure you don't break them.

In most cases the function is "internal" so I only search with C-s.


        Stefan

RE: Multiple major modes

[Drew Adams]

> >     I don't find advising difficult for debugging.
> > I am surprised.  Why not?
>
> No idea, but I just can't remember a single case where it has bothered me.

FWIW -

Perhaps it also depends on how one debugs? I tend not to use edebug; I use
`debug-on-entry' and `(debug)'. Advice bothers me during debugging because
the steps followed include a lot of advice stuff that muddies the waters,
for me.

Re: Multiple major modes

[Johan Bockgård]

Richard Stallman <rms@gnu.org> writes:

> This suggests an idea to me.  Maybe Emacs could highlight the function
> definition somehow, to indicate that the function has been advised.

FWIW, there is already an entry in TODO

    ** Emacs Lisp mode could put an overlay on the defun for every
      function that has advice.  The overlay could have `after-text'
      like " [Function has advice]".  It might look like (defun foo
      [Function has advice] (x y) The overlay could also be a button
      that you could use to view the advice.

> That might solve the problem that makes advising bad for debugging.

Here's an example where advice can be really confusing:

;; See ad-default-compilation-action
(require 'bytecomp)

(defun foo (x y)
  (+ x y))

(defadvice foo (before bar activate)
  (car x))

(foo 1 2)
  =>

Debugger entered--Lisp error: (wrong-type-argument listp 1)
  foo(1 2)
  eval((foo 1 2))
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp)


(If the advice is not compiled "car(1)" is present in the backtrace.)

-- 
Johan Bockgård

Re: Multiple major modes

[Richard Stallman]

    Perhaps it also depends on how one debugs? I tend not to use edebug; I use
    `debug-on-entry' and `(debug)'. Advice bothers me during debugging because
    the steps followed include a lot of advice stuff that muddies the waters,
    for me.

That is also how I do debugging.

Re: Multiple major modes

[Richard Stallman]

      Once I've localized to a function (usually via a backtrace),
    the first thing I normally do is `describe-function', and then jump to
    the source via find-function from the help buffer.

I often do that, but I don't read carefully thru the buffer thaht
`describe-function' displays.  If it said something about advice, I
wouldn't even notice, unless I read carefully thru the buffer to look
for such.  And I am in too much of a hurry to make a practice
of always reading that text carefully.

Maybe this is a consequence of getting older.

Re: Multiple major modes

[Lennart Borgman (gmail)]

Richard Stallman wrote:
>       Once I've localized to a function (usually via a backtrace),
>     the first thing I normally do is `describe-function', and then jump to
>     the source via find-function from the help buffer.
> 
> I often do that, but I don't read carefully thru the buffer thaht
> `describe-function' displays.  If it said something about advice, I
> wouldn't even notice, unless I read carefully thru the buffer to look
> for such.  And I am in too much of a hurry to make a practice
> of always reading that text carefully.

Did someone already say "highlight the advice info" in the help buffer?

> Maybe this is a consequence of getting older.

Yes, of course. There is still just as much to do in life and less time 
left.

Re: Multiple major modes

[Stephen J. Turnbull]

Richard Stallman writes:

 >     Perhaps it also depends on how one debugs? I tend not to use
 >     edebug; I use `debug-on-entry' and `(debug)'. Advice bothers me
 >     during debugging because the steps followed include a lot of
 >     advice stuff that muddies the waters, for me.

 > That is also how I do debugging.

Which suggests adding a feature to the debugger (or to advice) to skip
over advice when debugging.  (This is a "by the way2 suggestion to
improve debugging in the presence of advice one happens to encounter,
not intended as an argument that "advice is OK".)

No, I don't have time (or sufficient knowledge of the debugger and
advice, for that matter) to do it. :-(

Re: Multiple major modes

[Stephen J. Turnbull]

Richard Stallman writes:

 >       Once I've localized to a function (usually via a backtrace),
 >     the first thing I normally do is `describe-function', and then jump to
 >     the source via find-function from the help buffer.
 > 
 > I often do that, but I don't read carefully thru the buffer thaht
 > `describe-function' displays.

Aha.  It would be a good idea to put the "this function is advised"
notice first.  That way at least it would less often catch you by
surprise.

Re: Multiple major modes

[Richard Stallman]

    Which suggests adding a feature to the debugger (or to advice) to skip
    over advice when debugging.

I don't follow.  What would that feature do?

Re: Multiple major modes

[Richard Stallman]

    > I often do that, but I don't read carefully thru the buffer thaht
    > `describe-function' displays.  If it said something about advice, I
    > wouldn't even notice, unless I read carefully thru the buffer to look
    > for such.  And I am in too much of a hurry to make a practice
    > of always reading that text carefully.

    Did someone already say "highlight the advice info" in the help buffer?

I have doubts that it will help that much, but it can't hurt.  Would
someone please implement it?

Moving that advice info closer to the top might also be useful
if it is not already close to the top.

Re: Multiple major modes

[Richard Stallman]

    The use syntax of an overloadable function is similar to programs that
    call `indent-for-tab-command', or `indent-according-to-mode', but
    there is no restriction forcing `according-to-mode' as part of the
    function name.

I understand the similarity, but I see the difference too.
The source code of `indent-according-to-mode' shows that it
calls something that can be set differently by different modes.

That's why I prefer using a variable.

However, you didn't show precisely what this feature looks like.
Perhaps I got the wrong idea of the feature.  Can you show me
what it looks like?

Re[2]: Multiple major modes

[Eric M. Ludlam]

>>> Richard Stallman <rms@gnu.org> seems to think that:
>    The use syntax of an overloadable function is similar to programs that
>    call `indent-for-tab-command', or `indent-according-to-mode', but
>    there is no restriction forcing `according-to-mode' as part of the
>    function name.
>
>I understand the similarity, but I see the difference too.
>The source code of `indent-according-to-mode' shows that it
>calls something that can be set differently by different modes.
>
>That's why I prefer using a variable.
>
>However, you didn't show precisely what this feature looks like.
>Perhaps I got the wrong idea of the feature.  Can you show me
>what it looks like?

Hi,

  Here is a use case from semantic, my parsing tool.  This is a
function that would take a name as found in sources, and split it so
that namespaces (if any) and the name are returned.  In many
languages there is nothing special to do, but in C++, it needs to do
something special.  Instead of putting c++ code in the first file, it
is instead placed in a C specific file that contains many C specific
differences.  Of course, the doc string talks about C++, but only as
an example.

  I picked this case because it's very short.


semantic-analyze.el:
--------------------
(define-overload semantic-analyze-split-name (name)
  "Split a tag NAME into a sequence.
Sometimes NAMES are gathered from the parser that are compounded,
such as in C++ where foo::bar means:
  \"The class BAR in the namespace FOO.\"
Return the string NAME for no change, or a list if it needs to be split.")

(defun semantic-analyze-split-name-default (name)
  "Don't split up NAME by default."
  name)
--------------------

As you can see, there is no implementation at the initial
declaration.  In this example, the `-default' is a separate function,
because a body could be given to the original that would wrap either
the `-default' of a mode specific implementation.


semantic-c.el:
--------------
(define-mode-local-override semantic-analyze-split-name c-mode (name)
  "Split up tag names on colon (:) boundaries."
  (let ((ans (split-string name ":")))
    (if (= (length ans) 1)
	name
      (delete "" ans))))
--------------

If you use "C-h f semantic-analyze-split-name RET" you get:

---------------
semantic-analyze-split-name is a compiled Lisp function in `semantic-analyze.el'.
(semantic-analyze-split-name name)

This function can be overloaded (see `define-mode-local-override' for details).

Split a tag name into a sequence.
Sometimes NAMES are gathered from the parser that are compounded,
such as in C++ where foo::bar means:
  "The class BAR in the namespace FOO."
Return the string name for no change, or a list if it needs to be split.

[back]
---------------

which would be improved if the doc for individual overloads could be
specified, though I haven't done that yet.  Doing so would make it
much easier to find the implementation actually being run in a
particular case.

I hope this helps.
Eric

-- 
          Eric Ludlam:                 zappo@gnu.org, eric@siege-engine.com
   Home: http://www.ludlam.net            Siege: www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: www.gnu.org

Re: Multiple major modes

[Stephen J. Turnbull]

Richard Stallman writes:

 >     Which suggests adding a feature to the debugger (or to advice) to skip
 >     over advice when debugging.
 > 
 > I don't follow.  What would that feature do?

When you step through a function, you can either step into or step
across function calls.  By analogy, when you are stepping through an
advised function, it should be technically possible to step across any
advice code or infrastructure to the point of the (ad-do-it) call.
This might help keep the view of the function that you see in the
source and the view of the function that you get from stepping more
coherent.

Re: Multiple major modes

[Richard Stallman]

I now see that my previous concerns were misguided.
(I had misunderstood what the feature does.)
This feature doesn't cause a problem like that of advice.
It seems like basically a good idea.

I do have a criticism of one part of the API, though.

    (defun semantic-analyze-split-name-default (name)
      "Don't split up NAME by default."
      name)

It seems like a mistake to make the API depend on manipulating symbol
names.  I'd rather have a separate construct to define the default
meaning, giving the standard name, here `semantic-analyze-split-name'.

How does it actually work?

Re: Multiple major modes

[Richard Stallman]

    When you step through a function, you can either step into or step
    across function calls.  By analogy, when you are stepping through an
    advised function, it should be technically possible to step across any
    advice code or infrastructure to the point of the (ad-do-it) call.

To pretend that the advice is not there is counterproductive.
That would only exacerbate the problem caused by advice -- which is
that it runs and you don't notice.

When you do know about the advice, you might sometimes want to run
across it in the debugger without stopping.  A command to do that
could be useful on those special occasions, so I would be happy to
have it installed for that purpose.

Highlight advice (was: Multiple major modes)

[Johan Bockgård]

Richard Stallman <rms@gnu.org> writes:

>     > I often do that, but I don't read carefully thru the buffer thaht
>     > `describe-function' displays.  If it said something about advice, I
>     > wouldn't even notice, unless I read carefully thru the buffer to look
>     > for such.  And I am in too much of a hurry to make a practice
>     > of always reading that text carefully.
>
>     Did someone already say "highlight the advice info" in the help buffer?
>
> I have doubts that it will help that much, but it can't hurt.  Would
> someone please implement it?

2007-09-04  Johan Bockgård  <bojohan@dd.chalmers.se>

	* emacs-lisp/advice.el (ad-make-advised-docstring): Highlight note
	in doc string.


Index: lisp/emacs-lisp/advice.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/emacs-lisp/advice.el,v
retrieving revision 1.54
diff -u -r1.54 advice.el
--- lisp/emacs-lisp/advice.el	28 Jul 2007 19:57:22 -0000	1.54
+++ lisp/emacs-lisp/advice.el	4 Sep 2007 08:44:59 -0000
@@ -2996,7 +2996,9 @@
     (setq usage (if (null usage) t (setq origdoc (cdr usage)) (car usage)))
     (if origdoc (setq paragraphs (list origdoc)))
     (unless (eq style 'plain)
-      (push (concat "This " origtype " is advised.") paragraphs))
+      (push (propertize (concat "This " origtype " is advised.")
+			'face 'font-lock-warning-face)
+	    paragraphs))
     (ad-dolist (class ad-advice-classes)
       (ad-dolist (advice (ad-get-enabled-advices function class))
 	(setq advice-docstring


-- 
Johan Bockgård