Note on 109327

Note on 109327

[Dmitry Antipov]

This massive change may break Windows and NS builds. I checked all possible
GNU/Linux variants (without X, Lucid, Motif, GTK2/3), but can't check others.
Please report possible Windows/NS build errors as soon as possible.

Dmitry

Re: Note on 109327

[Dan Nicolaescu]

Dmitry Antipov <dmantipov@yandex.ru> writes:

> This massive change may break Windows and NS builds. I checked all possible
> GNU/Linux variants (without X, Lucid, Motif, GTK2/3), but can't check others.
> Please report possible Windows/NS build errors as soon as possible.

What's the motivation for changes like:
-  f->icon_name = Qnil;
+  FVAR (f, icon_name) = Qnil;
?

The new version looks more obfuscated ...

Re: Note on 109327

[Dmitry Antipov]

On 07/31/2012 05:13 PM, Dan Nicolaescu wrote:

> What's the motivation for changes like:
> -  f->icon_name = Qnil;
> +  FVAR (f, icon_name) = Qnil;
> ?
>
> The new version looks more obfuscated ...

Yes. I'm always thinking about improving internal stuff, GC at first.
For me, the main motivation for BVAR, KVAR, FVAR etc. is the ability
to catch the moment when the pointer (e.g. Lisp_Object) within buffer,
or keyboard, of frame, etc. is read or written, which may be useful
to implement write barriers (see, for exmaple,
http://www.hoelzle.org/publications/write-barrier.pdf, but do not
get confused with http://en.wikipedia.org/wiki/Memory_barrier).

Dmitry

Re: Note on 109327

[Tom Tromey]

Dmitry> Yes. I'm always thinking about improving internal stuff, GC at first.
Dmitry> For me, the main motivation for BVAR, KVAR, FVAR etc. is the ability
Dmitry> to catch the moment when the pointer (e.g. Lisp_Object) within buffer,
Dmitry> or keyboard, of frame, etc. is read or written, which may be useful
Dmitry> to implement write barriers (see, for exmaple,
Dmitry> http://www.hoelzle.org/publications/write-barrier.pdf, but do not
Dmitry> get confused with http://en.wikipedia.org/wiki/Memory_barrier).

I think to do this well you will need separate macros for getting and
setting.

Tom

Re: Note on 109327

[Dmitry Antipov]

On 07/31/2012 05:33 PM, Tom Tromey wrote:

> Dmitry> Yes. I'm always thinking about improving internal stuff, GC at first.
> Dmitry> For me, the main motivation for BVAR, KVAR, FVAR etc. is the ability
> Dmitry> to catch the moment when the pointer (e.g. Lisp_Object) within buffer,
> Dmitry> or keyboard, of frame, etc. is read or written, which may be useful
> Dmitry> to implement write barriers (see, for exmaple,
> Dmitry> http://www.hoelzle.org/publications/write-barrier.pdf, but do not
> Dmitry> get confused with http://en.wikipedia.org/wiki/Memory_barrier).
>
> I think to do this well you will need separate macros for getting and
> setting.

Sure, but it's almost impossible to do this at once. At the very beginning,
it's possible to "overestimate" barrier assuming that each XVAR (obj, field)
changes FIELD in OBJ; in the future, reads and writes may be separated, thus
giving a precise write barrier.

Dmitry

Re: Note on 109327

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 1345 bytes --]

() Dmitry Antipov <dmantipov@yandex.ru>
() Tue, 31 Jul 2012 17:50:43 +0400

   > I think to do this well you will need separate macros for getting
   > and setting.

   Sure, but it's almost impossible to do this at once.

Why?

   At the very beginning, it's possible to "overestimate" barrier
   assuming that each XVAR (obj, field) changes FIELD in OBJ; in the
   future, reads and writes may be separated, thus giving a precise
   write barrier.

I think you're saying that you prefer to do:

a1. substitute object access (whether LHS or RHS) w/ XVAR (obj, field)
a2. distinguish LHS (which could benefit from optimization) from RHS
a3. substitue LHS ‘XVAR (...) = value’ w/ SETVAR (obj, field, value)

instead of:

b1. distinguish LHS (which could benefit from optimization) from RHS
b2. substitute LHS object access w/ SETVAR (obj, field, value)
b3. substitute RHS object access w/ XVAR (obj, field)

Is my understanding correct?

-- 
Thien-Thi Nguyen ..................................... GPG key: 4C807502
.                  NB: ttn at glug dot org is not me                   .
.                 (and has not been since 2007 or so)                  .
.                        ACCEPT NO SUBSTITUTES                         .
........... please send technical questions to mailing lists ...........

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Note on 109327

[Dmitry Antipov]

On 08/01/2012 01:47 AM, Thien-Thi Nguyen wrote:

>     > I think to do this well you will need separate macros for getting
>     > and setting.
>
>     Sure, but it's almost impossible to do this at once.
>
> Why?

As an exercise, try this for 'contents' member of Lisp_Vector.

>
>     At the very beginning, it's possible to "overestimate" barrier
>     assuming that each XVAR (obj, field) changes FIELD in OBJ; in the
>     future, reads and writes may be separated, thus giving a precise
>     write barrier.
>
> I think you're saying that you prefer to do:
>
> a1. substitute object access (whether LHS or RHS) w/ XVAR (obj, field)
> a2. distinguish LHS (which could benefit from optimization) from RHS
> a3. substitue LHS ‘XVAR (...) = value’ w/ SETVAR (obj, field, value)
>
> instead of:
>
> b1. distinguish LHS (which could benefit from optimization) from RHS
> b2. substitute LHS object access w/ SETVAR (obj, field, value)
> b3. substitute RHS object access w/ XVAR (obj, field)
>
> Is my understanding correct?

a1) is definitely the first, since this gives a base to design an
"overestimated" write barrier (each XVAR (obj, field) issues the
barrier). Next, it should be possible to  implement generational
pass for GC (which collects only an objects changed since the last
collection). This is expected to be very inefficient because write
barrier is hugely overestimated. Finally, it's time for a3) or b2),
i.e. removing the barrier from XVAR (obj, field) and installing it
only at XSETVAR (obj, field, value), thus reducing "overestimation"
of the barrier.

Dmitry

Re: Note on 109327

[Juanma Barranquero]

On Tue, Jul 31, 2012 at 2:38 PM, Dmitry Antipov <dmantipov@yandex.ru> wrote:

> Please report possible Windows/NS build errors as soon as possible.

fontset.c: In function 'dump_fontset':
fontset.c:2119:6: error: 'struct frame' has no member named 'name'

    Juanma

Re: Note on 109327

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Tue, 31 Jul 2012 15:54:38 +0200
> Cc: Emacs development discussions <emacs-devel@gnu.org>
> 
> On Tue, Jul 31, 2012 at 2:38 PM, Dmitry Antipov <dmantipov@yandex.ru> wrote:
> 
> > Please report possible Windows/NS build errors as soon as possible.
> 
> fontset.c: In function 'dump_fontset':
> fontset.c:2119:6: error: 'struct frame' has no member named 'name'

Which has nothing to do with Windows, but with --enable-checking.

This problem, however, is more serious, IMO:

  gcc -I. -c -gdwarf-2 -g3  -DEMACSDEBUG -fno-crossjumping  -Id:/usr/include/libxml2 -DGLYPH_DEBUG=1 -Demacs=1 -I../lib -I../nt/inc -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -o oo/i386/w32menu.o w32menu.c
  w32menu.c: In function `w32_menu_show':
  w32menu.c:857: error: structure has no member named `title_'
  makefile:322: recipe for target `oo/i386/w32menu.o' failed
  make[1]: *** [oo/i386/w32menu.o] Error 1

AFAIU, it means that coccinelle made an incorrect replacement.

I fixed both of these in revision 109332.

Btw, can't we have in Emacs a feature that would allow such
refactoring?  The replacements don't seem too complex to me.  Adding
this to Emacs will both allow getting rid of an external tool, and be
a valuable addition to Emacs abilities.  WDYT?

Re: Note on 109327

[Juanma Barranquero]

On Tue, Jul 31, 2012 at 7:04 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> Which has nothing to do with Windows, but with --enable-checking.

I just saw the trouble, sent the report and ran away. Busy.

> Btw, can't we have in Emacs a feature that would allow such
> refactoring?  The replacements don't seem too complex to me.  Adding
> this to Emacs will both allow getting rid of an external tool, and be
> a valuable addition to Emacs abilities.  WDYT?

It'd be great.

    Juanma

Re: Note on 109327

[joakim]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juanma Barranquero <lekktu@gmail.com>
>> Date: Tue, 31 Jul 2012 15:54:38 +0200
>> Cc: Emacs development discussions <emacs-devel@gnu.org>
>> 
>> On Tue, Jul 31, 2012 at 2:38 PM, Dmitry Antipov <dmantipov@yandex.ru> wrote:
>> 
>> > Please report possible Windows/NS build errors as soon as possible.
>> 
>> fontset.c: In function 'dump_fontset':
>> fontset.c:2119:6: error: 'struct frame' has no member named 'name'
>
> Which has nothing to do with Windows, but with --enable-checking.
>
> This problem, however, is more serious, IMO:
>
>   gcc -I. -c -gdwarf-2 -g3  -DEMACSDEBUG -fno-crossjumping  -Id:/usr/include/libxml2 -DGLYPH_DEBUG=1 -Demacs=1 -I../lib -I../nt/inc -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -o oo/i386/w32menu.o w32menu.c
>   w32menu.c: In function `w32_menu_show':
>   w32menu.c:857: error: structure has no member named `title_'
>   makefile:322: recipe for target `oo/i386/w32menu.o' failed
>   make[1]: *** [oo/i386/w32menu.o] Error 1
>
> AFAIU, it means that coccinelle made an incorrect replacement.
>
> I fixed both of these in revision 109332.
>
> Btw, can't we have in Emacs a feature that would allow such
> refactoring?  The replacements don't seem too complex to me.  Adding
> this to Emacs will both allow getting rid of an external tool, and be
> a valuable addition to Emacs abilities.  WDYT?
>

http://cedet.sourceforge.net/srecode.shtml

The idea is that the two CEDET tools, Semantic and SRecode can cooperate
to provide such features, I think.

-- 
Joakim Verona

Re: Note on 109327

[Dmitry Antipov]

On 07/31/2012 09:35 PM, joakim@verona.se wrote:

>> Btw, can't we have in Emacs a feature that would allow such
>> refactoring?  The replacements don't seem too complex to me.  Adding
>> this to Emacs will both allow getting rid of an external tool, and be
>> a valuable addition to Emacs abilities.  WDYT?
>>
>
> http://cedet.sourceforge.net/srecode.shtml
>
> The idea is that the two CEDET tools, Semantic and SRecode can cooperate
> to provide such features, I think.

Hm... not familiar with it too much. IIUC it's mostly for an interactive
editing and navigation. But, if you know how to use it to do, for example,
project-wide replacement of foo (X, Y) to foo (Y, X), where X and Y are
arbitrary valid C expressions (which may include nested calls of foo),
this might be the right direction.

Dmitry

Re: Note on 109327

[Dmitry Antipov]

On 07/31/2012 09:04 PM, Eli Zaretskii wrote:

> This problem, however, is more serious, IMO:
>
>    gcc -I. -c -gdwarf-2 -g3  -DEMACSDEBUG -fno-crossjumping  -Id:/usr/include/libxml2 -DGLYPH_DEBUG=1 -Demacs=1 -I../lib -I../nt/inc -DHAVE_NTGUI=1 -DUSE_CRT_DLL=1 -o oo/i386/w32menu.o w32menu.c
>    w32menu.c: In function `w32_menu_show':
>    w32menu.c:857: error: structure has no member named `title_'
>    makefile:322: recipe for target `oo/i386/w32menu.o' failed
>    make[1]: *** [oo/i386/w32menu.o] Error 1
>
> AFAIU, it means that coccinelle made an incorrect replacement.

Yes, it's not a silver bullet (yet). And, of course, I'm not a heavily
experienced user.

> I fixed both of these in revision 109332.

Thanks.

> Btw, can't we have in Emacs a feature that would allow such
> refactoring?  The replacements don't seem too complex to me.  Adding
> this to Emacs will both allow getting rid of an external tool, and be
> a valuable addition to Emacs abilities.  WDYT?

IIUC real refactoring of C should imply preprocessing and full
syntactic analysis, so, I'm thinking about GCC plugin.

Nevertheless, I suppose that it should be possible to build
simple refactoring engine on top of cc-mode, and such an engine
may provide basic features like intelligent renaming of identifiers
or tweaking function arguments.

Dmitry

Re: Refactoring in Emacs (was: Note on 109327)

[Eli Zaretskii]

> Date: Tue, 31 Jul 2012 21:42:44 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> Cc: Juanma Barranquero <lekktu@gmail.com>, emacs-devel@gnu.org
> 
> > Btw, can't we have in Emacs a feature that would allow such
> > refactoring?  The replacements don't seem too complex to me.  Adding
> > this to Emacs will both allow getting rid of an external tool, and be
> > a valuable addition to Emacs abilities.  WDYT?
> 
> IIUC real refactoring of C should imply preprocessing and full
> syntactic analysis

Why is that?  Can you give a couple of examples?

Even if a full-fledged refactoring engine does need that, we could
have a somewhat limited one that doesn't, could we?  AFAICS, the
refactoring you did until now were all quite simple.

> so, I'm thinking about GCC plugin.

Why do we need a GCC plugin?  What kind of information is needed from
the compiler?

> Nevertheless, I suppose that it should be possible to build
> simple refactoring engine on top of cc-mode, and such an engine
> may provide basic features like intelligent renaming of identifiers
> or tweaking function arguments.

That would be very good, I think.

> From: joakim@verona.se
> Date: Tue, 31 Jul 2012 19:35:35 +0200
> Cc: Juanma Barranquero <lekktu@gmail.com>, dmantipov@yandex.ru,
> 	emacs-devel@gnu.org
> 
> http://cedet.sourceforge.net/srecode.shtml
> 
> The idea is that the two CEDET tools, Semantic and SRecode can cooperate
> to provide such features, I think.

Emacs has some of both; is what we have not enough?

Re: Refactoring in Emacs

[Dmitry Antipov]

On 07/31/2012 10:06 PM, Eli Zaretskii wrote:

>> IIUC real refactoring of C should imply preprocessing and full
>> syntactic analysis
>
> Why is that?  Can you give a couple of examples?

It's easy to use regexps to make

foo (2, 1);

from

foo (1, 2);

But it may be very tricky or even impossible to do the same for:

1) foo (bar (x, y, z), baz (a, b, c));

2) foo (bar (x, y, foo (z, 1)), baz (foo (a, b) > 0 ? c : d, e, f));

3) foo (
#ifdef AAA
      a,
#else
      b,
#endif
      c > 0 ? c : -c);

etc.

For 2), syntax tree may looks like:

          ++++++++++++[foo]++++++++++++
          +                           +
     +++[bar]+++              ++++++[baz]++++++
     +    +    +              +       +       +
     x    y +[foo]+     ++++[> ?]++++ e       f
            +     +     +     +     +
            z     1  +[foo]+  c     d
                     +     +
                     a     b

So, refactoring is (hm, relatively) simple: you should traverse the tree,
look for

    +[foo]+
    +     +
    x     y

pattern and swap subtrees.

Check http://gcc-melt.org, it looks very similar to what I'm thinking about.

Dmitry

Re: Refactoring in Emacs : CEDET

[Eric M. Ludlam]

On 08/01/2012 12:14 AM, Dmitry Antipov wrote:
> On 07/31/2012 10:06 PM, Eli Zaretskii wrote:
>
>>> IIUC real refactoring of C should imply preprocessing and full
>>> syntactic analysis
>>
>> Why is that? Can you give a couple of examples?
>

Hi,

   I missed earlier parts of this thread, so am replying a bit late.

   In CEDET, the function 'semantic-symref-symbol' uses the C parser 
built into semantic, plus external tools like GNU Global, or just plain 
grep to find symbols.  The list buffer that shows all the hits is also a 
refactoring mode.  You can select which hits are correct, and do mass 
renames.  It also has a fancy way to create a keyboard macro that it 
will apply to all the hits.

   I'm describing the state of things in the CEDET bzr repository.  I 
don't recall what state this mode was in when CEDET was last merged into 
Emacs.  The symref functions certainly has some limitations as it tends 
to stick to symbols defined by the user, and the code to handle 
polymorphism hasn't been written yet, but those could be overcome if 
there was sufficient interest.

Eric

Re: Refactoring in Emacs : CEDET

[Eli Zaretskii]

> Date: Wed, 01 Aug 2012 07:41:24 -0400
> From: "Eric M. Ludlam" <ericludlam@gmail.com>
> CC: Eli Zaretskii <eliz@gnu.org>, joakim@verona.se, emacs-devel@gnu.org
> 
>    In CEDET, the function 'semantic-symref-symbol' uses the C parser 
> built into semantic, plus external tools like GNU Global, or just plain 
> grep to find symbols.  The list buffer that shows all the hits is also a 
> refactoring mode.  You can select which hits are correct, and do mass 
> renames.  It also has a fancy way to create a keyboard macro that it 
> will apply to all the hits.

Is this described somewhere in documentation, so that one could
experiment with these features?

Re: Refactoring in Emacs : CEDET

[Eric Ludlam]

  On 08/01/2012 10:50 AM, Eli Zaretskii wrote:
>> Date: Wed, 01 Aug 2012 07:41:24 -0400
>> From: "Eric M. Ludlam"<ericludlam@gmail.com>
>> CC: Eli Zaretskii<eliz@gnu.org>, joakim@verona.se, emacs-devel@gnu.org
>>
>>     In CEDET, the function 'semantic-symref-symbol' uses the C parser
>> built into semantic, plus external tools like GNU Global, or just plain
>> grep to find symbols.  The list buffer that shows all the hits is also a
>> refactoring mode.  You can select which hits are correct, and do mass
>> renames.  It also has a fancy way to create a keyboard macro that it
>> will apply to all the hits.
> Is this described somewhere in documentation, so that one could
> experiment with these features?


There is a section on the symbol reference functions in the semantic 
user guide.  It doesn't get into the rename and macro feature though.  
I'll look into updating the texi file in the CEDET repository.

Eric

Re: Refactoring in Emacs

[Richard Stallman]

    2) foo (bar (x, y, foo (z, 1)), baz (foo (a, b) > 0 ? c : d, e, f));

It seems to me that this code would be clearer if it used a variables
or two to avoid having such a complex expression.

--
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call

Re: Refactoring in Emacs

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Tue, 31 Jul 2012 21:42:44 +0400
>> From: Dmitry Antipov <dmantipov@yandex.ru>
>> 
>> IIUC real refactoring of C should imply preprocessing and full
>> syntactic analysis
>
> Why is that?  Can you give a couple of examples?
>
> Even if a full-fledged refactoring engine does need that, we could
> have a somewhat limited one that doesn't, could we?  AFAICS, the
> refactoring you did until now were all quite simple.

Preprocessing might even be a disadvantage, as it would mean skipping
blocks of code that are currently #ifdef'ed out because of the
configuration constants of the platform on which the refactoring is
being done.

Re: Note on 109327

[Jan Djärv]

Hello.

31 jul 2012 kl. 14:38 skrev Dmitry Antipov:

> This massive change may break Windows and NS builds. I checked all possible
> GNU/Linux variants (without X, Lucid, Motif, GTK2/3), but can't check others.
> Please report possible Windows/NS build errors as soon as possible.
> 

I fixed the NS build errors.

	Jan D.