unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
       [not found] ` <E1a3UDm-0005my-HI@vcs.savannah.gnu.org>
@ 2016-01-03 13:59   ` Daniel Colascione
  2016-01-03 15:45     ` Paul Eggert
  0 siblings, 1 reply; 21+ messages in thread
From: Daniel Colascione @ 2016-01-03 13:59 UTC (permalink / raw)
  To: emacs-devel, Stefan Monnier

[-- Attachment #1: Type: text/plain, Size: 338 bytes --]


On 11/30/2015 11:34 AM, Stefan Monnier wrote:> branch: emacs-25
> commit 3eb93c07f7a60ac9ce8a16f10c3afd5a3a31243a
> Author: Stefan Monnier <monnier@iro.umontreal.ca>
> Commit: Stefan Monnier <monnier@iro.umontreal.ca>
> 
>     Rely on conservative stack scanning to find "emacs_value"s

There was no consensus on this change.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-01-03 13:59   ` [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s Daniel Colascione
@ 2016-01-03 15:45     ` Paul Eggert
  2016-03-29 10:07       ` Philipp Stephani
  0 siblings, 1 reply; 21+ messages in thread
From: Paul Eggert @ 2016-01-03 15:45 UTC (permalink / raw)
  To: Daniel Colascione, emacs-devel, Stefan Monnier

Daniel Colascione wrote:
> There was no consensus on this change.

What are the problems with it? Although Eli and I tweaked it later to fix some 
warnings and sign-extension issues and make it go a bit faster, I didn't see any 
major issues with the basic idea.



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-01-03 15:45     ` Paul Eggert
@ 2016-03-29 10:07       ` Philipp Stephani
  2016-03-29 12:59         ` Stefan Monnier
  2016-03-30  1:35         ` Paul Eggert
  0 siblings, 2 replies; 21+ messages in thread
From: Philipp Stephani @ 2016-03-29 10:07 UTC (permalink / raw)
  To: Paul Eggert, Daniel Colascione, emacs-devel, Stefan Monnier

[-- Attachment #1: Type: text/plain, Size: 622 bytes --]

Paul Eggert <eggert@cs.ucla.edu> schrieb am So., 3. Jan. 2016 um 16:46 Uhr:

> Daniel Colascione wrote:
> > There was no consensus on this change.
>
> What are the problems with it? Although Eli and I tweaked it later to fix
> some
> warnings and sign-extension issues and make it go a bit faster, I didn't
> see any
> major issues with the basic idea.
>
>
- From a user's perspective, it removes NULL as a sentinel value to signal
an error.
- From a developer perspective, it adds low-level complexity,
bit-twiddling, and possibly undefined behavior to a module that should only
contain glue code between two interfaces.

[-- Attachment #2: Type: text/html, Size: 936 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-29 10:07       ` Philipp Stephani
@ 2016-03-29 12:59         ` Stefan Monnier
  2016-03-30  1:35         ` Paul Eggert
  1 sibling, 0 replies; 21+ messages in thread
From: Stefan Monnier @ 2016-03-29 12:59 UTC (permalink / raw)
  To: emacs-devel

> - From a user's perspective, it removes NULL as a sentinel value to signal
> an error.

In Lisp_Object (and hence in emacs_value) you can have as many sentinel
values as you want already (we typically use Qnil or uninterned symbols
for that).

> - From a developer perspective, it adds low-level complexity,
> bit-twiddling, and possibly undefined behavior to a module that should only
> contain glue code between two interfaces.

I agree, so long as "it" refers to "wide-int".


        Stefan




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-29 10:07       ` Philipp Stephani
  2016-03-29 12:59         ` Stefan Monnier
@ 2016-03-30  1:35         ` Paul Eggert
  2016-03-30 18:26           ` Philipp Stephani
  1 sibling, 1 reply; 21+ messages in thread
From: Paul Eggert @ 2016-03-30  1:35 UTC (permalink / raw)
  To: Philipp Stephani, Daniel Colascione, emacs-devel, Stefan Monnier

On 03/29/2016 03:07 AM, Philipp Stephani wrote:
>
>
>
> - From a user's perspective, it removes NULL as a sentinel value to 
> signal an error.

It's not clear that this feature is worth the hassle of supporting it. 
As Stefan mentioned, one can easily use Qnil etc. as sentinels.

> - From a developer perspective, it adds low-level complexity, 
> bit-twiddling, and possibly undefined behavior to a module that should 
> only contain glue code between two interfaces.

The old approach was also complicated; its data structures were more 
complex and typically a bit slower, and it had more lines of code and it 
was typically more of a pain to debug at the C level. The new approach's 
use of undefined bit-twiddling isn't bad, given that even the old 
approach bit-twiddled with abandon by calling XCAR, XCDR and the like.

If the main objection here is seeing bit operations like "<<" in 
emacs-module.c, we could move uses of these operations from 
emacs-module.c to lisp.h so that emacs-module.c could be more abstract. 
Rearrange the deck chairs, so to speak.



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30  1:35         ` Paul Eggert
@ 2016-03-30 18:26           ` Philipp Stephani
  2016-03-30 18:28             ` Daniel Colascione
                               ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: Philipp Stephani @ 2016-03-30 18:26 UTC (permalink / raw)
  To: Paul Eggert, Daniel Colascione, emacs-devel, Stefan Monnier

[-- Attachment #1: Type: text/plain, Size: 1728 bytes --]

Paul Eggert <eggert@cs.ucla.edu> schrieb am Mi., 30. März 2016 um 03:35 Uhr:

> On 03/29/2016 03:07 AM, Philipp Stephani wrote:
> >
> >
> >
> > - From a user's perspective, it removes NULL as a sentinel value to
> > signal an error.
>
> It's not clear that this feature is worth the hassle of supporting it.
>

If emacs_value* were a real pointer, then the feature would come for free.


> As Stefan mentioned, one can easily use Qnil etc. as sentinels.
>

These aren't sentinels because functions can return nil without existing
non-locally.


>
> > - From a developer perspective, it adds low-level complexity,
> > bit-twiddling, and possibly undefined behavior to a module that should
> > only contain glue code between two interfaces.
>
> The old approach was also complicated; its data structures were more
> complex and typically a bit slower, and it had more lines of code and it
> was typically more of a pain to debug at the C level.


The old approach relied only on standard C library functions and
well-defined and well-known behavior. It didn't require knowledge about the
internals of Lisp_Object.


> The new approach's
> use of undefined bit-twiddling isn't bad, given that even the old
> approach bit-twiddled with abandon by calling XCAR, XCDR and the like.
>

XCAR etc. certainly can't be avoided. We should try to reduce undefined
behavior though, not add more.


>
> If the main objection here is seeing bit operations like "<<" in
> emacs-module.c,


It is not the main objection.


> we could move uses of these operations from
> emacs-module.c to lisp.h so that emacs-module.c could be more abstract.
> Rearrange the deck chairs, so to speak.
>

[-- Attachment #2: Type: text/html, Size: 2797 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 18:26           ` Philipp Stephani
@ 2016-03-30 18:28             ` Daniel Colascione
  2016-03-30 18:42             ` Paul Eggert
  2016-03-30 21:13             ` Stefan Monnier
  2 siblings, 0 replies; 21+ messages in thread
From: Daniel Colascione @ 2016-03-30 18:28 UTC (permalink / raw)
  To: Philipp Stephani, Paul Eggert, emacs-devel, Stefan Monnier


[-- Attachment #1.1: Type: text/plain, Size: 872 bytes --]

On 03/30/2016 11:26 AM, Philipp Stephani wrote:
> 
> 
> Paul Eggert <eggert@cs.ucla.edu <mailto:eggert@cs.ucla.edu>> schrieb am
> Mi., 30. März 2016 um 03:35 Uhr:
> 
>     On 03/29/2016 03:07 AM, Philipp Stephani wrote:
>     >
>     >
>     >
>     > - From a user's perspective, it removes NULL as a sentinel value to
>     > signal an error.
> 
>     It's not clear that this feature is worth the hassle of supporting it.
> 
> 
> If emacs_value* were a real pointer, then the feature would come for free.

As would support for arbitrary future GC schemes and robustness against
arbitrary clever things compilers might do with pointers in memory.
There are lots of reasons to use a real handle table and lots of reasons
to avoid this kind of value stuffing.

But hey, we can't even get signal handlers right, so we shouldn't expect
too much.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 18:26           ` Philipp Stephani
  2016-03-30 18:28             ` Daniel Colascione
@ 2016-03-30 18:42             ` Paul Eggert
  2016-03-30 18:48               ` Daniel Colascione
  2016-03-30 21:13             ` Stefan Monnier
  2 siblings, 1 reply; 21+ messages in thread
From: Paul Eggert @ 2016-03-30 18:42 UTC (permalink / raw)
  To: Philipp Stephani, Daniel Colascione, emacs-devel, Stefan Monnier

On 03/30/2016 11:26 AM, Philipp Stephani wrote:
>
>     It's not clear that this feature is worth the hassle of supporting it.
>
>
> If emacs_value* were a real pointer, then the feature would come for free.

This is begging the question, no? There are good reasons to be 
suspicious of null values, regardless of whether they are represented by 
null pointers or by something else.

>     As Stefan mentioned, one can easily use Qnil etc. as sentinels.
>
>
> These aren't sentinels because functions can return nil without 
> existing non-locally.

If there is a need for C code to return a value that is either an Emacs 
value or something else, then a cleaner way to specify this is as some 
sort of discriminated union that is either an Emacs value, or something 
else. Attempting to shoehorn an "I'm not an Emacs value!" value into the 
emacs_value type leads to unnecessary confusion.  At least, any such 
union type should be called "emacs_value_option", or something like 
that, to clearly warn the maintainer that the type is an Emacs value or 
is something else.

>
> XCAR etc. certainly can't be avoided. We should try to reduce 
> undefined behavior though, not add more.
>

Yes, there are good arguments against the XCAR etc. approach that is 
used throughout the Emacs interpreter. However, there are also good 
arguments for the approach, and the current Emacs design clearly uses 
and assumes it and it works well enough in practice to overcome the 
mostly-theoretical objections raised in this thread. Regardless of the 
implementation strategy chosen by the Emacs core, it's better if the 
module system lives in harmony with the Emacs interpreter instead of 
attempting to fight against it.



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 18:42             ` Paul Eggert
@ 2016-03-30 18:48               ` Daniel Colascione
  0 siblings, 0 replies; 21+ messages in thread
From: Daniel Colascione @ 2016-03-30 18:48 UTC (permalink / raw)
  To: Paul Eggert, Philipp Stephani, emacs-devel, Stefan Monnier


[-- Attachment #1.1: Type: text/plain, Size: 2488 bytes --]

On 03/30/2016 11:42 AM, Paul Eggert wrote:
> On 03/30/2016 11:26 AM, Philipp Stephani wrote:
>>
>>     It's not clear that this feature is worth the hassle of supporting
>> it.
>>
>>
>> If emacs_value* were a real pointer, then the feature would come for
>> free.
> 
> This is begging the question, no? There are good reasons to be
> suspicious of null values, regardless of whether they are represented by
> null pointers or by something else.

And there are good reasons to want unambiguous sentinels. See "in-band
signaling" under "awful ideas that should already be dead, but aren't".

> 
>>     As Stefan mentioned, one can easily use Qnil etc. as sentinels.
>>
>>
>> These aren't sentinels because functions can return nil without
>> existing non-locally.
> 
> If there is a need for C code to return a value that is either an Emacs
> value or something else, then a cleaner way to specify this is as some
> sort of discriminated union that is either an Emacs value, or something
> else.

A handle indirection serves exactly this purpose.

> Attempting to shoehorn an "I'm not an Emacs value!" value into the
> emacs_value type leads to unnecessary confusion. 

It doesn't in Java, which uses precisely the approach Philipp and I prefer.

> At least, any such
> union type should be called "emacs_value_option", or something like
> that, to clearly warn the maintainer that the type is an Emacs value or
> is something else.

Oh, come on. That's like saying malloc should return
memory_or_something_t and not void*. This isn't Rust.


>> XCAR etc. certainly can't be avoided. We should try to reduce
>> undefined behavior though, not add more.
>>
> 
> Yes, there are good arguments against the XCAR etc. approach that is
> used throughout the Emacs interpreter. However, there are also good
> arguments for the approach, and the current Emacs design clearly uses
> and assumes it and it works well enough in practice to overcome the
> mostly-theoretical objections raised in this thread. Regardless of the
> implementation strategy chosen by the Emacs core, it's better if the
> module system lives in harmony with the Emacs interpreter instead of
> attempting to fight against it.

An indirection table in no way fights against the interpreter. In fact,
it makes life better for the interpreter, since if raw Lisp_Value
objects are no longer public, the interpreter has fewer constraints
under which it must operate.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 18:26           ` Philipp Stephani
  2016-03-30 18:28             ` Daniel Colascione
  2016-03-30 18:42             ` Paul Eggert
@ 2016-03-30 21:13             ` Stefan Monnier
  2016-03-30 21:18               ` Daniel Colascione
  2 siblings, 1 reply; 21+ messages in thread
From: Stefan Monnier @ 2016-03-30 21:13 UTC (permalink / raw)
  To: Philipp Stephani; +Cc: Paul Eggert, Daniel Colascione, emacs-devel

>> It's not clear that this feature is worth the hassle of supporting it.
> If emacs_value* were a real pointer, then the feature would come for free.

We have 30 years of experience with Lisp_Object.
It works well and is efficient.

The other scheme originally implemented was terribly inefficient (just
like the still present signal-emasculating scheme) and with no
experience to show for it.

If you want a "emacs_value_pointer" abstraction, you can layer it on top
of the current API.  The reverse is not true.


        Stefan



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 21:13             ` Stefan Monnier
@ 2016-03-30 21:18               ` Daniel Colascione
  2016-03-30 21:30                 ` Stefan Monnier
  0 siblings, 1 reply; 21+ messages in thread
From: Daniel Colascione @ 2016-03-30 21:18 UTC (permalink / raw)
  To: Stefan Monnier, Philipp Stephani; +Cc: Paul Eggert, emacs-devel


[-- Attachment #1.1: Type: text/plain, Size: 860 bytes --]

On 03/30/2016 02:13 PM, Stefan Monnier wrote:
>>> It's not clear that this feature is worth the hassle of supporting it.
>> If emacs_value* were a real pointer, then the feature would come for free.
> 
> We have 30 years of experience with Lisp_Object.
> It works well and is efficient.

And zero years of experience with anything not statically linked into
Emacs core.

> The other scheme originally implemented was terribly inefficient

No it wasn't. It's one additional pointer dereference.

> (just
> like the still present signal-emasculating scheme) and with no
> experience to show for it.
> 
> If you want a "emacs_value_pointer" abstraction, you can layer it on top
> of the current API.  The reverse is not true.

And I'd bet you'd still love to just ship lisp.h with Emacs and make
(call-process "gcc" ...) to load modules.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 21:18               ` Daniel Colascione
@ 2016-03-30 21:30                 ` Stefan Monnier
  2016-03-31 18:46                   ` Philipp Stephani
  2016-03-31 19:24                   ` Daniel Colascione
  0 siblings, 2 replies; 21+ messages in thread
From: Stefan Monnier @ 2016-03-30 21:30 UTC (permalink / raw)
  To: Daniel Colascione; +Cc: Philipp Stephani, Paul Eggert, emacs-devel

> And zero years of experience with anything not statically linked into
> Emacs core.

Actually not exactly since we've used dynamic linking on the Windows
side for a few years now.

But I see no reason to expect any special issues showing up because of
it in any case: none of the Lisp_Object low-level twiddling we do seems
to depend in any way on whether code is linked statically or dynamically.

>> The other scheme originally implemented was terribly inefficient
> No it wasn't. It's one additional pointer dereference.

I'm not talking about that.  I'm talking about the cost of
allocating/freeing those boxes to which you point.

> And I'd bet you'd still love to just ship lisp.h with Emacs and make
> (call-process "gcc" ...) to load modules.

I would have been happy with it, but I'm also happy with a more
abstract option like the emacs-module.h we have (save for the insane
signal mangling, obviously).


        Stefan



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 21:30                 ` Stefan Monnier
@ 2016-03-31 18:46                   ` Philipp Stephani
  2016-03-31 19:24                   ` Daniel Colascione
  1 sibling, 0 replies; 21+ messages in thread
From: Philipp Stephani @ 2016-03-31 18:46 UTC (permalink / raw)
  To: Stefan Monnier, Daniel Colascione; +Cc: Paul Eggert, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> schrieb am Mi., 30. März 2016 um
23:31 Uhr:

> >> The other scheme originally implemented was terribly inefficient
> > No it wasn't. It's one additional pointer dereference.
>
> I'm not talking about that.  I'm talking about the cost of
> allocating/freeing those boxes to which you point.
>
>
Please provide benchmarks that clearly show that this is "terribly
inefficient".

[-- Attachment #2: Type: text/html, Size: 750 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-30 21:30                 ` Stefan Monnier
  2016-03-31 18:46                   ` Philipp Stephani
@ 2016-03-31 19:24                   ` Daniel Colascione
  2016-03-31 21:29                     ` Stefan Monnier
  1 sibling, 1 reply; 21+ messages in thread
From: Daniel Colascione @ 2016-03-31 19:24 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Philipp Stephani, Paul Eggert, emacs-devel


[-- Attachment #1.1: Type: text/plain, Size: 828 bytes --]

On 03/30/2016 02:30 PM, Stefan Monnier wrote:
>> And zero years of experience with anything not statically linked into
>> Emacs core.
> 
> Actually not exactly since we've used dynamic linking on the Windows
> side for a few years now.
> 
> But I see no reason to expect any special issues showing up because of
> it in any case: none of the Lisp_Object low-level twiddling we do seems
> to depend in any way on whether code is linked statically or dynamically.
> 
>>> The other scheme originally implemented was terribly inefficient
>> No it wasn't. It's one additional pointer dereference.
> 
> I'm not talking about that.  I'm talking about the cost of
> allocating/freeing those boxes to which you point.

That's cheap: you can do it with linear allocation out of an array. Why
would that be expensive?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-31 19:24                   ` Daniel Colascione
@ 2016-03-31 21:29                     ` Stefan Monnier
  2016-04-01 18:34                       ` Daniel Colascione
  0 siblings, 1 reply; 21+ messages in thread
From: Stefan Monnier @ 2016-03-31 21:29 UTC (permalink / raw)
  To: Daniel Colascione; +Cc: Philipp Stephani, Paul Eggert, emacs-devel

> That's cheap: you can do it with linear allocation out of an array.
> Why would that be expensive?

That's very expensive compared to doing nothing.

It means that you have to allocate a new array, loop through the old one
calling your "cheap allocation" function on each element, instead of
just passing the array pointer untouched.

Of course, the advantage of your scheme is that you get to do manual
memory management, with the endless hours of fun that entails.

"There's worse, but it's more expensive!"


        Stefan



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-03-31 21:29                     ` Stefan Monnier
@ 2016-04-01 18:34                       ` Daniel Colascione
  2016-04-01 19:05                         ` Stefan Monnier
  0 siblings, 1 reply; 21+ messages in thread
From: Daniel Colascione @ 2016-04-01 18:34 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Philipp Stephani, Paul Eggert, emacs-devel


[-- Attachment #1.1: Type: text/plain, Size: 711 bytes --]

On 03/31/2016 02:29 PM, Stefan Monnier wrote:
>> That's cheap: you can do it with linear allocation out of an array.
>> Why would that be expensive?
> 
> That's very expensive compared to doing nothing.
> 
> It means that you have to allocate a new array, 

Once.

> loop through the old one
> calling your "cheap allocation" function on each element, instead of
> just passing the array pointer untouched.

It's a pointer comparison of something that will be in L1 cache anyway.

Of course it's slower than doing nothing. But you have not demonstrated
that it is meaningfully slower, meanwhile, you're ignoring the
compatibility benefits and consigning everyone to stack scanning forever.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-04-01 18:34                       ` Daniel Colascione
@ 2016-04-01 19:05                         ` Stefan Monnier
  2016-04-01 19:15                           ` Daniel Colascione
  0 siblings, 1 reply; 21+ messages in thread
From: Stefan Monnier @ 2016-04-01 19:05 UTC (permalink / raw)
  To: Daniel Colascione; +Cc: Philipp Stephani, Paul Eggert, emacs-devel

>>> That's cheap: you can do it with linear allocation out of an array.
>>> Why would that be expensive?
>> That's very expensive compared to doing nothing.
>> It means that you have to allocate a new array, 
> Once.

No, what I describe is what happens every single time you go from one
side of the fence to the other (i.e. when Elisp calls to an external
module, as well as every time that external module does a funcall to an
Elisp function).

>> loop through the old one calling your "cheap allocation" function on
>> each element, instead of just passing the array pointer untouched.
> It's a pointer comparison of something that will be in L1 cache anyway.

What pointer comparison?  I'm talking about an allocation of an
array (one element per argument of the function being called) plus
a loop through this array.

> Of course it's slower than doing nothing. But you have not demonstrated
> that it is meaningfully slower,

We're talking about the building blocks of a language construct.

There's no reason why a funcall from a module to an Elisp function (or
vice versa) should be significantly slower than if it were implemented
"in core".

Currently we're still pretty far from this ideal, because of the
signal-catching (which additionally forces us to allocate+fill+pass
a whole new "struct emacs_env_25" every time, instead of passing it once
and for all when opening the module).

Using "Lisp_Object = emacs_value" lets us get a bit closer to
this ideal, tho.

> meanwhile, you're ignoring the compatibility benefits and consigning
> everyone to stack scanning forever.

I don't foresee any disappearance of stack-scanning in the next ten
years.  And any such disappearance if it ever happens will have much
further reaching consequences on Emacs's C code, so we'll be *thrilled*
to break backward compatibility for external modules at that point.


        Stefan



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-04-01 19:05                         ` Stefan Monnier
@ 2016-04-01 19:15                           ` Daniel Colascione
  2016-04-01 22:46                             ` Stefan Monnier
  0 siblings, 1 reply; 21+ messages in thread
From: Daniel Colascione @ 2016-04-01 19:15 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Philipp Stephani, Paul Eggert, emacs-devel


[-- Attachment #1.1: Type: text/plain, Size: 3180 bytes --]

On 04/01/2016 12:05 PM, Stefan Monnier wrote:
>>>> That's cheap: you can do it with linear allocation out of an array.
>>>> Why would that be expensive?
>>> That's very expensive compared to doing nothing.
>>> It means that you have to allocate a new array, 
>> Once.
> 
> No, what I describe is what happens every single time you go from one
> side of the fence to the other (i.e. when Elisp calls to an external
> module, as well as every time that external module does a funcall to an
> Elisp function).

I meant that we don't need to allocate a new array on every call. We can
keep it around thread-locally and reuse it.

>>> loop through the old one calling your "cheap allocation" function on
>>> each element, instead of just passing the array pointer untouched.
>> It's a pointer comparison of something that will be in L1 cache anyway.
> 
> What pointer comparison?  I'm talking about an allocation of an
> array (one element per argument of the function being called) plus
> a loop through this array.
> 
>> Of course it's slower than doing nothing. But you have not demonstrated
>> that it is meaningfully slower,
> 
> We're talking about the building blocks of a language construct.

My point stands. I can't think of a real-world application of modules
where the extra cycles matter. I'd much making modules truly independent
of the Emacs internal ABI. What specific use cases do you imagining this
scheme hurting?

> There's no reason why a funcall from a module to an Elisp function (or
> vice versa) should be significantly slower than if it were implemented
> "in core".

It's going to be slower no matter what due to the indirection through a
function pointer.

> Currently we're still pretty far from this ideal, because of the
> signal-catching (which additionally forces us to allocate+fill+pass
> a whole new "struct emacs_env_25" every time, instead of passing it once
> and for all when opening the module).

Are we? If so, that's a bug. We be reusing this structure. We shouldn't
incur a penalty any more severe than setjmp, which in my benchmarks is
very fast.

> Using "Lisp_Object = emacs_value" lets us get a bit closer to
> this ideal, tho.
> 
>> meanwhile, you're ignoring the compatibility benefits and consigning
>> everyone to stack scanning forever.
> 
> I don't foresee any disappearance of stack-scanning in the next ten
> years.

Emacs may end up moving to another VM, and that VM may very well do
precise GC.

There's a cautionary tale from Python here: PyPy is much more efficient
implementation of the language than the traditional CPython interpreter
is, but PyPy gets very little use, mostly because it doesn't interact
well with CPython extension modules, which have CPython implementation
details baked in. If Python had used a scheme more like the one Philipp
and I favor, using PyPy would be a much smaller hurdle.

> And any such disappearance if it ever happens will have much
> further reaching consequences on Emacs's C code, so we'll be *thrilled*
> to break backward compatibility for external modules at that point.

Why should modules have to break?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-04-01 19:15                           ` Daniel Colascione
@ 2016-04-01 22:46                             ` Stefan Monnier
  2016-04-03  1:11                               ` Daniel Colascione
  0 siblings, 1 reply; 21+ messages in thread
From: Stefan Monnier @ 2016-04-01 22:46 UTC (permalink / raw)
  To: Daniel Colascione; +Cc: Philipp Stephani, Paul Eggert, emacs-devel

>> We're talking about the building blocks of a language construct.
> My point stands. I can't think of a real-world application of modules
> where the extra cycles matter.

How 'bout a module which implements an LALR parser and hence makes
repeated "module_funcalls" to an Elisp primitive that reads the next character?
The overhead of the call itself compared to the work done in that call
is likely to be very important to the overall performance.

>> There's no reason why a funcall from a module to an Elisp function (or
>> vice versa) should be significantly slower than if it were implemented
>> "in core".
> It's going to be slower no matter what due to the indirection through a
> function pointer.

A bit slower, yes, it's indeed inevitable.  But we should strive to get
as close as possible to that ideal.

>> Currently we're still pretty far from this ideal, because of the
>> signal-catching (which additionally forces us to allocate+fill+pass
>> a whole new "struct emacs_env_25" every time, instead of passing it once
>> and for all when opening the module).
> Are we? If so, that's a bug. We be reusing this structure. We shouldn't
> incur a penalty any more severe than setjmp, which in my benchmarks is
> very fast.

Look at the code, it calls initialize_environment right there in
Finternal_module_call.

> Emacs may end up moving to another VM, and that VM may very well do
> precise GC.

Sure.  I still predict that if we do switch to another VM, backward
compatibility of external modules because "emacs_value == Lisp_Object"
rather than "emacs_value == Lisp_Object*" will be the least of
our concerns.


        Stefan



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-04-01 22:46                             ` Stefan Monnier
@ 2016-04-03  1:11                               ` Daniel Colascione
  2016-04-03  5:49                                 ` Stefan Monnier
  0 siblings, 1 reply; 21+ messages in thread
From: Daniel Colascione @ 2016-04-03  1:11 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: Philipp Stephani, Paul Eggert, emacs-devel


[-- Attachment #1.1: Type: text/plain, Size: 2196 bytes --]



On 04/01/2016 03:46 PM, Stefan Monnier wrote:
>>> We're talking about the building blocks of a language construct.
>> My point stands. I can't think of a real-world application of modules
>> where the extra cycles matter.
> 
> How 'bout a module which implements an LALR parser and hence makes
> repeated "module_funcalls" to an Elisp primitive that reads the next character?
> The overhead of the call itself compared to the work done in that call
> is likely to be very important to the overall performance.

You shouldn't have to talk to lisp to read buffers. Even if you wanted
to talk to lisp for it, you could batch requests and returning the next
N characters, amortizing the overhead.

> 
>>> There's no reason why a funcall from a module to an Elisp function (or
>>> vice versa) should be significantly slower than if it were implemented
>>> "in core".
>> It's going to be slower no matter what due to the indirection through a
>> function pointer.
> 
> A bit slower, yes, it's indeed inevitable.  But we should strive to get
> as close as possible to that ideal.
> 
>>> Currently we're still pretty far from this ideal, because of the
>>> signal-catching (which additionally forces us to allocate+fill+pass
>>> a whole new "struct emacs_env_25" every time, instead of passing it once
>>> and for all when opening the module).
>> Are we? If so, that's a bug. We be reusing this structure. We shouldn't
>> incur a penalty any more severe than setjmp, which in my benchmarks is
>> very fast.
> 
> Look at the code, it calls initialize_environment right there in
> Finternal_module_call.
> 
>> Emacs may end up moving to another VM, and that VM may very well do
>> precise GC.
> 
> Sure.  I still predict that if we do switch to another VM, backward
> compatibility of external modules because "emacs_value == Lisp_Object"
> rather than "emacs_value == Lisp_Object*" will be the least of
> our concerns.

On several occasions, you've mentioned that a VM move would be a
compatibility flag day for modules no matter what. I don't see why you'd
say so. The module API under discussion could be implemented equally
well by guilemacs today.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s
  2016-04-03  1:11                               ` Daniel Colascione
@ 2016-04-03  5:49                                 ` Stefan Monnier
  0 siblings, 0 replies; 21+ messages in thread
From: Stefan Monnier @ 2016-04-03  5:49 UTC (permalink / raw)
  To: Daniel Colascione; +Cc: Philipp Stephani, Paul Eggert, emacs-devel

> You shouldn't have to talk to lisp to read buffers. Even if you wanted
> to talk to lisp for it, you could batch requests and returning the next
> N characters, amortizing the overhead.

That's exactly my point: the slower API will force you to structure your
code differently to work around the inefficiency.


        Stefan



^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2016-04-03  5:49 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <20151130193446.22218.14685@vcs.savannah.gnu.org>
     [not found] ` <E1a3UDm-0005my-HI@vcs.savannah.gnu.org>
2016-01-03 13:59   ` [Emacs-diffs] emacs-25 3eb93c0: Rely on conservative stack scanning to find "emacs_value"s Daniel Colascione
2016-01-03 15:45     ` Paul Eggert
2016-03-29 10:07       ` Philipp Stephani
2016-03-29 12:59         ` Stefan Monnier
2016-03-30  1:35         ` Paul Eggert
2016-03-30 18:26           ` Philipp Stephani
2016-03-30 18:28             ` Daniel Colascione
2016-03-30 18:42             ` Paul Eggert
2016-03-30 18:48               ` Daniel Colascione
2016-03-30 21:13             ` Stefan Monnier
2016-03-30 21:18               ` Daniel Colascione
2016-03-30 21:30                 ` Stefan Monnier
2016-03-31 18:46                   ` Philipp Stephani
2016-03-31 19:24                   ` Daniel Colascione
2016-03-31 21:29                     ` Stefan Monnier
2016-04-01 18:34                       ` Daniel Colascione
2016-04-01 19:05                         ` Stefan Monnier
2016-04-01 19:15                           ` Daniel Colascione
2016-04-01 22:46                             ` Stefan Monnier
2016-04-03  1:11                               ` Daniel Colascione
2016-04-03  5:49                                 ` Stefan Monnier

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).