bug#21683: 25.0.50; `advice-add` doesn't work for compiled primitive functions

bug#21683: 25.0.50; `advice-add` doesn't work for compiled primitive functions

[Vitalie Spinu]

Hi,

Put this function into a file, compile the file and then load the file:

    
  (defun foobar ()
   (save-restriction
     (widen)))


then:

  
  (defun restrict-widen (orig-widen)
    (message "here")
    (apply orig-widen nil))
  
  (advice-add 'widen :around #restrict-widen)
  
  (foobar)



The advice is not triggered.


Eval the `foobar` definition interactively and the advice will start
working. The problem occurs only for primitive c functions. Elisp
functions are fine. It doesn't work with 24.4.1 either.


  Vitalie




In GNU Emacs 25.0.50.3 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2015-09-26
Repository revision: 139e55a58466058a8c9ae54536cb790ac7b7cc64
Windowing system distributor 'The X.Org Foundation', version 11.0.11600000
System Description:	Ubuntu 14.10

bug#21683: 25.0.50; `advice-add` doesn't work for compiled primitive functions

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

This is documented behavior:
https://www.gnu.org/software/emacs/manual/html_node/elisp/Advising-Named-Functions.html
It is possible to advise a primitive (see What Is a Function
<https://www.gnu.org/software/emacs/manual/html_node/elisp/What-Is-a-Function.html#What-Is-a-Function>),
but one should typically *not* do so, for two reasons. Firstly, some
primitives are used by the advice mechanism, and advising them could cause
an infinite recursion. Secondly, many primitives are called directly from
C, and such calls ignore advice; hence, one ends up in a confusing
situation where some calls (occurring from Lisp code) obey the advice and
other calls (from C code) do not.

Vitalie Spinu <spinuvit@gmail.com> schrieb am Mi., 14. Okt. 2015 um
03:55 Uhr:

>
> Hi,
>
> Put this function into a file, compile the file and then load the file:
>
>
>   (defun foobar ()
>    (save-restriction
>      (widen)))
>
>
> then:
>
>
>   (defun restrict-widen (orig-widen)
>     (message "here")
>     (apply orig-widen nil))
>
>   (advice-add 'widen :around #restrict-widen)
>
>   (foobar)
>
>
>
> The advice is not triggered.
>
>
> Eval the `foobar` definition interactively and the advice will start
> working. The problem occurs only for primitive c functions. Elisp
> functions are fine. It doesn't work with 24.4.1 either.
>
>
>   Vitalie
>
>
>
>
> In GNU Emacs 25.0.50.3 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
>  of 2015-09-26
> Repository revision: 139e55a58466058a8c9ae54536cb790ac7b7cc64
> Windowing system distributor 'The X.Org Foundation', version 11.0.11600000
> System Description:     Ubuntu 14.10
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 2665 bytes --]

bug#21683: 25.0.50; `advice-add` doesn't work for compiled primitive functions

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1776 bytes --]

On 10/18/2015 02:16 PM, Philipp Stephani wrote:
> This is documented behavior:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Advising-Named-Functions.html
> It is possible to advise a primitive (see What Is a Function
> <https://www.gnu.org/software/emacs/manual/html_node/elisp/What-Is-a-Function.html#What-Is-a-Function>),
> but one should typically /not/ do so, for two reasons. Firstly, some
> primitives are used by the advice mechanism, and advising them could
> cause an infinite recursion. Secondly, many primitives are called
> directly from C, and such calls ignore advice; hence, one ends up in a
> confusing situation where some calls (occurring from Lisp code) obey the
> advice and other calls (from C code) do not.

If we wanted to allow advising of primitives such that we also redirect
direct calls from C, it wouldn't be that hard: the idea is to either 1)
insert enough NOPs at the start of every function to encode an absolute
jump[1], or 2) associate with every function a Lisp variable and rest it
for nil-ness (which would be a very cheap and very predictable test
against zero with Qnil being all zero bits) on entry.

Option #1 is more clever and thus more fun to write, but option #2
probably suffices.

I just haven't had a good reason to want to advise a primitive, but if
someone wants to do this work, I wouldn't object to it.

[1] You don't actually have to encode enough bytes for an absolute jump.
You just need to be able to encode enough bytes for a backward jump to
the region before the function; this region can then encode your
absolute jump. If we align functions properly, we get this space for
free half the time. See
http://blogs.msdn.com/b/oldnewthing/archive/2011/09/21/10214405.aspx


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#21683: 25.0.50; `advice-add` doesn't work for compiled primitive functions

[Vitalie Spinu]

>> On Sun, Oct 18 2015 14:25, Daniel Colascione wrote:

> On 10/18/2015 02:16 PM, Philipp Stephani wrote:

>> This is documented behavior:

Well, conceptually the call is from elisp, even if it's a compiled code. What is
technically happening in the compiled code shouldn't be the user's concern.

I think that piece of documentation should be nuanced that advice doesn't work
in compiled elisp calls. I took me 4 hours to figure out why my font-lock code
was chaotically working here and not there.

> I just haven't had a good reason to want to advise a primitive, but if someone
> wants to do this work, I wouldn't object to it.

I am trying hard to make polymode [1] functional for complex sub-modes like
c-lang. For that I need to restrict `widen` and `parse-partial-sexp` in order
for the sub-mode to have an illusion that a region of a buffer is a whole
buffer.

> If we wanted to allow advising of primitives such that we also redirect direct
> calls from C, it wouldn't be that hard:

That would be so great!

 Vitalie

[1] https://github.com/vspinu/polymode/

bug#21683: 25.0.50; `advice-add` doesn't work for compiled primitive functions

[Dmitry Gutov]

On 10/19/2015 06:51 PM, Vitalie Spinu wrote:
> I am trying hard to make polymode [1] functional for complex sub-modes like
> c-lang. For that I need to restrict `widen` and `parse-partial-sexp` in order
> for the sub-mode to have an illusion that a region of a buffer is a whole
> buffer.

Yesss. This has been discussed a few times already: you can't advice 
`widen', and hence you can't make a "perfect" multiple-major-modes solution.

The current thinking on that subject is to use prog-indentation-context 
for indentation (it must be supported in the respective major modes as 
well), and introduce something similar for font-lock/syntax-ppss.

Doing anything with parse-partial-sexp is dommed to fail.