bug#22737: 25.1; Finalizer should be optional in dynamic modules

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Jess Balint]

Dynamic modules are really cool so far, but I think finalizers should
not be mandatory for user pointers (alloc.c):

#ifdef HAVE_MODULES
	      else if (mblk->markers[i].m.u_any.type == Lisp_Misc_User_Ptr)
		{
		  struct Lisp_User_Ptr *uptr = &mblk->markers[i].m.u_user_ptr;
		  uptr->finalizer (uptr->p); <----- should NULL-check first
		}
#endif

c.f. https://github.com/emacs-mirror/emacs/blob/master/src/alloc.c#L6893

Thanks!

Jess

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Eli Zaretskii]

> From: Jess Balint <jbalint@gmail.com>
> Date: Thu, 18 Feb 2016 15:52:55 -0600
> 
> Dynamic modules are really cool so far, but I think finalizers should
> not be mandatory for user pointers (alloc.c):
> 
> #ifdef HAVE_MODULES
> 	      else if (mblk->markers[i].m.u_any.type == Lisp_Misc_User_Ptr)
> 		{
> 		  struct Lisp_User_Ptr *uptr = &mblk->markers[i].m.u_user_ptr;
> 		  uptr->finalizer (uptr->p); <----- should NULL-check first
> 		}
> #endif
> 
> c.f. https://github.com/emacs-mirror/emacs/blob/master/src/alloc.c#L6893

Can you tell more about the use case where you needed this change?  A
user-pointer holds a pointer to some unspecified data, and that data
needs to be free'd when the user-pointer object is GC'ed; failure to
do so will cause memory leaks.  When is the above incorrect, or gets
in your way?

Thanks.

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Jess Balint]

[-- Attachment #1: Type: text/plain, Size: 1303 bytes --]

If the data is unspecified it doesn't *necessarily* need to be freed. If I
return a pointer to some global data then I need to create a no-op
finalizer just to please this GC code. In some cases I will be managing
memory a bit more manually and don't care to have Emacs doing anything for
me.

Thx.

Jess

On Fri, Feb 19, 2016 at 3:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Jess Balint <jbalint@gmail.com>
> > Date: Thu, 18 Feb 2016 15:52:55 -0600
> >
> > Dynamic modules are really cool so far, but I think finalizers should
> > not be mandatory for user pointers (alloc.c):
> >
> > #ifdef HAVE_MODULES
> >             else if (mblk->markers[i].m.u_any.type == Lisp_Misc_User_Ptr)
> >               {
> >                 struct Lisp_User_Ptr *uptr =
> &mblk->markers[i].m.u_user_ptr;
> >                 uptr->finalizer (uptr->p); <----- should NULL-check first
> >               }
> > #endif
> >
> > c.f. https://github.com/emacs-mirror/emacs/blob/master/src/alloc.c#L6893
>
> Can you tell more about the use case where you needed this change?  A
> user-pointer holds a pointer to some unspecified data, and that data
> needs to be free'd when the user-pointer object is GC'ed; failure to
> do so will cause memory leaks.  When is the above incorrect, or gets
> in your way?
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 1977 bytes --]

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Eli Zaretskii]

> Date: Tue, 23 Feb 2016 16:47:12 -0600
> From: Jess Balint <jbalint@gmail.com>
> Cc: 22737@debbugs.gnu.org
> 
> If the data is unspecified it doesn't *necessarily* need to be freed. If I return a pointer to some global data then
> I need to create a no-op finalizer just to please this GC code. In some cases I will be managing memory a bit
> more manually and don't care to have Emacs doing anything for me.

I don't think I follow.  How can you manage memory manually when Emacs
does GC whenever it feels like it?  The memory of the objects it GCs
will simply be leaked if you don't have a finalizer.

Can you describe a specific use case where a finalizer would not be
needed?

Thanks.

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Jess Balint]

[-- Attachment #1: Type: text/plain, Size: 1660 bytes --]

Situation #1 - globals:

I have pointers to data that are global (not on the heap). I return these
pointers from module functions so that they may be used as parameters to
other module function calls. These pointers should *never* be freed. In
this case I need to supply a no-op finalizer when creating the user pointer.

Situation #2 - manual memory management:

I have heap-allocated structures whose memory should not be managed by
Emacs. I may return pointers to this data one or many times from module
calls. The data should be freed only when explicitly requested. I may
return many user pointers to the same heap-allocated structure. Even when
all these are freed by Emacs, I still retain a pointer in my module which
may be returned in a future module call. Again, I'm required to supply a
no-op finalizer when creating these user pointers.

Thanks.

Jess


On Tue, Feb 23, 2016 at 9:40 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Tue, 23 Feb 2016 16:47:12 -0600
> > From: Jess Balint <jbalint@gmail.com>
> > Cc: 22737@debbugs.gnu.org
> >
> > If the data is unspecified it doesn't *necessarily* need to be freed. If
> I return a pointer to some global data then
> > I need to create a no-op finalizer just to please this GC code. In some
> cases I will be managing memory a bit
> > more manually and don't care to have Emacs doing anything for me.
>
> I don't think I follow.  How can you manage memory manually when Emacs
> does GC whenever it feels like it?  The memory of the objects it GCs
> will simply be leaked if you don't have a finalizer.
>
> Can you describe a specific use case where a finalizer would not be
> needed?
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 2279 bytes --]

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Eli Zaretskii]

> Date: Fri, 26 Feb 2016 10:17:26 -0600
> From: Jess Balint <jbalint@gmail.com>
> Cc: 22737@debbugs.gnu.org
> 
> Situation #1 - globals:
> 
> I have pointers to data that are global (not on the heap). I return these pointers from module functions so that
> they may be used as parameters to other module function calls. These pointers should *never* be freed. In
> this case I need to supply a no-op finalizer when creating the user pointer.
> 
> Situation #2 - manual memory management:
> 
> I have heap-allocated structures whose memory should not be managed by Emacs. I may return pointers to
> this data one or many times from module calls. The data should be freed only when explicitly requested. I may
> return many user pointers to the same heap-allocated structure. Even when all these are freed by Emacs, I
> still retain a pointer in my module which may be returned in a future module call. Again, I'm required to supply
> a no-op finalizer when creating these user pointers.

What will happen if such objects are exposed to Lisp, copied or
assigned to other Lisp variables, etc.?  Won't this cause all kinds of
trouble, like modifying one such object will magically modify several
others, which share its storage?

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Jess Balint]

[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]

On Fri, Feb 26, 2016 at 12:36 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Fri, 26 Feb 2016 10:17:26 -0600
> > From: Jess Balint <jbalint@gmail.com>
> > Cc: 22737@debbugs.gnu.org
> >
> > Situation #1 - globals:
> >
> > I have pointers to data that are global (not on the heap). I return
> these pointers from module functions so that
> > they may be used as parameters to other module function calls. These
> pointers should *never* be freed. In
> > this case I need to supply a no-op finalizer when creating the user
> pointer.
> >
> > Situation #2 - manual memory management:
> >
> > I have heap-allocated structures whose memory should not be managed by
> Emacs. I may return pointers to
> > this data one or many times from module calls. The data should be freed
> only when explicitly requested. I may
> > return many user pointers to the same heap-allocated structure. Even
> when all these are freed by Emacs, I
> > still retain a pointer in my module which may be returned in a future
> module call. Again, I'm required to supply
> > a no-op finalizer when creating these user pointers.
>
> What will happen if such objects are exposed to Lisp, copied or
> assigned to other Lisp variables, etc.?  Won't this cause all kinds of
> trouble, like modifying one such object will magically modify several
> others, which share its storage?
>

This is how C code works. If you return a pointer from a function, you may
have to free that pointer yourself or you may not. You may get the same
pointer back from multiple calls to the same function. If you use the
pointer after it's been freed, it's your problem. You need to agree with
the owner of the pointer how the memory is to be managed. With pointers,
modifications to the underlying data are visible by all who have a pointer
to the data. I wouldn't call this "magically modifying others".

Jess

[-- Attachment #2: Type: text/html, Size: 2503 bytes --]

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Eli Zaretskii]

> Date: Fri, 26 Feb 2016 12:53:20 -0600
> From: Jess Balint <jbalint@gmail.com>
> Cc: 22737@debbugs.gnu.org
> 
>  What will happen if such objects are exposed to Lisp, copied or
>  assigned to other Lisp variables, etc.? Won't this cause all kinds of
>  trouble, like modifying one such object will magically modify several
>  others, which share its storage?
> 
> This is how C code works. If you return a pointer from a function, you may have to free that pointer yourself or
> you may not. You may get the same pointer back from multiple calls to the same function. If you use the
> pointer after it's been freed, it's your problem. You need to agree with the owner of the pointer how the
> memory is to be managed. With pointers, modifications to the underlying data are visible by all who have a
> pointer to the data. I wouldn't call this "magically modifying others".

In C, yes.  But we are talking about Lisp objects here.

Am I the only one who is uneasy with supporting such Lisp objects?  If
so, I will shut up and install the changes.  Daniel, John, what's your
opinion on this?

Thanks.

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Jess Balint]

[-- Attachment #1: Type: text/plain, Size: 1540 bytes --]

On Fri, Feb 26, 2016 at 3:33 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Fri, 26 Feb 2016 12:53:20 -0600
> > From: Jess Balint <jbalint@gmail.com>
> > Cc: 22737@debbugs.gnu.org
> >
> >  What will happen if such objects are exposed to Lisp, copied or
> >  assigned to other Lisp variables, etc.? Won't this cause all kinds of
> >  trouble, like modifying one such object will magically modify several
> >  others, which share its storage?
> >
> > This is how C code works. If you return a pointer from a function, you
> may have to free that pointer yourself or
> > you may not. You may get the same pointer back from multiple calls to
> the same function. If you use the
> > pointer after it's been freed, it's your problem. You need to agree with
> the owner of the pointer how the
> > memory is to be managed. With pointers, modifications to the underlying
> data are visible by all who have a
> > pointer to the data. I wouldn't call this "magically modifying others".
>
> In C, yes.  But we are talking about Lisp objects here.
>
> Am I the only one who is uneasy with supporting such Lisp objects?  If
> so, I will shut up and install the changes.  Daniel, John, what's your
> opinion on this?
>
> Thanks.
>

All I'm asking for is to allow the code to accept a NULL finalizer. This
means no finalizer will be called. It's a clear and simple semantic. Upside
is that I (and others who do not want Emacs to free their pointers) will
not have to create a no-op function unnecessarily to supply a finalizer to
Emacs.

Thanks.

Jess

[-- Attachment #2: Type: text/html, Size: 2260 bytes --]

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 2102 bytes --]

On 02/26/2016 01:51 PM, Jess Balint wrote:
> On Fri, Feb 26, 2016 at 3:33 PM, Eli Zaretskii <eliz@gnu.org
> <mailto:eliz@gnu.org>> wrote:
> 
>     > Date: Fri, 26 Feb 2016 12:53:20 -0600
>     > From: Jess Balint <jbalint@gmail.com <mailto:jbalint@gmail.com>>
>     > Cc: 22737@debbugs.gnu.org <mailto:22737@debbugs.gnu.org>
>     >
>     >  What will happen if such objects are exposed to Lisp, copied or
>     >  assigned to other Lisp variables, etc.? Won't this cause all kinds of
>     >  trouble, like modifying one such object will magically modify several
>     >  others, which share its storage?
>     >
>     > This is how C code works. If you return a pointer from a function, you may have to free that pointer yourself or
>     > you may not. You may get the same pointer back from multiple calls to the same function. If you use the
>     > pointer after it's been freed, it's your problem. You need to agree with the owner of the pointer how the
>     > memory is to be managed. With pointers, modifications to the underlying data are visible by all who have a
>     > pointer to the data. I wouldn't call this "magically modifying others".
> 
>     In C, yes.  But we are talking about Lisp objects here.
> 
>     Am I the only one who is uneasy with supporting such Lisp objects?  If
>     so, I will shut up and install the changes.  Daniel, John, what's your
>     opinion on this?
> 
>     Thanks.
> 
> 
> All I'm asking for is to allow the code to accept a NULL finalizer. This
> means no finalizer will be called. It's a clear and simple semantic.
> Upside is that I (and others who do not want Emacs to free their
> pointers) will not have to create a no-op function unnecessarily to
> supply a finalizer to Emacs.

A no-op function is trivial though; creating it forces you to think
about whether you actually need to free the resulting memory. I think
it's more important to discourage memory leaks and simplify the
semantics of the finalizer parameter than to make this rare (I think)
use case slightly easier for module implementors.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#22737: 25.1; Finalizer should be optional in dynamic modules

[Jess Balint]

[-- Attachment #1: Type: text/plain, Size: 2419 bytes --]

On Fri, Feb 26, 2016 at 3:55 PM, Daniel Colascione <dancol@dancol.org>
wrote:

> On 02/26/2016 01:51 PM, Jess Balint wrote:
> > On Fri, Feb 26, 2016 at 3:33 PM, Eli Zaretskii <eliz@gnu.org
> > <mailto:eliz@gnu.org>> wrote:
> >
> >     > Date: Fri, 26 Feb 2016 12:53:20 -0600
> >     > From: Jess Balint <jbalint@gmail.com <mailto:jbalint@gmail.com>>
> >     > Cc: 22737@debbugs.gnu.org <mailto:22737@debbugs.gnu.org>
> >     >
> >     >  What will happen if such objects are exposed to Lisp, copied or
> >     >  assigned to other Lisp variables, etc.? Won't this cause all
> kinds of
> >     >  trouble, like modifying one such object will magically modify
> several
> >     >  others, which share its storage?
> >     >
> >     > This is how C code works. If you return a pointer from a function,
> you may have to free that pointer yourself or
> >     > you may not. You may get the same pointer back from multiple calls
> to the same function. If you use the
> >     > pointer after it's been freed, it's your problem. You need to
> agree with the owner of the pointer how the
> >     > memory is to be managed. With pointers, modifications to the
> underlying data are visible by all who have a
> >     > pointer to the data. I wouldn't call this "magically modifying
> others".
> >
> >     In C, yes.  But we are talking about Lisp objects here.
> >
> >     Am I the only one who is uneasy with supporting such Lisp objects?
> If
> >     so, I will shut up and install the changes.  Daniel, John, what's
> your
> >     opinion on this?
> >
> >     Thanks.
> >
> >
> > All I'm asking for is to allow the code to accept a NULL finalizer. This
> > means no finalizer will be called. It's a clear and simple semantic.
> > Upside is that I (and others who do not want Emacs to free their
> > pointers) will not have to create a no-op function unnecessarily to
> > supply a finalizer to Emacs.
>
> A no-op function is trivial though; creating it forces you to think
> about whether you actually need to free the resulting memory. I think
> it's more important to discourage memory leaks and simplify the
> semantics of the finalizer parameter than to make this rare (I think)
> use case slightly easier for module implementors.


Ok, I can respect that. I don't really agree, but... so be it. If this is
the way you want it to work, maybe make_user_ptr() should return nil.
Otherwise this will lead to segfaults.

Jess

[-- Attachment #2: Type: text/html, Size: 3425 bytes --]