From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: =?UTF-8?Q?Aur=C3=A9lien_Aptel?= Newsgroups: gmane.emacs.devel Subject: Re: Dynamic loading progress Date: Sun, 23 Aug 2015 21:12:40 +0200 Message-ID: References: <54E0DEF8.7020901@dancol> <83egpruiyp.fsf@gnu.org> <54E0FF93.2000104@dancol.org> <833865vp4d.fsf@gnu.org> <54E2355A.90@87.69.4.28> <83vbj1u020.fsf@gnu.org> <54E24CA4.9020601@dancol.org> <83h9uk7ddb.fsf@gnu.org> <54E382A5.5030408@dancol.org> <54F789B2.6030105@dancol.org> <87egnel6ac.fsf@lifelogs.com> <87vbgpk1po.fsf@lifelogs.com> <85mw20gmeo.fsf@stephe-leake.org> <878u97nyjn.fsf@lifelogs.com> <86d1yirnqw.fsf@stephe-leake.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Trace: ger.gmane.org 1440357172 5221 80.91.229.3 (23 Aug 2015 19:12:52 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 23 Aug 2015 19:12:52 +0000 (UTC) Cc: Emacs development discussions To: Stephen Leake , Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Aug 23 21:12:52 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZTahE-0001cK-QX for ged-emacs-devel@m.gmane.org; Sun, 23 Aug 2015 21:12:49 +0200 Original-Received: from localhost ([::1]:48905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZTahD-0001va-L0 for ged-emacs-devel@m.gmane.org; Sun, 23 Aug 2015 15:12:47 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48029) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZTah8-0001sj-RF for emacs-devel@gnu.org; Sun, 23 Aug 2015 15:12:43 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZTah7-0000hp-Jg for emacs-devel@gnu.org; Sun, 23 Aug 2015 15:12:42 -0400 Original-Received: from mail-ig0-x22b.google.com ([2607:f8b0:4001:c05::22b]:38074) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZTah7-0000hZ-FV for emacs-devel@gnu.org; Sun, 23 Aug 2015 15:12:41 -0400 Original-Received: by igfj19 with SMTP id j19so43221423igf.1 for ; Sun, 23 Aug 2015 12:12:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=BEUUUwojJfyzk0YxmLceTs+R4nMsJcglpf4AE9plKOo=; b=TCCXu1th84KqNrPx7hBDxn1PpZqOv4LNgINvmltjSYFfFMS9/MEJJCdtjnMj4ESogd 7F0n28rVe9Vge58hegZmrLB6SrQ0C7EXeNV8grO+tDY3l+JcF7OMIyfaJo8P0LQlUB5k YeMENnahifCFrGdF5AcvvKb+7LeKPxNwEaE1C3PE+W43qjOyA7U1gVbuBGx50awFVxqZ y90y7XZ0oOHS2hYr14MPstggy3UerOn7GnAzJaPX0yo3XCI1ZOviFgeIIhpgli6A21vH Lth15oB/1Ml1xXhDJCtt70uG7m/0uoxBrIHouXgAfbVN8PIgT8yBPVi/Q7+GzfXqxAqs HVxg== X-Received: by 10.50.13.100 with SMTP id g4mr12056619igc.62.1440357160590; Sun, 23 Aug 2015 12:12:40 -0700 (PDT) Original-Received: by 10.36.146.193 with HTTP; Sun, 23 Aug 2015 12:12:40 -0700 (PDT) In-Reply-To: <86d1yirnqw.fsf@stephe-leake.org> X-Google-Sender-Auth: m_2sNmEeldV5KcphLTJ1Hz5pk_s X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2607:f8b0:4001:c05::22b X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:189086 Archived-At: Hi all, Daniel, Following Ted last off-list email I looked again at the module code. Thanks again to Daniel for the API design proposal you did few month ago and the general explanations, it was insightful and it really helped me. I would really like to have the module API in emacs 25 so I recently revisited my code and re-read the dynamic loading threads on emacs-devel but I'm still stuck on the memory aspect. I'll re-explain briefly the problem: The module API [1] lets module writer make new Lisp_Object (hidden as opaque pointers named 'emacs_value') via make_(), funcall(), type_of() and intern() functions. But if a module function calls core Emacs functions, those functions it turn might call the GC, which would free and invalidate the objects made by the module. Here are 2 examples of the same problem: - Local values which are created/can-be-freed inside the module function. a = make_string("foo") b = make_string("bar") len = funcall(eval, 1, [length_sym, b]) // a (and b?) might be invalid now if the GC was triggered inside the funcall call. In the core Emacs code, we have the GCPRO macro to deal with that use-case. - Values created and reused between 2 module calls. emacs_value global_a; module_func_A() global_a = make_string("a") module_func_B(): // global_a might be invalid now if the GC was triggered between func_A and func_B return global_a In the core Emacs code, global_a is usually a defvar. Daniel was talking about tables mapping emacs_value to Lisp_Object, and having manual memory management in the module API. But I'm not sure I understood what you meant. Here's another proposition (which might just be what you were trying to explain): We add a hash table at the GC root that maps module names to a list of values currently owned by the module. Any markable/collectable emacs_value (all Lisp_Object types except for numeric ones basically) created by the module API are added to that hash table (appended at the module's list). When a module doesn't need a value anymore, it explicitly calls a function in the module API that removes it from the module's list. The GC walks the object tree like usual, including the module hash table. This is more or less what GCPRO/UNGCPRO does, I think (temporally add objects to the GC root). And if remember correctly Stefan was against exposing it in the module API. I've proposed a hash table to know which values a module owns. That way we can sort of "unload" a module by simply removing the module entry from the hash table (we still have to deal with unbinding the module's functions). The GC will do the rest. Any feedback, other ideas, welcome. 1: https://github.com/aaptel/emacs-dynamic-module/blob/dynamic-modules-2/src/emacs_module.h