From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Thompson, David" Subject: Re: Hack the (init) system! Date: Thu, 3 Sep 2015 20:25:29 -0400 Message-ID: References: <87oahjz05m.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33730) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZXeow-0008UO-0K for Guix-devel@gnu.org; Thu, 03 Sep 2015 20:25:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZXeos-0001OJ-Mv for Guix-devel@gnu.org; Thu, 03 Sep 2015 20:25:33 -0400 Received: from mail-yk0-f174.google.com ([209.85.160.174]:32820) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZXeos-0001O6-IP for Guix-devel@gnu.org; Thu, 03 Sep 2015 20:25:30 -0400 Received: by ykei199 with SMTP id i199so6319753yke.0 for ; Thu, 03 Sep 2015 17:25:30 -0700 (PDT) In-Reply-To: <87oahjz05m.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Cc: Guix-devel On Thu, Sep 3, 2015 at 5:02 PM, Ludovic Court=C3=A8s wrote: > Howdy Guix! > > We=E2=80=99ve all been talking about it for some time: a REPL server in d= md! > > This can be done by changing zero lines in dmd: > > > > Then you can do: > > deco start dmd-repl > socat unix-connect:/var/run/dmd/repl stdio > > and tinker from there. New ways to cra^W experiment with your system! > > I=E2=80=99m tempted to just commit that. There are shortcomings: (1) the= REPL > server runs in a thread and threads + fork don=E2=80=99t go together well > (although in practice dmd only does fork followed by exec, so it=E2=80=99= s OK), > and (2) for some reason =E2=80=98stop-server-and-clients!=E2=80=99 seems = to leave open > sockets behind it, so if you restart the REPL on the same socket, it > fails with EADDRINUSE. > > Thoughts? Awesome little hack. Thanks! I probably won't use this for PID 1, but I will use something similar for my userspace dmd process. Now, I think this solves the immediate need of being able to live hack dmd, but I'd like to note an additional shortcoming: Thread synchronization isssues. As you know, there's potential for the main dmd thread and a REPL thread to write to the same memory and blow things up. In practice, this would be unlikely, but it shouldn't even be a possibility. To accommodate programs that run in an event loop (though I know dmd doesn't truly have this yet), Mark and I developed the (system repl coop-server) module available in Guile 2.0.11. This "cooperative" REPL server can be integrated into an event loop and guarantee that expressions are only evaluated in the context of a single thread, in this case the main thread. I think that this should be the approach taken in the not-so-long term, which will require modifying dmd itself. I have been using the cooperative REPL server for my game engine since Guile 2.0.11 was released and I've been able to comfortably live hack games without fear. Problem (1) still exists because threads are used for each client, but the real magic happens via coroutines. Not sure if we can reasonably get rid of threads altogether and uses a non-blocking I/O model. Might be worth looking into, but I'm getting off-topic. Now that we can live hack dmd, we'll need some things to help make it pleasant. Most of the time I am tweaking service definitions until they are just right. Currently, that means calling a procedure to unload the version that exists, and then registering a new one. I'd like to reduce that to a single step to tighten the feedback loop. What do you think about adding a 'register-services*' procedure, or maybe a 'define-service' form, that first unregisters the old service before registering the new one? If we can achieve a safe live hacking environment, then we'll have one hell of an init system, I say. Thanks! - Dave