From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: Gap buffer problem? Date: Wed, 11 Dec 2024 16:33:18 +0100 Message-ID: References: <86wmg7bso1.fsf@gnu.org> <87cyhzmzbp.fsf@telefonica.net> <87bjximwnj.fsf@protonmail.com> <87ttbal9qx.fsf@protonmail.com> <877c86l1aj.fsf@protonmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32227"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: "Pip Cet via \"Emacs development discussions.\"" , =?utf-8?Q?=C3=93scar?= Fuentes To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Dec 11 16:34:04 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tLOix-0008E9-KU for ged-emacs-devel@m.gmane-mx.org; Wed, 11 Dec 2024 16:34:03 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tLOiL-00066P-U0; Wed, 11 Dec 2024 10:33:26 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tLOiK-00064j-4P for emacs-devel@gnu.org; Wed, 11 Dec 2024 10:33:24 -0500 Original-Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tLOiH-0000p0-LM for emacs-devel@gnu.org; Wed, 11 Dec 2024 10:33:23 -0500 Original-Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-434a766b475so65846895e9.1 for ; Wed, 11 Dec 2024 07:33:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733931200; x=1734536000; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eZQrKLvOQpIXH3PGdNvN0Ud7keoq6maIhnmvO04/ChI=; b=b/g495tBLYpQ/3a/ZhnDacO+gxp1hTyqCaiEK1PG398Fs4YbAdCRDm8ocNltuLg5Uu 26qLj/qQdwuUn8jf1ckQLtcjJdXkK3bkK7lDX+rIwGOtioPqMImzFuRVuBJZ9W5jRFEu CPXVeH+LvE+CCBJKAYu065YvEcnwvq6E0YCwKZ9INgKz2r8aUQB9wpngtaBNUw3hME1P ojY29px6U7WZwwguUW0QoBuYLE8E0v3qlgFDXo37PLnNEwImWwGP9gvntQxle9Lybj9o P8oMc47FH+HrCuLv5DKEwKoS1PGBZ1tsOA7APkWZN0gUA34iwHVGMOzZP0BWCKHZChHq 92dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733931200; x=1734536000; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=eZQrKLvOQpIXH3PGdNvN0Ud7keoq6maIhnmvO04/ChI=; b=rFbDIGLaqwG/dyqCLpq55G4o2tpFrsoA3NOB/O3WgnzJZrSYE+NCkzim02IBXHzUxD 5LJLOnZhVcwiJRqiSpQMNZ6bFFgICB+IzBSY3I9IrbJrc+97yZUxg/WXe0L2FOEvNCLF tElS7/QA6v3uR1g5P1P+EELYJ/Gp+AaltMS0+au/iT220byv0BF/wxJj+7T80+z7XVdz QgtbVeiIPGLVI5+Gs7xOG5fgm1Z0P03oB2oyQKLab7hGctCfU5qAEIoB1wK/301mfFzN zX1zCmvXMWzbHIDVyQ3Ago5A8n9Lt7AFuHBYTK6liJ3PAz4CCEabzOg7aPt/yry7vVkg cytw== X-Gm-Message-State: AOJu0YyF37xqbFL8jqn1cLkwC3OqEE71GHRBOB6bKGc8oDesL8uOejS6 2tsCDZLyYrcprchA0ZXVz2c6zLBealtEVWh0qCUVPiq8uVpLIWvR X-Gm-Gg: ASbGnctrf/XN3dY/VU8lHoGKmezpdubyEJ34r9vBT7sCKQU5mjydWnkhwQOix14T3SB PtQ++NgsX2KsZ9ruqpXWyb11bPv9XzuLSQ+62Ahtu+xf7zeDCLvjb62+oOksngxRGEhg2oiO6ZJ qOzpyq+59t5sgf4nmCWMCKuNJNPibo10XmQoWgMseKPsWuf/QCX+Oaok2er+lv1eR0e9YdZK6Ek 2zKI560pqZazBoQWO8vfczi1+ZBaxlhmM+zIB2uKvj0Ke6q0NGjcKpMKhvv43/ABhvzFz301v1q BMc2nTwCvhiQ2a7YxTtGwD/CNHNwon9gyoPMlcuQGYGdZVAkdJAAs2SLZGaT/ms= X-Google-Smtp-Source: AGHT+IFolzk/gxEn9DLy5OP6ewGkoi/zs3yUJHgNi2s3IHNeZw7M/rm6ZOt49pRkkTQ9gD1iD5DVig== X-Received: by 2002:a05:600c:468a:b0:434:f804:a9b0 with SMTP id 5b1f17b1804b1-4361c4233e8mr24214305e9.29.1733931199681; Wed, 11 Dec 2024 07:33:19 -0800 (PST) Original-Received: from pro2 (p200300e0b709ba00c5d5596d9959c8a7.dip0.t-ipconnect.de. [2003:e0:b709:ba00:c5d5:596d:9959:c8a7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4361b7a46c1sm31372595e9.33.2024.12.11.07.33.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Dec 2024 07:33:19 -0800 (PST) In-Reply-To: <877c86l1aj.fsf@protonmail.com> (Pip Cet's message of "Wed, 11 Dec 2024 14:53:22 +0000") Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=gerd.moellmann@gmail.com; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:326355 Archived-At: Pip Cet writes: > Gerd M=C3=B6llmann writes: > >> Pip Cet writes: >> >>> Gerd M=C3=B6llmann writes: >>> >>>> Pip Cet writes: >>>>> if we ever replace the gap buffer code, we should make sure its >>>>> replacement actually handles buffer text and text properties/intervals >>>>> in an integrated manner, rather than storing just buffer text). >>>>> >>>>> Pip >>>> >>>> And if I may add a wish to the future author: Make whatever you use >>>> persistent data structures, so that one could think of letting redispl= ay >>>> run concurrently. Really! :-) >>> >>> You won't be surprised to hear I've been playing with some code, >> >> Indeed, I was just thinking to myself "I knew it" :-). >> Two thumbs up! >> >>> so could I ask you to expand on this point? What precisely does >>> redisplay require? Full snapshotting or would it be sufficient to have >>> fine-grained locking? >> >> Maybe it's helpful when I tell something about the background. Some time >> last year I asked myself if I could make Emacs more than one of my >> plenty of CPU cores without solving the multi-threaded Elisp problem. >> And the idea was that I could do that, possibly, by letting redisplay >> happen in another thread. > > This may be a very stupid idea, but why not use a separate process? Not stupid at all. I thought about something similar in a different context, namely if one could decouple the GUI part of Emacs from the rest. Something like that has been done by Eberhard Mattes for OS/2 with the old redisplay. He had to do that because the whole Presentation Manager (GUI) in OS/2 would block, for all process, when an application did not timely handle events and return to the PM. Something like that. Eberhard's OS/2 Emacs had one process doing the GUI stuff, and one for the rest. Both communicated with each other using a defined message protocol. It worked. Don't remember what he used for process communication, pipes or something else. I got stuck with this idea because everything seemed to depend on everything else nowadays. Redisplay needs to execute Lisp, Font backends I think, not sure. Some GUIs call redisplay (nsterm). And then I imagined the licensing issues, and dropped the idea. Although - NS could really need something done, IMO, which was the reason I thought about that in the first place. NS is not working for me at least. I always wonder why nobody else has the same freezing problems that I have. I think the same dependency problems also creep up with to concurrent redisplay, don't know. Values of variables, faces, jit-lock, and so on. I think it would be "easier" to handle if one has everything in one process. But in principle both could be done. An actor model. > fork() is fast on GNU/Linux, and I suspect on macOS too, and the > redisplay child would receive a consistent snapshot of the data to > inspect and/or modify while coming up with the redisplay instructions, > which it would then send back via a pipe or shared memory to be executed > in the main process. > > I suggested doing something similar for GC (the GC child would perform a > full GC and send back the Lisp_Objects which are definitely unreachable > via a pipe. No, I never figured out how to make that work for weak hash > tables which may resurrect references, I just made all hash tables > strong...), and in that case the pipe seemed sufficient for the amount > of data that was transferred, but I'm not sure how compact (or > otherwise) serialized redisplay "instructions" would be. > > One issue I see is that fork() does a lot of housekeeping work in > addition to marking the child's memory as a COW copy of the parent's > memory at the time of the fork(). ISTR you can split that process on > GNU/Linux (probably not Android), so you'd already have a prepared > thread/LWP which wouldn't need to "start up" when you un-share the > memory, but I can't find the relevant manpage right now. However, I have > no real idea just how bad the fork() latency would be (as you point out, > most people have more CPU cores than they can use, so I don't consider > the approximate doubling of CPU usage a problem). > > This would deal very nicely with fontification code attempting to modify > data it shouldn't, by ignoring such modifications. It would also deal > with catastrophic failure in the redisplay code, as it's insulated in a > separate process and we could just print a nice message in the main > process rather than crashing all of Emacs. > > I'm emphatically not suggesting letting the redisplay child actually > communicate with the X server or equivalent. That would be much more > difficult. > > In fact, I think a good way to test this approach would be to use the > tty code, since there's already a standard serialization of redisplay > instructions for tty displays: VT100 escape sequences. > >> I later realized while thinking about the details, that this undertaking >> is an order of magnitude too large for me. Everything taking more than a >> few months is. And, in addition, I wouldn't want to do data structures >> in C anyway. > > I think the VT100 case could be done as a weekend project (those always > end up taking several weeks for me...), but I'm not sure it's worth it > as VT100 redisplay isn't the common use case, and the performance > problems are more visible on GUI terminals. Yes. In a way, it's already the case that the GUI part of Emacs that I described above for OS/2, is the terminal emulator, and the protocol is VT100. > And, like pretty much all Emacs ideas, this depends on having a better > GC. > > (However, I've just experimented with an 8 GB process forking, and it's > much slower than I'd hoped for - about 70 ms. I wouldn't be surprised > if most of that cost is setting up page tables for the ridiculously > small 4KB page size x86 uses, so it may work a lot better for AArch64 > systems such as yours). > > Pip