From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: chad Newsgroups: gmane.emacs.devel Subject: Re: Request for pointers and advice: displaying several buffers inside a single window Date: Fri, 10 Apr 2020 17:05:00 -0700 Message-ID: References: <83a73swwd7.fsf@gnu.org> <87wo6nxsjz.fsf@localhost> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000854a5705a2f89a58" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="87099"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Adam Porter , Yuan Fu , Eli Zaretskii , Ihor Radchenko , emacs-devel To: Dmitrii Korobeinikov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Apr 11 02:06:00 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jN3ep-000MXl-0s for ged-emacs-devel@m.gmane-mx.org; Sat, 11 Apr 2020 02:05:59 +0200 Original-Received: from localhost ([::1]:40086 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jN3eo-0001TT-4P for ged-emacs-devel@m.gmane-mx.org; Fri, 10 Apr 2020 20:05:58 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:58513) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jN3e9-00011U-Ti for emacs-devel@gnu.org; Fri, 10 Apr 2020 20:05:19 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jN3e7-0008EZ-TV for emacs-devel@gnu.org; Fri, 10 Apr 2020 20:05:17 -0400 Original-Received: from mail-yb1-xb2f.google.com ([2607:f8b0:4864:20::b2f]:35766) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jN3e5-0008DB-0M; Fri, 10 Apr 2020 20:05:13 -0400 Original-Received: by mail-yb1-xb2f.google.com with SMTP id i2so2018146ybk.2; Fri, 10 Apr 2020 17:05:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6g4zSTnLXrVduFG0Nzt1u8AA+LCJnjxXeTFPAFYnS70=; b=pTYzSguODP8hf1XqcXKULWhOd4SrcCEnwH1arbsSFDkF3bkpC4M0CTGxo451pBl4+7 O79ruBchyKp6ee+L1hHT004awCStGadBEofo7nRoI2JgBOfYvFmfJE3JT/ijG+6+Rtq9 aslIdCGJEHoPHEgApXPspqcF1GVcoO4tWjnawaZI3g5L8Uwnv1pplyUjyXBxEdkymv5n 92TF8l2Bft8nwar+dGauX2zehJnxYc8OT1uQU+Nt0yV8QixZGcrnZNN5TfddSflVYVkG EbzfnFv450Pa2bn0WaV72z6evXH6u9xCtx34MMIPz9mTXhlV3DsHrvlHEm2HL6P3/6B9 1Cbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6g4zSTnLXrVduFG0Nzt1u8AA+LCJnjxXeTFPAFYnS70=; b=egymKinT7r4DiwHXVxDJlt5UIIK0jrN7NV0ymXKKQu9Hfw0pGQ+7M3D3fbtRSN4MIz 7atdB0PLetO/sAqN3D4ZGKorcvG/AWse24ZQJSSN4CFILy/XPnep3dWlAgEmaTH5EglO QXCobR5AbuGYtBUqrGbQtyWlur+/OxB8w8qDUG6ZBQBi4qDj2PkJ3zd9cryEWY1r0z9Z xifRffZQU1KgD9D2PeMk/BgHzDQa6h0QzL8U4/12gEbH1nHxm/FDzry6xO18A3OA6R6b bIxPaLPu92RwVQT809RcxenY5EsH+cwsg8PrY7kA8MLvcEvFQca8uL8SXKDxkHr/VcI7 Hnrg== X-Gm-Message-State: AGi0PuZ2YrOPDg5LNduDmMyTCCoVk2b3LK1ZDQE/SKUSdwkS/Df1W8oy U9DQqzQuUxmJKjBVqo2v0qUgg7ilSqFX43prfvA= X-Google-Smtp-Source: APiQypJtM6ra1e5Cws1SPc6WDle8cNKmjGI90WpzzdqVWMPkqHNR5Fzq7g7K7Xkgs9U6ZUcdT97T/0aaa/VeLBY9fcE= X-Received: by 2002:a25:b21e:: with SMTP id i30mr9978059ybj.502.1586563511567; Fri, 10 Apr 2020 17:05:11 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::b2f X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:246811 Archived-At: --000000000000854a5705a2f89a58 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable These are deep potential changes to emacs, but reading it today reminded me of an article I read recently from the people who implement VS Code: https://code.visualstudio.com/blogs/2018/03/23/text-buffer-reimplementation This article describes a series of choices, revisions, and reasoning leading up to VS Code's internal representation of the buffer text. In particular, the way the found and added caching of line starts seemed like something that could help emacs' problems with very long lines, and perhaps also the concerns with parsing large files for highlighting and indenting. ~Chad On Fri, Apr 10, 2020 at 12:10 PM Dmitrii Korobeinikov wrote: > > Following mode plus atomic windows seems doable. But it wouldn=E2=80=99= t be > > transparent, AFAIK follow-mode needs to add special handlers for > > things like isearch and query-replace. OTOH, it would be much more > > difficult to implement in xdisp.c. I wonder if there is an > > intermediate layer that=E2=80=99s higher level than redisplay (in lisp,= or at > > least not in redisplay) but still more or less transparent. > > I have come to the conclusion that extending the underlying data > structure for the storage is unavoidable and, really, the simplest + > most efficient method to achieve transclusion / islands / seamless > mode regions / buffer lenses / whatever, and to avoid having to deal > with a whole lotta bugs, corner cases and synchronization trickery. > > Emacs uses gap buffer for storing text (a c array with some semantics, > basically). What I have in mind is to keep it that way, but to have a > list of references to other buffers along with their intended > positions. > > From what I saw, there's not a unified core interface on top of that c > array. So, would need to modify each place that makes a use of it. > (for a couple of examples, see text_outside_line_unchanged_p in > xdisp.c, looking_at_1 in search.c) > > I tried to eyeball the effected landscape by looking at where GPT > (position of the gap) is referenced. There are 287 occurences of GPT > in the src directory, most of them are in buffer.c, buffer.h, > insdel.c. The distribution is: coding.c(24), syntax.c(10), > changelog(17), xdisp(8), insdel(131), buffer(35), > editfns/fns/w32xfns(11), search(7), marker(6), xml(1), charset(3), > composite(3), decompress(1), pdumper(1), process(1), character(3), > json(1), fileio(11), indent(3), sysdep(2). > > The good news is: I believe xdisp.c wouldn't have to be modified all > that much if the regions are enforced to start on different lines. > > This whole ordeal could easily take a few months. I, unfortunately, am > unlikely to find the time to do it this year. > > PS A possibly useful outcome of this is the possibility to mitigate > extremely long gap motions by breaking up a big buffer into several > smaller ones. > > =D0=BF=D1=82, 10 =D0=B0=D0=BF=D1=80. 2020 =D0=B3. =D0=B2 21:38, Ihor Radc= henko : > > > > > Following mode plus atomic windows seems doable. But it wouldn=E2=80= =99t be > > > transparent, AFAIK follow-mode needs to add special handlers for > > > things like isearch and query-replace. OTOH, it would be much more > > > difficult to implement in xdisp.c. I wonder if there is an > > > intermediate layer that=E2=80=99s higher level than redisplay (in lis= p, or at > > > least not in redisplay) but still more or less transparent. > > > > What about extending the idea of indirect buffers? In indirect buffers, > > the buffer strings are associated with the same memory address storing > > the text (if I understand correctly). Similar thing (theoretically) can > > be done for individual segments of text. Indeed, there will still be a > > question on how the fontification is done, if the overlays should be > > shared, and how the key bindings should behave on such segments, but th= e > > basic functionality of automatically sharing text segments between > > buffers may be a good framework to start considering more complicated > > cases. > > > > Best, > > Ihor > > > > Yuan Fu writes: > > > > > Following mode plus atomic windows seems doable. But it wouldn=E2=80= =99t be > transparent, AFAIK follow-mode needs to add special handlers for things > like isearch and query-replace. OTOH, it would be much more difficult to > implement in xdisp.c. I wonder if there is an intermediate layer that=E2= =80=99s > higher level than redisplay (in lisp, or at least not in redisplay) but > still more or less transparent. > > > > > > Yuan > > > > -- > > Ihor Radchenko, > > PhD, > > Center for Advancing Materials Performance from the Nanoscale (CAMP-nan= o) > > State Key Laboratory for Mechanical Behavior of Materials, Xi'an > Jiaotong University, Xi'an, China > > Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg > > --000000000000854a5705a2f89a58 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
These are deep potential changes to emacs, but reading it = today reminded me of an article I read recently from the people who impleme= nt VS Code:

=
This article describes a series of choices, revisions, and r= easoning leading up to VS Code's internal representation of the buffer = text. In particular, the way the found and added caching of line starts see= med like something that could help emacs' problems with very long lines= , and perhaps also the concerns with parsing large files for highlighting a= nd indenting.

~Chad

On Fri, Apr 10, 2020 at 1= 2:10 PM Dmitrii Korobeinikov <dim1= 212k@gmail.com> wrote:
> Following mode plus atomic windows seems doable. But it = wouldn=E2=80=99t be
> transparent, AFAIK follow-mode needs to add special handlers for
> things like isearch and query-replace. OTOH, it would be much more
> difficult to implement in xdisp.c. I wonder if there is an
> intermediate layer that=E2=80=99s higher level than redisplay (in lisp= , or at
> least not in redisplay) but still more or less transparent.

I have come to the conclusion that extending the underlying data
structure for the storage is unavoidable and, really, the simplest +
most efficient method to achieve transclusion / islands / seamless
mode regions / buffer lenses / whatever, and to avoid having to deal
with a whole lotta bugs, corner cases and synchronization trickery.

Emacs uses gap buffer for storing text (a c array with some semantics,
basically). What I have in mind is to keep it that way, but to have a
list of references to other buffers along with their intended
positions.

>From what I saw, there's not a unified core interface on top of that c<= br> array. So, would need to modify each place that makes a use of it.
(for a couple of examples, see text_outside_line_unchanged_p in
xdisp.c, looking_at_1 in search.c)

I tried to eyeball the effected landscape by looking at where GPT
(position of the gap) is referenced. There are 287 occurences of GPT
in the src directory, most of them are in buffer.c, buffer.h,
insdel.c. The distribution is: coding.c(24), syntax.c(10),
changelog(17), xdisp(8), insdel(131), buffer(35),
editfns/fns/w32xfns(11), search(7), marker(6), xml(1), charset(3),
composite(3), decompress(1), pdumper(1), process(1), character(3),
json(1), fileio(11), indent(3), sysdep(2).

The good news is: I believe xdisp.c wouldn't have to be modified all that much if the regions are enforced to start on different lines.

This whole ordeal could easily take a few months. I, unfortunately, am
unlikely to find the time to do it this year.

PS A possibly useful outcome of this is the possibility to mitigate
extremely long gap motions by breaking up a big buffer into several
smaller ones.

=D0=BF=D1=82, 10 =D0=B0=D0=BF=D1=80. 2020 =D0=B3. =D0=B2 21:38, Ihor Radche= nko <yantar92@gm= ail.com>:
>
> > Following mode plus atomic windows seems doable. But it wouldn=E2= =80=99t be
> > transparent, AFAIK follow-mode needs to add special handlers for<= br> > > things like isearch and query-replace. OTOH, it would be much mor= e
> > difficult to implement in xdisp.c. I wonder if there is an
> > intermediate layer that=E2=80=99s higher level than redisplay (in= lisp, or at
> > least not in redisplay) but still more or less transparent.
>
> What about extending the idea of indirect buffers? In indirect buffers= ,
> the buffer strings are associated with the same memory address storing=
> the text (if I understand correctly). Similar thing (theoretically) ca= n
> be done for individual segments of text. Indeed, there will still be a=
> question on how the fontification is done, if the overlays should be > shared, and how the key bindings should behave on such segments, but t= he
> basic functionality of automatically sharing text segments between
> buffers may be a good framework to start considering more complicated<= br> > cases.
>
> Best,
> Ihor
>
> Yuan Fu <cas= ouri@gmail.com> writes:
>
> > Following mode plus atomic windows seems doable. But it wouldn=E2= =80=99t be transparent, AFAIK follow-mode needs to add special handlers for= things like isearch and query-replace. OTOH, it would be much more difficu= lt to implement in xdisp.c. I wonder if there is an intermediate layer that= =E2=80=99s higher level than redisplay (in lisp, or at least not in redispl= ay) but still more or less transparent.
> >
> > Yuan
>
> --
> Ihor Radchenko,
> PhD,
> Center for Advancing Materials Performance from the Nanoscale (CAMP-na= no)
> State Key Laboratory for Mechanical Behavior of Materials, Xi'an J= iaotong University, Xi'an, China
> Email: yantar9= 2@gmail.com, ihor_radchenko@alumni.sutd.edu.sg

--000000000000854a5705a2f89a58--