From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Arthur Miller Newsgroups: gmane.emacs.devel Subject: Re: Rethinking the design of xwidgets Date: Thu, 15 Oct 2020 14:35:46 +0200 Message-ID: References: <864kmzupp0.fsf@akirakyle.com> <835z7e2ouj.fsf@gnu.org> <86v9fet5sg.fsf@akirakyle.com> <83imbe1040.fsf@gnu.org> <86pn5luak4.fsf@akirakyle.com> <83362g27y6.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4190"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Eli Zaretskii , Akira Kyle , emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Oct 15 14:37:25 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kT2VZ-0000z8-F7 for ged-emacs-devel@m.gmane-mx.org; Thu, 15 Oct 2020 14:37:25 +0200 Original-Received: from localhost ([::1]:34122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kT2VY-0005ul-GV for ged-emacs-devel@m.gmane-mx.org; Thu, 15 Oct 2020 08:37:24 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36132) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kT2U6-0005Hb-EO for emacs-devel@gnu.org; Thu, 15 Oct 2020 08:35:54 -0400 Original-Received: from mail-oln040092075055.outbound.protection.outlook.com ([40.92.75.55]:41958 helo=EUR04-VI1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kT2U3-0000WB-Cs; Thu, 15 Oct 2020 08:35:53 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XsiztMX0MGwykm/YX8eCZFcIs2r4FKdN08yEN1IhXXoLRTGOQ6fI0ETsDpAvXnAPetZajHwo0socehG++XkW8k7V26kvbueU/nHrMK+gGOjGd0+imHOHNQ7l+gmAsIrGWjG/zEWwJzHKKt62oY8AA6rwy0qOb66Lk0jDC8e1fqXHGjHLWAjyN63vDuD+GncYv2uqZBKQW/ZiXF+btOvD/Gx7Yafy88aHFXAbchNORVijJKlYXq/oc74C5U1yYBBeGQGTEFf6q7mMDlWU0F/q5salGNLoybQnyJAjCJDBALv3q05BhOxvrlkGyRYBS2eQy3uJgL427gXylNOsJMIxxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+2mgbsmKxMKgx2Q8On/YZy7u94ybQF13KUU9ia+eGzA=; b=nIvGt5gLm889WLyzLYzkWmy5eNsGa6X+GwbiGeMCwZyMWJ8XIu+Dt/2+6/OWnHb9AgTZwFwxnY33Y4VPGEr1r52YDxMo2kiAa99DzjVXqr0NfBaa2oi0LQNi1KYeio7bHAhvpKHOK3/m+LZUDXc37Tks/H1VSFG/A+shwodG/b6mURiLQ6XM0doNDuLYdZ99ZvgwjiCruxL5nAmPqLyRIJe4fKcwe2PMR/E5Hu1FoQreMaWS/UN0PQZwp8uaxC16wM9aYUJfUAUneKqTpg9kjPtmmnrDO+Pd4Qr+uyhH7KOtLWz+vEw6AysRkXxr39n1ksHrJG3qYGgDo/RmZLv/CA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=live.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+2mgbsmKxMKgx2Q8On/YZy7u94ybQF13KUU9ia+eGzA=; b=XiOv8qhh1QJ8UzuHgL2NgOkxHiSd5xuA4e2KvCY0FJD5IGLz7a9yG0vM7hYCv8Vi8txTNuHOv1NzbWPFH2gbDqmVte8cp2bDXR0rjC5puu3ZIdNLb4gpuJHcMhz90PIXlMACPkHEn/8MUxTJsiqG6udwDByihVWxFQUusSUpIG0PHGPiQ7w4FYGbzQMnAv9hZfP/ooQ/5+HS5mw4uMTC5LomKoRJfLrCgvP5lo+mjlI4Vo3UjTvygUEMQRl3jcg3c4wsUglb1Vg9qknacQ99FE/MCymq6BPoF0poQQOl3nA/hHZsNPXTtcdaeN9kpO3nf+mBk1KNTVvyVu3c9Q8W9g== Original-Received: from DB3EUR04FT022.eop-eur04.prod.protection.outlook.com (2a01:111:e400:7e0c::44) by DB3EUR04HT097.eop-eur04.prod.protection.outlook.com (2a01:111:e400:7e0c::222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.21; Thu, 15 Oct 2020 12:35:48 +0000 Original-Received: from AM6PR06MB4518.eurprd06.prod.outlook.com (2a01:111:e400:7e0c::46) by DB3EUR04FT022.mail.protection.outlook.com (2a01:111:e400:7e0c::285) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.21 via Frontend Transport; Thu, 15 Oct 2020 12:35:48 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:0CA25DF17B5C9BE7EC3AC5767B29862AB598F51C60CE011940F5090DAD6A6E00; UpperCasedChecksum:9121545C359B309F953C06944201C758E9DF26E6C822AD9642406092402EE462; SizeAsReceived:7745; Count:46 Original-Received: from AM6PR06MB4518.eurprd06.prod.outlook.com ([fe80::bcb9:3133:8a66:dc8e]) by AM6PR06MB4518.eurprd06.prod.outlook.com ([fe80::bcb9:3133:8a66:dc8e%6]) with mapi id 15.20.3477.021; Thu, 15 Oct 2020 12:35:48 +0000 In-Reply-To: (Stefan Monnier's message of "Wed, 14 Oct 2020 12:53:17 -0400") X-TMN: [WIBHduH2M8MRelN9eiXzNlg3lqCPhCM0] X-ClientProxiedBy: AM5PR0602CA0011.eurprd06.prod.outlook.com (2603:10a6:203:a3::21) To AM6PR06MB4518.eurprd06.prod.outlook.com (2603:10a6:20b:6b::13) X-Microsoft-Original-Message-ID: <87k0vrvf8d.fsf@live.com> X-MS-Exchange-MessageSentRepresentingType: 1 Original-Received: from pascal.homepc (90.230.29.56) by AM5PR0602CA0011.eurprd06.prod.outlook.com (2603:10a6:203:a3::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.21 via Frontend Transport; Thu, 15 Oct 2020 12:35:47 +0000 X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 46 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: cc40edb8-48d4-4e16-3df5-08d87106d79f X-MS-TrafficTypeDiagnostic: DB3EUR04HT097: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4kLQd4G/fxUDVEccXrwoqwoFAet+NjYXJzQegM2X4JIZgLwVhcLCvkZP67ksqHvr3qwdsfYohih2lGTN7+qIuehBtJ/ks0UMMbREHE7IMtK81QgojCJR7iMRNHf6JOzdL+GueH2iSka0Dc0p/NkIZpK+qKt98dpYSjCsiDQwmWCQDhpZsbIzOkRcLt4O5r75NqeTLK14lvk5wUXB1i+g8Q== X-MS-Exchange-AntiSpam-MessageData: ZO85nj4+HnPpwMYOdmS/opp42FOhKh+QG05OUs35SoDto1Wu17l5IDG5cepM1vmGu5rGeB5Qef5IiBdU4SgHAG5t+PKrUNab8JsRwH4Oc9GKoJcRISIoU4oA0aEfa1TbIafw+W0R7pMSK51YPVgEkA== X-OriginatorOrg: live.com X-MS-Exchange-CrossTenant-Network-Message-Id: cc40edb8-48d4-4e16-3df5-08d87106d79f X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2020 12:35:47.9552 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-AuthSource: DB3EUR04FT022.eop-eur04.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3EUR04HT097 Received-SPF: pass client-ip=40.92.75.55; envelope-from=arthur.miller@live.com; helo=EUR04-VI1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/15 08:35:49 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:257721 Archived-At: Stefan Monnier writes: >>> I don't think that algorithm is less optimal nowadays than it was back >>> then; the really interesting question is how much it saves us when >>> compared to simply redisplaying all those lines. > > Right. The relative performance of different kinds of code has probably > changed significantly, so it's possible that it's faster to just redraw > the whole window than to try and figure out which part to redraw (and > even try and find some scrolling opportunities in there). > Then again, it's also quite possible that those optimizations are still > very much worthwhile. I have never really thought that the graphic performance is bottleneck in Emacs either. I thinking more in terms of code simplification. I have read the comment in disp.c and looked a bit into the display routine at some time just for the fun, and I am impressed and as currently wouldn't dare to touch anything there :-). >> In case where there is a discret gfx card (i.e. Nvidia/AMD) it is >> probably faster to send everything to GPU and ask it to render a >> giant texture and then use it as XWindow pixmap, or something similar >> then to figure out on CPU all the stuff that should not be displayed. > > I think it's much harder than it sounds: most of the work needs access > to data structures that aren't friendly to GPUs, so the work of > providing data to a GPU in an appropriate form would probably not be > much more less in most cases than the drawing itself. As said above, I don't consider myself as well introduced to display code, so consider this mostly as musings and curiousity: are you sure it is that hard? What I think of is that display routine has to get this data from lisp structures to draw to OS window anyway. While it is done, instead of drawing to the screen, stash it into some GPU friendly structure, i.e. "render" it into a VBO or something similar that can be sent to gpu. Maybe this does not need to be pulled-out; I am not sure about this; but where and when is rendering state updated? As I understand all colours and other rendering data for text goes through properties? Couldn't that data be updated at same time in rendering structure when they are updated as lisp property? By adding a hook to propertize or whatever does that and then mark text as dirty? Just wild thoughts, don't take it as concrete suggestion. Maybe I completely don't understand how it works. I am not sure about the details how rendering could be done, that is just a thought about the portion of getting state data for text from lisp. > (talking-about-things-one-does-not-understand-mode 1) Nice markers you have :-) > But indeed, maybe we could split the "drawing" from the glyph matrices > to the glass into a first step that draws into a "pixmap matrix" and > a second step that draws from it onto the glass. This should make it > unnecessary to try and "scroll" (parts of) the display since it should > be "just as fast" to copy from the pixmap matrix as it is to copy from > the current glass's content. If you think in terms of "glass" or "layers" would it be possible to do this: Create a list of textures (to be used as layers) for a displayed buffer (temp buffers and maybe some special buffers that are not displayed does not need it). Render window background into one texture. Render buffer text into another texture, render cursor into third texture and render current line into last texture. Renderer could then composit those into final display. Then for example if you enabled GL, you could just move a textured polygon up or down which should be extremely fast. Zoom in zoom out would be adjusting moving volume (I suppose glOrtho for projection). Rendering highlight for current line, selections etc would be composing the texture with a colored polygon and doing some blending operations possibly in shader to invert rendered text (I think), alternatively render to some other texture and then compose over the old one; I don't know just fast thinking. If you are familiar with Qt, I think that is similar strategy they do in their Qml. They have a scenegraph and they render each widget's state (I think) to a texture and then just compose/display those textures at render time. I am not sure though, it was long time I was reading about how they do stuff. By the way, about egl: nice thing with it is that it enables easy render to texture, and also headless rendering (no X11 required) on gpu, but it can also be used to render to an OS surface directly (a window). Consider Emacs image: emacs inserts images as character properties and display routine is aware of it's dimensions etc. If one could insert image as a placeholder (say some imaginary format 'egl-image') one could then render and image as a texture via egl with some lisp commands; some graphics api not yet existing in elisp that would render via opengl into egl texture, and then the image buffer, a pixel array as floats, could be copied back to Emacs and displayed in image placeholder. I don't know how to hook it up into image routines in Emacs, was looking at it some time ago, but I didn't found it back then, and didn't have time to go back. That is less efficient since it copies data back and forth between cpu/gpu, but it would still be much more efficient then going through external process and file system as Emacs does now. Maybe if gl context is enable in entire frame, it would be possible to enable directly render opengl content to the Emacs window portion that image placeholder is. Some translation of pixel coordinate and other synchronisation would be needed, but I don't think it is impossible or extremenly hard to do from the opengl point of view. Maybe I don't understand all the pecularities and intricacies, I myself, certainly don't know how to hook up all this into Emacs nor if it was even possible to pull of; I am just theorycrafting atm :-). > > I think the performance of the redisplay is by and large poorly > understood. While there are known cases where people experience "slow > redisplay" it's usually very unclear what that means concretely. > In many cases this can be completely unrelated to the actual redisplay > code (e.g. the time is spent running some expensive code off of > `post-command-hook` or font-lock or younameit). You are probably correct about that one.