From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Cl=c3=a9ment_Pit-Claudel?= Newsgroups: gmane.emacs.devel Subject: Re: How to add pseudo vector types Date: Tue, 20 Jul 2021 16:36:42 -0400 Message-ID: <2524265f-60c7-24f5-b9f4-98447c91acab@gmail.com> References: <83h7gw6pyj.fsf@gnu.org> <45EBF16A-C953-42C7-97D1-3A2BFEF7DD01@gmail.com> <83y2a764oy.fsf@gnu.org> <83v95b60fn.fsf@gnu.org> <00DD5BFE-D14E-449A-9319-E7B725DEBFB3@gmail.com> <83r1fz5xr9.fsf@gnu.org> <1AAB1BCC-362B-4249-B785-4E0530E15C60@gmail.com> <83czri67h0.fsf@gnu.org> <46BBFF88-76C3-4818-8805-5437409BEA93@gmail.com> <83wnpq46uk.fsf@gnu.org> <533BD53B-4E85-4E9E-B46A-346A5BBAD0F5@gmail.com> <258CB68D-1CC1-42C8-BDCD-2A8A8099B783@gmail.com> <1a776770-50b7-93cd-6591-c9a5b3a56eb8@gmail.com> <83lf654dhk.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24371"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Jul 20 22:37:42 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m5wUn-0006DD-Kr for ged-emacs-devel@m.gmane-mx.org; Tue, 20 Jul 2021 22:37:41 +0200 Original-Received: from localhost ([::1]:58954 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m5wUm-0005wH-Lp for ged-emacs-devel@m.gmane-mx.org; Tue, 20 Jul 2021 16:37:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36738) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m5wTu-0005Fe-Og for emacs-devel@gnu.org; Tue, 20 Jul 2021 16:36:46 -0400 Original-Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]:46652) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m5wTt-0001TH-4s; Tue, 20 Jul 2021 16:36:46 -0400 Original-Received: by mail-qk1-x72d.google.com with SMTP id k4so38276qkj.13; Tue, 20 Jul 2021 13:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=/USTGqdheLTUiIXnAKHQfibW3M84euMnEg7lXkuvh8c=; b=RIJwrvECLSCGGRWVmMDmSvwd2o83DIo8bDgYtlZShyZz3aThbNfURLd0sOt6CoLwUP N01+19iZClrp526jdrW4m+2JWfukx9veaMXcR+tCVJnHJxAs8P0Hys5UJzjD8NYUY00N bzbHJ+VLsmFFIajiqJWs+FmjvOA8hb2mSNm+SJhsfRd8H1gU8tFYDSqYYtYfaxzNR4w1 SPJzgl6Dzy7UMvu25RXjd+LFnSBjy8yZRVQk97rUNPCOZ6ICWvDylswzWok+xsaeYA8j 8UkEEs1MQUA5muiiC9WsC5HJZiI52kukcqJEEz6ttB+o8ZT+ZM4x9/OBn6zT4fFYhw2c G+PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=/USTGqdheLTUiIXnAKHQfibW3M84euMnEg7lXkuvh8c=; b=nxUQE7Zx22eJ92M6O+wbjmKB2LQk8/d2LRCPLxIPaG9QH+NOUXJxyHoRrIkCLw5Q7a MeoplYD+mEtBLFr2o9E1IuTws34RvJlof1ELBvRr6Ix4qBKhP86YI98Sj28Io9FqIbmB AKE/h6mYkuA8UlU04XADPSfKTClNGRHQmsTk4Lkpr6DmXDvEEdIrCafsqO1rYrZ6V4o0 fSMKtqOLWCKEu9PCJdsSuSF2odyAReOi7FB7DyYPvrS7cE0SMSrQhv5zgiezKa62Kddi j5ny/ZbnoIg/Kxsy/XR3xAy9EwCZ4U/eU4cyuxBLy9vaKmQkbIvMkJQ9jbqDK5OCQyoz EKEA== X-Gm-Message-State: AOAM532rBJsAqvPn0KJxTYx9LiZT4ObJACZfinvv6u+5t9wwknv+yF7w UUZYqs+fcI9xkDK2iDBmFG2DmGSG+WE= X-Google-Smtp-Source: ABdhPJybd1VdKOetBnJyG8N1Rnio3+kdDNmcRKs7zB9Q5zVUqS8k0EnPHSGJWtSCNUrHTiwziejPhA== X-Received: by 2002:a05:620a:e04:: with SMTP id y4mr31393326qkm.196.1626813403652; Tue, 20 Jul 2021 13:36:43 -0700 (PDT) Original-Received: from [192.168.1.11] (c-24-61-240-80.hsd1.ma.comcast.net. [24.61.240.80]) by smtp.googlemail.com with ESMTPSA id s81sm9936008qka.82.2021.07.20.13.36.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Jul 2021 13:36:43 -0700 (PDT) In-Reply-To: <83lf654dhk.fsf@gnu.org> Content-Language: en-GB Received-SPF: pass client-ip=2607:f8b0:4864:20::72d; envelope-from=cpitclaudel@gmail.com; helo=mail-qk1-x72d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271391 Archived-At: Thanks for the detailed reply. On 7/17/21 3:16 AM, Eli Zaretskii wrote: > When Emacs moves or enlarges/shrinks the gap, that affects the entire > buffer text after the gap, regardless of where the gap is. So it will > affect the TS reader if it reads stuff after the gap. Doesn't enlarging the gap require allocating a new buffer and copying data to it? If so it wouldn't affect the TS reader. Moving is indeed trickier, that's what I referred to as "limited contention". >> You do need to be careful to not read the garbage data from the gap, but otherwise seeing stale or even inconsistent data from the parser thread shouldn't be an issue, since tree-sitter is supposed to be robust to bad parses. > > What would be the purpose of calling the parser if we know in advance > it will fail when it gets to the "garbage" caused by async access to > the buffer text? It won't fail, will it? I thought this was the point of TS, that it would reuse the initial parse on the "good" parts in subsequent parses. > So I don't see how this could be done without some inter-locking. Yes, there probably need to be some care around the gap area. But that's what I was referring to re. "optimistic concurrency". > And what do you want the code which requested parsing do while the > parse thread runs? The requesting code is in the main thread, so if > it just waits, you don't gain anything. You'd have the parser running continuously in the background, every time there is a change. When a piece of code requests a parse it blocks and waits, but presumably for not too long because a very recent previous parse means that the blocking parse is fast. >> In fact, depending on how robust tree-sitter is, you might even be able to do the concurrency-control optimistically (parse everything up to close to the gap, check that the gap hasn't moved into the region that you read, and then resume reading or rollback). > > I don't understand what you suggest here. For starters, the gap could > move (assuming you are still talking about a separate thread that does > the parsing), and what do we do then? Nothing, we start the next parse when this one completes. > I don't understand what could recording the gap solve. The stuff in > the gap is generally garbage, and can easily include invalid multibyte > sequences. I don't think it's a good idea to pass that to TS. Also, > recording the gap changes in the main thread and accessing that > information from a concurrent thread again opens a window for races, > and requires synchronization. This list of gap changes wouldn't be accessed concurrently: you would (message-)pass a copy of it to the parser thread every time it starts a new parse. > Bottom line, I think what you are suggesting is premature > optimization: we don't yet know that we will need this. I thought we knew that a full parse of some files could take over a second; but yes, it will be nice if we can find a synchronous way to avoid having to do a full parse.