From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: JD Smith Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter navigation time grows as sqrt(line-number) Date: Sat, 19 Aug 2023 20:18:11 -0400 Message-ID: References: <3E82D409-6903-4679-9031-939CA35791FF@gmail.com> <32507689-3b2c-ccbf-dd14-e7bf0bed1ac7@gutov.dev> <6db52945-5459-197c-405d-153ff395a824@gutov.dev> <1F7C956D-6D22-4CC1-8656-6E2A4D07D5FB@gmail.com> <69D18963-D94F-4792-9FF1-159897A99E50@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="6919"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Dmitry Gutov , emacs-devel@gnu.org To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Aug 20 02:19:31 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qXWAE-0001aA-CO for ged-emacs-devel@m.gmane-mx.org; Sun, 20 Aug 2023 02:19:30 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXW9F-0003lQ-F3; Sat, 19 Aug 2023 20:18:29 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXW9D-0003jQ-Hd for emacs-devel@gnu.org; Sat, 19 Aug 2023 20:18:27 -0400 Original-Received: from mail-yw1-x1136.google.com ([2607:f8b0:4864:20::1136]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXW9A-0006nT-S0 for emacs-devel@gnu.org; Sat, 19 Aug 2023 20:18:27 -0400 Original-Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-58fb8963617so8397707b3.3 for ; Sat, 19 Aug 2023 17:18:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692490703; x=1693095503; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hYh2I4rFUYTLY5gp5Y1V0aX1oM9Y3NcDnui7GPwXUzo=; b=R5oiBf7RNfgwhkmmsEC939g1mlzdXvM9onqsYkciHthvjg/oMuy1HxhEh7j9hGpA4J jQiYXqeY2cbMz5+p6ry3jfPjE+de/GPSikDTB8a/uWZGrit+HuJV0TNys1J2Sld1VvUR vCV77lYvlMRIDi/oAWOxnRrlHyRAyFP6Fr+Km/LT3geRKjRacygdjvrTpbTFabxjbfmT 5aQHiQUTUO52OCNvwdGs7zZ9syZvJmGUERFAdOBH5OfJYSbAPdaV+73APySehj1tychi Tu4rEoCEikoRQdybWNRrukp4F9Kz9BdRh7nh4aiMWg6l8viYjV+97jgEWrl3AtC5yrHi R1aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692490703; x=1693095503; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hYh2I4rFUYTLY5gp5Y1V0aX1oM9Y3NcDnui7GPwXUzo=; b=Sa+eAxBy7A/WhjFdE5wqrwpwN7Ohr6vJBgpPj6q66jdgYEJdJDGnEq1qve320bYxiG pGlZqtQ5UizfyFCjpZMTKAxEkiIcPpKjX2ILuCuyQr4x7PBThZ0QROym3FmY6fQzSNIT Cxx/53aU75XUVZlRtXLPSUpZJNR0yTQB9M2VxBxS4UdVBtRlUr1Z8MQ0Bd+8nbPqsWdN 7R7HD2JOIj1jqm4QCA4SE+QffwjmjS193Er8TqQLTuJTaxZk6ADYFyPDWTYJ3PCcXrbj x/wvdnNDNm5IaV731CNgSfa3KxPM8xl0+AYlaS5odsSUq3/mYRJgt3W7bJKkFs9Xymy6 uajw== X-Gm-Message-State: AOJu0YwJRTd1nR/et1ACw+fvS2iKVhXmKgSb+aMf42H/IGIHUfeDSCaW n/+hIJeT/JtTBsjB5Pf1y2ViGXWeQT/BAA== X-Google-Smtp-Source: AGHT+IG/A+9AG6YMd7+aJ3PzNId3o9xZMHYvPv2zXHqDKpB+XfLtHfeCfMtnMWBSa9NheKHRVALXtA== X-Received: by 2002:a0d:eb94:0:b0:589:e66e:abe4 with SMTP id u142-20020a0deb94000000b00589e66eabe4mr3700524ywe.11.1692490703340; Sat, 19 Aug 2023 17:18:23 -0700 (PDT) Original-Received: from smtpclient.apple ([206.121.37.170]) by smtp.gmail.com with ESMTPSA id j195-20020a8192cc000000b00576c727498dsm1407679ywg.92.2023.08.19.17.18.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 19 Aug 2023 17:18:22 -0700 (PDT) In-Reply-To: <69D18963-D94F-4792-9FF1-159897A99E50@gmail.com> X-Mailer: Apple Mail (2.3731.700.6) Received-SPF: pass client-ip=2607:f8b0:4864:20::1136; envelope-from=jdtsmith@gmail.com; helo=mail-yw1-x1136.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:308942 Archived-At: Great, thanks. I tried this patch out, and there is indeed about 10x of = improvement. Check the bottom of the gist. That said, node_parent = remains 10x faster yet (at worst, in a long file), so maybe there=E2=80=99= s room for further improvement? May be worth looking at how others are = doing it, e.g. the python API. =20 Alternatively, have we ruled the seemingly simplest node_parent out = prematurely? If the issue is a node being its own parent in some odd = trees, wouldn=E2=80=99t a simple check suffice to guard against this = rare possibility? > On Aug 19, 2023, at 6:16 PM, Yuan Fu wrote: >=20 >=20 >=20 >> On Aug 19, 2023, at 7:24 AM, JD Smith wrote: >>=20 >> Thanks for your patch, Dmitry. I had a chance to test it this = morning (the new, non-crashing version). I made a new NS build, with = and without the patch. The results are really striking (scroll to = bottom): >>=20 >> https://gist.github.com/jdtsmith/7fa6263a13559d587abb51827e6ae472 >>=20 >> Summary: >>=20 >> - Applying the same test above on _axes.py reproduces the earlier = emacs-mac/29 results: the time to navigate from the node at line = beginning to root starts at under 10=C2=B5s, but rises as sqrt(N) by = ~100x, reaching over 3000=C2=B5s. >>=20 >> - With Dimitry=E2=80=99s patch, it performs much, much better, = starting off with similar timing at early positions in the file, but = rising no higher than 50=C2=B5s, scaling much shallower than sqrt(N). >>=20 >> I should emphasize this is a new fast machine; I fully expect my old = laptop would be much slower (10x ?) than 3ms in files this large, which = makes using parent navigation for things like font-lock problematic. >>=20 >> The patched version results also make a lot more sense in terms of = their similar logarithmic growth as node-at-point, since the method of = search for a node at point and for its parent is, as Yuan points at, = quite similar. >=20 > I inspected the descending algorithm, and there=E2=80=99s indeed an = oversight made by me. Here=E2=80=99s a patch that should fix it. I = tested it briefly and it does speeds things up greatly. Thanks for = investigating this, JD! >=20 > I think the patch is relatively safe, so maybe we can push it to = emacs-29 instead of master. >=20 > Yuan >=20 >