From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Newsgroups: gmane.emacs.bugs Subject: bug#47711: bug#48841: bug#47711: [PATCH VERSION 2] Add new `completion-filter-completions` API and deferred highlighting Date: Mon, 16 Aug 2021 15:20:36 +0100 Message-ID: <8735r9ii8b.fsf@gmail.com> References: <9f59f87c-2489-aaa0-5b3f-0e911b7ffa6c@daniel-mendler.de> <8a36e61a-1c5b-bf3b-a454-077348589c4f@yandex.ru> <87y29471ov.fsf@gmail.com> <837dgob6yo.fsf@gnu.org> <87wnootec0.fsf@gnus.org> <5d70b0ad-3838-ddb8-d341-9a5531d9da73@yandex.ru> <0cbf224b-b382-8a02-3f06-9f6d7e5e9741@yandex.ru> <87a6lihv7b.fsf@gmail.com> <63e03b18-db81-3b11-c692-0af9df20c506@daniel-mendler.de> <83k0kl8sxm.fsf@gnu.org> <83bl5x8r02.fsf@gnu.org> <77e98cdf-427d-ec7a-7a49-04dcd65d6a94@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39485"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Daniel Mendler , monnier@iro.umontreal.ca, 48841@debbugs.gnu.org, larsi@gnus.org, 47711@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Aug 16 16:21:10 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mFdUE-000A50-1R for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 16 Aug 2021 16:21:10 +0200 Original-Received: from localhost ([::1]:52464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mFdUC-0003MT-Kl for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 16 Aug 2021 10:21:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:32768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mFdU5-0003KP-VL for bug-gnu-emacs@gnu.org; Mon, 16 Aug 2021 10:21:01 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39152) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mFdU5-0005Q2-Oa for bug-gnu-emacs@gnu.org; Mon, 16 Aug 2021 10:21:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mFdU5-000577-J0 for bug-gnu-emacs@gnu.org; Mon, 16 Aug 2021 10:21:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: =?UTF-8?Q?Jo=C3=A3o_?= =?UTF-8?Q?T=C3=A1vora?= Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 16 Aug 2021 14:21:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47711 X-GNU-PR-Package: emacs Original-Received: via spool by 47711-submit@debbugs.gnu.org id=B47711.162912365519632 (code B ref 47711); Mon, 16 Aug 2021 14:21:01 +0000 Original-Received: (at 47711) by debbugs.gnu.org; 16 Aug 2021 14:20:55 +0000 Original-Received: from localhost ([127.0.0.1]:50696 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mFdTv-00056W-My for submit@debbugs.gnu.org; Mon, 16 Aug 2021 10:20:55 -0400 Original-Received: from mail-wm1-f43.google.com ([209.85.128.43]:33449) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mFdTr-00056D-4m; Mon, 16 Aug 2021 10:20:51 -0400 Original-Received: by mail-wm1-f43.google.com with SMTP id a201-20020a1c7fd2000000b002e6d33447f9so86640wmd.0; Mon, 16 Aug 2021 07:20:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=Q3zK+7/1DJLNkrgytux3CwuqtrXlG4P/nZ9MLv0C3rI=; b=q4ZJFDL5eGfX2CRUdiokAWiTFfpZX6sdfZG1Bmol2X32LXvsGO4PgHunFx9OOCHMHS hhubbq4S3qwKaRRg/VyUZt9Ne+B2SHSoye5ikVnpum0MYQ0tQ0Rh5/GKPyVnuupSOb6e N1o/xtqh48zuYcZ7mNcaSDJnIR1wXIKl73LW5XqUkITVc83o1qg9g8Pn+F1roNEriOjn NAaeQZGuTTRn/TkZWDUHWh2HWH+FFsIITfBezL9tR2HdvmDasEKjYPBZTUOtsE8cT2FQ inb/y1q0XRpIY++lQiMbWL/yfnKVLvzTX/jQzP4RTRU9lI59T7XJy2pnLa3G6EFFSzfK Cpfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=Q3zK+7/1DJLNkrgytux3CwuqtrXlG4P/nZ9MLv0C3rI=; b=orBlOMDWSYSokVJeNicx1REH0HCxNGdkDIKwnjgCsAO80QkVLWDg+jzGQIGJduIhcP Wy2PXzj6DX3usChzpEIOfhic4SrAa5m0yD36s2RaPQFXnv1PQE43r5blMd0eTmthw3Z3 XTEQtcIeMoJ8aG9xoGY0a9iMY4kZiI5+CYBa0XnkL1cNOfABRLrfXgK0BfeeaHqdR8W/ W8FwJoXE9QZSVk7OmagF5BfetRBGA3hJvdHkHN3GSl8WYw0VHyTeI7aYwy61WH93MwA3 jCl8gSDKXP1rJDb4OGOjKKLhBvm3wSjA87cXlwhRx+XVmlheySzbZM1G3X7QYAtEh74F gGnw== X-Gm-Message-State: AOAM5326udjmBnS/FhmgIqVlf2bBHnFksidk0ipJAv/WMWJBkEqSyloS lJ0MJuGRwlQ5c6v0QgiZkLRpVrJ0VgA= X-Google-Smtp-Source: ABdhPJw5NZ8vcICaa27Insw1jRBGjjyzCTgnp2FYPaKhpFVm7go25OInr9NoQswLu0r8eb97wJ6smQ== X-Received: by 2002:a7b:c005:: with SMTP id c5mr15424096wmb.59.1629123640952; Mon, 16 Aug 2021 07:20:40 -0700 (PDT) Original-Received: from krug (a94-133-27-132.cpe.netcabo.pt. [94.133.27.132]) by smtp.gmail.com with ESMTPSA id n3sm11414236wmi.0.2021.08.16.07.20.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Aug 2021 07:20:40 -0700 (PDT) In-Reply-To: <77e98cdf-427d-ec7a-7a49-04dcd65d6a94@yandex.ru> (Dmitry Gutov's message of "Mon, 16 Aug 2021 17:00:09 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:212033 Archived-At: Dmitry Gutov writes: > On 16.08.2021 16:21, Eli Zaretskii wrote: > >>>> FWIW, I also don't understand how adding properties could cause a >>>> memory leak. When a string is GCed, its properties get GCed as well, >>>> all of them. Am I missing something? >>> >>> If you add string properties to all symbol names this memory will stay >>> alive for much longer than necessary. >> That's a very extreme example, something that I wouldn't expect a >> Lisp >> program to do, without removing the properties shortly thereafter. > > And that *will* happen with Joao's approach, as soon as some package > implements a Lisp completion backend in a certain (legal) fashion. There is no leak, not in the strong or weak sense. There is a constant usage memory footprint proportional to the size of obarray, yes, but the factor isn't huge. From the top of my head, I think it's about two conses and a fp number for each sym. does anyone know how much that is or can teach me how to measure? Thanks. Anyway the current situation is constant copies of strings that put pressure on the GC, as the benchmarks show. Anyhoo, in the interest of placating this string property bogeyman, I've prepared a version of my patch that is based on weak-keyed hash tables. Slightly slower, but not much. Here are the usual benchmarks: (defmacro heyhey () `(progn ,@(cl-loop repeat 300000 collect `(defun ,(intern (symbol-name (gensym "heyhey")))= () 42)))) (setq completion-styles '(flex)) (heyhey) (when nil ;; Press C-u C-x C-e C-m quickly to produce a sample (benchmark-run (completing-read "" obarray)) =20=20=20=20 ;; my patch with private string properties (3.596972924 4 1.125298095999998) (3.584963294 4 1.1266740010000014) (3.4622216089999998 4 1.0924070069999985) (3.565632813 4 1.1066678320000012) (3.456130054 4 1.099950519) (3.49538751 4 1.1085273779999998) (3.4882531059999997 4 1.0928655200000001) (3.526581152 4 1.0909935229999999) (3.710919876 4 1.13883417) (3.576690379 4 1.09514685) =20=20=20=20 ;; my patch with an no string properties (global weak hts) ;; Probably the extra gc sweeps are paranoid cleaning up of the ;; hash tables. (3.981110008 7 1.6466288340000013) (3.819598429 7 1.5200578379999996) (3.823931386 7 1.5175787589999992) (3.9161236720000003 7 1.6184865899999998) (3.835148066 7 1.5686207249999988) (3.791906221 7 1.5481051090000015) (3.798378812 7 1.5164137029999996) (4.049880173 7 1.7670989089999996) (3.716469474 6 1.3442434509999996) (3.422806969 6 1.3272816180000002) =20=20=20=20 ;; current master (5.405534502 12 2.8778620629999994) (5.038353216999999 12 2.553688440000002) (4.94358915 12 2.4917956500000003) (4.950710861 12 2.4638737510000013) (5.0242796929999995 12 2.5226992029999984) (5.020964648 12 2.495171900999999) (4.914088866 12 2.4218276250000024) (5.003779622 12 2.502260272000001) (4.969347707 12 2.4814790469999988) (5.376038238 11 2.565757513000001) =20=20=20=20 ;; didn't bother with daniel's patch as I've already shown it to be ;; about 10% slower than current master. =20=20=20=20=20 ) The patch lives in the branch scratch/icomplete-lazy-highlight-no-string-props. It's a bit more complicated to follow, but not much if you understand hash tables. The interface to icomplete.el is completely unchanged. All in all, still a very good improvement over the current situation, and I think I can make it faster. (Though really do consider Eli's arguments the fastest approach) >> And even that isn't a leak. >> Note that we already put all kind of properties (although not text >> properties) on symbols. > > Those do not, generally, change over time. Neither does this one! At least in size, which is the thing that matters. So in terms of "negative" consequences it's exactly equivalent. Read the patch it will be obvious, I think. Jo=C3=A3o