From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: What's missing in ELisp that makes people want to use cl-lib? Date: Fri, 17 Nov 2023 04:09:37 +0200 Message-ID: <3341d985-ec82-7702-53c0-9b083c63f18b@gutov.dev> References: <320999cc-6c83-2315-0044-cc0403400af3@gutov.dev> <9ab5d2bd-a648-cae0-a4a7-ae86be10af0f@gutov.dev> <87r0kuqxbf.fsf@gmail.com> <54e115a2-fc36-3056-a030-0dbf32416ddb@gutov.dev> <43f290b0-4119-597b-c89a-0fb4c7db1665@gutov.dev> <1e7fe1ef-af7d-3222-7b9e-b569b3c97ccf@gutov.dev> <22e4cb4d-a8f3-1530-881d-b8c59c5d969b@gutov.dev> <339b58d6-5a44-8393-c2cd-4c935147dde3@gutov.dev> <877cmhrcsf.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17804"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: =?UTF-8?Q?Gerd_M=c3=b6llmann?= , Eli Zaretskii , michael_heerdegen@web.de, emacs-devel@gnu.org To: =?UTF-8?B?Sm/Do28gVMOhdm9yYQ==?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Nov 17 03:10:36 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r3oJX-0004Qi-Eu for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Nov 2023 03:10:36 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r3oIn-0004Ob-A8; Thu, 16 Nov 2023 21:09:49 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r3oIl-0004GT-VQ for emacs-devel@gnu.org; Thu, 16 Nov 2023 21:09:48 -0500 Original-Received: from out5-smtp.messagingengine.com ([66.111.4.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r3oIj-00023V-Ob; Thu, 16 Nov 2023 21:09:47 -0500 Original-Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 51D9A5C024D; Thu, 16 Nov 2023 21:09:42 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 16 Nov 2023 21:09:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1700186982; x=1700273382; bh=zaKyj2yOAc1vSEvWDxC4PUUSsNoe0NjbbA+ FdGpNe+c=; b=DYDk783juloJQ5zNOIM8Bnm8EuQrWlcar7x8SRd4O8xLOEo+kkd S1TbiGp2u2lU5yz/yNgpg6swJ05Ty4gbLH00XYdWdUSAId0xNfIHk7bpmeWJH1zD y+q/GN5w0lV90raq22s8tNICMEeTW3EjUMaPscsH9Gijef905Z+e05bpnl3CA8hR pWNP9ZX7v6UfPsUIxq/ud+j/NNFyDRGjR5CgRwMymqP8BqnQZ2QVqtM4JpepnLSc Xpg2HtW4qvUx92edm/lLfTlqgAznQHWrLQ2tEg8v7lm+X2pSdv/oBlDVeNtk9qNU bFjJMeMtOaa5j2c3bHbpNwj6TJRoTJM9xrg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1700186982; x=1700273382; bh=zaKyj2yOAc1vSEvWDxC4PUUSsNoe0NjbbA+ FdGpNe+c=; b=Ju0HMwMOpfaXyx5b3bS6HB2E5YL4BzckxvpfQEZNyb2RB5VQ9Sf nBUuhkc/cREzo5FZLLoPFgrnhOEPCp7ib6zVezrdzc3aCBJTpjZiuVKK69Bze/8A p+hpUMxebjQsD3QBIuvuDUVLqWA5j3pUE3IDTZTatmfbimbB1PRRZuW61GZGjhzp U3ZAVbJTDbBxV3S7qAsM/OIiQOF1/iCOwfnAq54XiqM55GP4/p0h1WPWRWyQZFYC U+YhxPy7baoGN8gbxt/bzqJpbYEQU9gdo35odgUuDEp+z8gYpiAAmEO0pnfz5hKY iy4cpskxlL4LZ3FH8sh/eaNzCn7oeQj3/7A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudefledggeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug hmihhtrhihsehguhhtohhvrdguvghv X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 16 Nov 2023 21:09:40 -0500 (EST) Content-Language: en-US In-Reply-To: <877cmhrcsf.fsf@gmail.com> Received-SPF: pass client-ip=66.111.4.29; envelope-from=dmitry@gutov.dev; helo=out5-smtp.messagingengine.com X-Spam_score_int: -49 X-Spam_score: -5.0 X-Spam_bar: ----- X-Spam_report: (-5.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-2.193, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:312841 Archived-At: On 16/11/2023 16:30, João Távora wrote: > But it's not only a matter of performance. Take the > 'seq-remove-at-position' generic. Presumably someone thought that > operationq was common enough to merit a separate entry point. In CL, > this very operation can be done (probably faster) with cl-remove using > keyword arguments. Probably. Like I said, it looks more like Clojure's stdlib, but due to the absence of efficient "persistent data structures" the performance profile is different. >>>> If that is the only drawback we can find, it is an unavoidable one. > >> Could you try to explain what I should find in the second example? >> What do you mean by "outlaw"? > > What I meant is that the only way to get the same performance out of > seq.el is to have early #'sequencep checks that bypass the generics > completely, and this makes custom sequences based on sequences > impossible. If we optimize for short lists -- maybe. But for longer lists the dynamic dispatch (if performed once) shouldn't be a problem. >> Does causing worse performance for a short time constitute "outlawing" >> something? > > But you should also find in that "m6sparse" example that the logic is > broken -- not only in terms of performance -- until its author > implements seq-contains-pred. So this is pure "Incompatible Lisp > changes" material (which I also think tanked performance should be btw.) > > But in fact is is already broken by all the "list optimzations" in > seq.el. Optimizations, before yours, that caused whatever the contract > was to be violated. To be able to use `seq-drop-while` for my m6sparse > sequence, I have to add implementations to all those generic entry > points, which is just akward. Were those the 'list' type specializations? >>> And even outlaw more stuff. For example, these generics even have >>> specializers on all mandatory arguments? >>> For example, why does seq-do have FUNCTION as a required argument??? >> >> Because FUNCTION is applied to SEQUENCE? > >> >>> It should be &optional or a &key, with a default value. Which >>> cl-degeneric obviously supports. That way specializations on FUNCTION >>> become impossible, or at least become much harder and there's less >>> risk of tanking user code. Design mistake IMO. >> >> I'm reasonably sure nobody expects function to be anything but a >> straight function (symbol or a lambda), because that's how 'seq-do' is >> used throughout the code. > > Yes, but putting as a required argument in the arglist means users can > specialize for it, and that isn't needed. Not sure anyone does it or > even if that makes the generic even slower. And what would happen if FUNCTION were optional and omitted by the caller? >>>> But that is the price one has to pay for correcting a design mistake. >>>> We're not going to do this every Tuesday. >>> Sure, but that price increases manyfold if we start suggesting >>> seq.el as a replacement for all your sequence processing needs. >> >> We can first fix the mistake and then go on to continue "suggesting it >> as a replacement". Or not. >> >> I don't exactly see it that way, though. And you give an impression of >> arguing for the opposite: toward never using it at all. > > Not at all. Maybe you missed some of my previous messages. I think > seq.el's support for polymorphic sequences, though a little bit flawed > in some respects, is very useful. For example, I've been pondering > using it in eglot.el to try to speed up JSON parsing. If some kind of > seq-plist-get and seq-plist-destructuring-bind can be designed, I might > be able to skip consing much of the useless elements of a gigantic JSON > blob and parse just the parts I need. Of course, not easy, but I think > seq.el is the tool for that. plist like Elisp plist? It's difficult to write a type predicate for. >>> Why working on the :m6-sparse extension, I noticed Emacs becomes >>> noticeably slower, and I suspect that's because while I was editing, >>> all the seq functions I was writing where being probed for >>> applicability, while core things like completion, buffer-switching >>> etc are calling seq-foo generics. >> >> It could be helpful to do some profiling and see where the slowdown >> came from. Could it come exactly from the set operations? > > Not sure. It might have come from tracing seq.el functions, for > example. You might say that it's my fault I was tracing them, but > should I be punished in Emacs usability just for trying to use Emacs to > iteratively develop a seq.el extension? I'm not sure I understand. If you added tracing to 'car', it would also slow Emacs down, wouldn't it? Is that a problem with the design of 'car'? > Anyway tracing basic staples such as seq-do and seq-let gives some > insight as to where they are used and what shape of arguments they are > called with in your normal programming activities. Small lists seem to > appear a lot more often. But expect a massive slowdown while tracing: > even with modest seq.el usage in current core, these generics are called > a lot already. That both sounds like a compliment and a pressing motivation to iron out the most apparent performance pitfalls. >>> I find this performance aspect very bad. Maybe it can be obviated, >>> but only if you drop the '(if (sequence-p seq)' bomb into seq.el >>> somehow. I don't see how we can avoid that one. >> >> I don't quite see the need. And it's unlikely to be of reliable help: >> my observations say that method dispatch simply becomes slower as soon >> as a generic function gets a second implementation. And that >> implementation might arrive from any third-party code. > > Exactly. The entry point generics probably can never be avoided. I > think we agreed that noone -- user or library -- should add > implementations to them. Um, no. I'm saying that seq.el function should be written in a way that causes the dynamic dispatch to happen only once (or perhaps a few times), as opposed to doing it N times (for every element of a sequence) or more. seq-let might suffer for a similar problem. Having overrides for entry points, OTOH, should be fine enough, and whatever added cost the dispatch itself incurs, it still happens once, and can be made up for by the more efficient specialized implementation of the function's logic. > That's why I think not making them defuns was > another design mistake. But other intermediary generics _can_ be > skipped and would bring a performance boost to seq.el. Other intermediate generics, in general, can be skipped by defining different methods at earlier points. > Alright. That all said, here's the latest results, which I gathered > using the attached sequence-benchmarks.el are also attached in > results.txt. > > I gathered each set of timings by running these two things > > src/emacs -Q -nw sequence-benchmarks.el -f emacs-lisp-byte-compile-and-load First of all, when I try to run the above command (on your branch, with your attachments from the last email), it ends with: Compiling file /home/dgutov/examples/cl-lib-vs-seq/sequence-benchmarks.el at Fri Nov 17 03:57:31 2023 sequence-benchmarks.el:73:2: Error: Wrong type argument: listp, more So before I proceed further, could you make sure that you ran the tests with these exact files? Superficially, it looks like just (require 'cl-lib) is missing, but maybe something else inside was also different? I also tried to produce some pretty-printed comparisons, but the above command (even with "fixed" require statement) doesn't produce anything to *Messages* or stdout when ran with --batch. I can evaluate individual joaot/with-benchmark-group forms, and they print -- how'd you call it -- minified Lisp data to messages, but it's not formatted the same way as in your results.txt, nor is it easy to get out of the -nw session because the clipboard is naturally not shared. Is -nw needed? I expected it'd be used to print something to stdout.