From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Paul Pogonyshev Newsgroups: gmane.emacs.devel Subject: Re: [patch] generator function optimizations Date: Sun, 12 Mar 2017 20:12:23 +0100 Message-ID: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Trace: blaine.gmane.org 1489345955 21660 195.159.176.226 (12 Mar 2017 19:12:35 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 12 Mar 2017 19:12:35 +0000 (UTC) To: Emacs developers , Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Mar 12 20:12:30 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cn8ur-0004gF-3a for ged-emacs-devel@m.gmane.org; Sun, 12 Mar 2017 20:12:29 +0100 Original-Received: from localhost ([::1]:48139 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cn8ut-0000uG-LD for ged-emacs-devel@m.gmane.org; Sun, 12 Mar 2017 15:12:31 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:42041) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cn8un-0000u8-NO for emacs-devel@gnu.org; Sun, 12 Mar 2017 15:12:26 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cn8um-0008Nz-Ni for emacs-devel@gnu.org; Sun, 12 Mar 2017 15:12:25 -0400 Original-Received: from mail-pf0-x22a.google.com ([2607:f8b0:400e:c00::22a]:33937) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cn8um-0008Nm-Hx for emacs-devel@gnu.org; Sun, 12 Mar 2017 15:12:24 -0400 Original-Received: by mail-pf0-x22a.google.com with SMTP id v190so60842007pfb.1 for ; Sun, 12 Mar 2017 12:12:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=VZjWcnmMKV4L6n3mVpUb5ayGoCV7NqmmrbZco0z5V8g=; b=DvRh6Rtobrr+KC6EqzIEN/QgiNVCL52cQ4HcVdEpg8QXL3X+rVZSETWjhCLZlkm6RH SVv6DWrU3Rffx4/SsXK1p8BP8tQaaWs9qm7j8rt1lslD3Vsq6wFediMV9f/DucJMUt+Q bzlFg64sn5aVQAoQhejl0qF/sz8a1aFBr7GRcjIj92+1NhANjoGty1Uj6rIGOo6GpaaW Iv9THGq39O1RYw+7hntK8uR8enqPUP0Yih4rdLax8ZexjMXbkgHWjP4lZoO7hgtwfvCp Z3mNkKMrnsxpnu39K6X5jG+1sxx3tUdUp/pgEV99iZjArlok7O5wgJfIQrAUtCiAX/PB J6Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=VZjWcnmMKV4L6n3mVpUb5ayGoCV7NqmmrbZco0z5V8g=; b=i8yga1IX3PGXv0stsZ+nubBrmK9LWckX93nHCxLs4cwkXlJ97QFr+iBQnNWykpFyMV 3BMhtQQF+1cm0HJnXwC8QebpoicjP3/Fvv1SC4baPyO6PIdpdxiQYBogp3B+y3RVNlLR zymW8Faah9CoLc5hpkoALplQDo9oiy3aE4QIph80uV4SnaD0myJnILdLVVT4CbDDg8uB +WEfgESyhnz3x3lRTV5tC6Elft4xvxrHpZqZDJ2Kd1VLk0R2TDj7Fr9p8Ucf0T5DWMAX U01yieAAa22V65x8X7vYgtD93BRkNzaNrlpF/CKz3Goy00pSg9Qg7XQ0QlhrIDaf8+tF 01OQ== X-Gm-Message-State: AMke39nIEVAQ3M5d5LKya9mTIHNPf/Q3zn4R/rpS+CCD4q2bOl2e7mgf1oVx2I53lTVNT55MpwJgJc7kuA3wYg== X-Received: by 10.84.132.33 with SMTP id 30mr42158357ple.44.1489345943443; Sun, 12 Mar 2017 12:12:23 -0700 (PDT) Original-Received: by 10.100.176.176 with HTTP; Sun, 12 Mar 2017 12:12:23 -0700 (PDT) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400e:c00::22a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:212959 Archived-At: [please keep me on CC, I'm not subscribed to the list] > > In my real code (a very large function used as a co-routine) > > with 60 yields, generator function has over 1000 lambdas. > > [...] > > Hmm... I don't think anyone should really care about the number of > lambdas, right? So what is the actual problem (caused by this large > number of lambdas) that you're trying to improve? Code size? > Run time? Both? Something else? Both. This huge function takes a lot of time to compile, produces lots of bytecode and is fairly slow. Of course it is difficult to compare with non-generator routine: it is so large and complicated I won't try to rewrite it just for speed comparisons, not to mention that it is run by a "master" that can pause at any yield-point to resolve dependencies with other code. However, note that there are 60 points where it can yield and thus lose control. But, there are over 1000 generator states. Meaning that even between this points it switches many generator states, in a sense pointlessly, because intermediate states have predetermined successor (some of them have several, if they are branching points). Lambda calls are not free and it is certainly faster not to call them if you don't have to. Basically, this patch eliminates some intermediate states and makes generator form transformer generate a single lambda where it would previously create several. > Have you tried to measure the impact of your patch on the actual problem? I mentioned 3%. Yes, it is practically negligible, but this is a first small patch and I'm willing to work on improving generators further. > > If this patch is accepted, I can work on some other ideas > > I have. > > Indeed, there's probably a lot of opportunity for improvement. > E.g. cconv.el was designed based on the assumption that there wouldn't > be many different lambdas within the same scope (i.e. we don't try to > share the data part of various related closures). There are also, unfortunately, bugs. I have found and submitted two already (in the normal bug tracker), for one I attached a fix. Paul