From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Aemon Cannon Newsgroups: gmane.emacs.devel Subject: Re: Difficulties byte-compiling very large .el file Date: Tue, 1 Sep 2009 18:41:27 -0400 Message-ID: <794057160909011541u11a42bcg8012a4a2d261a087@mail.gmail.com> References: <794057160908191449t4b23080cjf2b85cb0f8e4f589@mail.gmail.com> <19091.47590.276871.440616@parhasard.net> <19091.53277.453695.888871@parhasard.net> <794057160908250804j2eb49424md39f1c71f403516@mail.gmail.com> Reply-To: aemoncannon@gmail.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=00148539257add3d5b04728bd955 X-Trace: ger.gmane.org 1251845016 18041 80.91.229.12 (1 Sep 2009 22:43:36 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 1 Sep 2009 22:43:36 +0000 (UTC) Cc: emacs-devel@gnu.org, Miles Bader To: Aidan Kehoe Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Sep 02 00:43:29 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1Mic4U-00033W-1U for ged-emacs-devel@m.gmane.org; Wed, 02 Sep 2009 00:43:26 +0200 Original-Received: from localhost ([127.0.0.1]:44520 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Mic4T-0005mG-Aa for ged-emacs-devel@m.gmane.org; Tue, 01 Sep 2009 18:43:25 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Mic2e-00045h-Vr for emacs-devel@gnu.org; Tue, 01 Sep 2009 18:41:33 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Mic2d-00043X-UZ for emacs-devel@gnu.org; Tue, 01 Sep 2009 18:41:32 -0400 Original-Received: from [199.232.76.173] (port=49036 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Mic2d-000434-Mq for emacs-devel@gnu.org; Tue, 01 Sep 2009 18:41:31 -0400 Original-Received: from qw-out-1920.google.com ([74.125.92.144]:29487) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1Mic2b-0000aK-9P; Tue, 01 Sep 2009 18:41:29 -0400 Original-Received: by qw-out-1920.google.com with SMTP id 5so322473qwf.24 for ; Tue, 01 Sep 2009 15:41:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=/pYfnfu3+aY2HFUuOJnX/xLQk6L18g7n6+DTNa6cRjc=; b=lEOA6v6aFWFS8LbJKxM9URNc31uAdAR9AMTFkL4ZBfSFJgwsMmPHlXjYsMtygviPoC ij8DZqJcQnTRQOcfG5TPnxkChigVUit+qPVrHzul1+RGbzyVFtjruZmmzK26JsPlVZN7 s+USmEhDeeDxm8W7ixmg4XC6I03Y1Yq92rfiU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=ODEmnj4KjIoY1Aj2tesCJounr3wp2la2ui8KiLjiwUdsHOKmK2v1lHUjaTrUM3awQE U9MvUEm+tL22LyQ/7f4yKcklSjoqZNxr9whZyv3mV6BPdfHO30foFsA4weCizcVBOsu/ XaW+Q9ZtAF7SVbJx8TZQFRkXP4GdtN+R+w1I0= Original-Received: by 10.229.2.19 with SMTP id 19mr2101950qch.94.1251844888017; Tue, 01 Sep 2009 15:41:28 -0700 (PDT) In-Reply-To: <794057160908250804j2eb49424md39f1c71f403516@mail.gmail.com> X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 2) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:114967 Archived-At: --00148539257add3d5b04728bd955 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable To followup: I'm now working around the original issue by wrapping parts of the generate= d parser in in lambdas thusly: (defmacro a3el-do-in-thunk (&rest body) (let ((thunk-name (gensym "thunk"))) `(progn (setq ,thunk-name (lambda () ,@body)) (funcall ,thunk-name)))) Works like a charm. Thanks to everyone for the help. On Tue, Aug 25, 2009 at 11:04 AM, Aemon Cannon wrote= : > Aidan, > Thanks for pointing out the overflow issue. As you may have guessed, thos= e > constants are both max-value for a signed 32 bit integer, which is what > ANTLR spits out in LL* situations (unbounded values for max-k). I may be > able to specify the max value for elisp in the ANTLR template.... > > RE: My original problem, > I narrowed it down a bit and submitted a report at: > http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=3D4251 > I've since sent a followup with the following code, that also fails: > ;;---------------- > > (defmacro many-forms () > (let ((body '())) > (dotimes (i 20000) > (setq body (cons '(message "more") body))) > `(progn ,@body))) > > (many-forms) > > (if (eq 1 a) > (message "dude") > (message "else")) > > ;;-------------- > > I had assumed that the goto opcode used a 16 bit relative offset, but thi= s > snippet seems to indicate that it's not relative to the conditional itsel= f. > I also noticed in bytecode.c that the goto opcodes all use: > ..... > stack.pc =3D stack.byte_string_start + op; > ..... > where 'op' is a 16 bit int. This indicates that the jump is measured in > relation to the top of the current byte_string, so if the conditional is = in > its own byte_string, it should work: > ;;-------------------- > (defmacro many-forms () > (let ((body '())) > (dotimes (i 20000) > (setq body (cons '(message "more") body))) > `(progn ,@body))) > > (many-forms) > > (defun bar () > (if (eq 1 a) > (message "dude") > (message "else"))) > > ;; ---------------------------- > > Which does work. Wrapping in lambda also works. > > So now I'm thinking that as3_elispParser.el must have a single function > body whose bytecode overflows the 16bit limit. As removing chunks of code > from the top-level seems to alleviate the error (the lines with > (a3el-parser-bitset ..) for instance), I'm guessing that the problematic > byte_string corresponds to the top-level of the file. > > I may be able to wrap this top-level setup code into smaller functions...= . > > > > > > > > > > > > > On Tue, Aug 25, 2009 at 7:50 AM, Aidan Kehoe wrote= : > >> >> Ar an c=FAigi=FA l=E1 is fiche de m=ED L=FAnasa, scr=EDobh Miles Bader: >> >> > Aidan Kehoe writes: >> > > GNU Emacs has (IIRC) 27-bit integers on 32-bit platforms >> > >> > ``The range of values for integers in Emacs Lisp is -268435456 to >> > 268435455 (29 bits; i.e., -2**28 to 2**28 - 1) on most machines.'' >> >> Thanks for the correction. Aemon=92s problem remains, though, independen= t of >> that. Is there a good reason you don=92t error on encountering integers = the >> emacs binary can=92t represent? Portable code needs to be prepared for t= hat >> anyway. >> >> -- >> =BFD=F3nde estar=E1 ahora mi sobrino Yoghurtu Nghe, que tuvo que huir >> precipitadamente de la aldea por culpa de la escasez de rinocerontes? >> > > --00148539257add3d5b04728bd955 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable To followup:

I'm now working around the original issue by wrappi= ng parts of the generated parser in in lambdas thusly:

(defmacro a3e= l-do-in-thunk (&rest body)
=A0 (let ((thunk-name (gensym "thunk= ")))
=A0=A0=A0 `(progn
=A0=A0=A0=A0=A0=A0 (setq ,thunk-name (lambda () ,@body= ))
=A0=A0=A0=A0=A0=A0 (funcall ,thunk-name))))

Works like a charm= .
Thanks to everyone for the help.

On= Tue, Aug 25, 2009 at 11:04 AM, Aemon Cannon <aemoncannon@gmail.com> wrot= e:
Aidan,
Thanks = for pointing out the overflow issue. As you may have guessed, those constan= ts are both max-value for a signed 32 bit integer, which is what ANTLR spit= s out in LL* situations (unbounded values for max-k). I may be able to spec= ify the max value for elisp in the ANTLR template....

RE: My original problem,
I narrowed it down a bit and submitted a re= port at: http://emacsbugs.donarmstrong.com/cgi-bin/bu= greport.cgi?bug=3D4251
I've since sent a followup with the following code, that also fails:;;----------------

(defmacro many-forms ()
=A0 (let ((body '= ()))
=A0=A0=A0 (dotimes (i 20000)
=A0=A0=A0=A0=A0 (setq body (cons &#= 39;(message "more") body)))
=A0=A0=A0 `(progn ,@body)))

(many-forms)

(if (eq 1 a)
=A0= =A0=A0 (message "dude")
=A0 (message "else"))
;;--------------

I had assumed that the goto opcode used a 16 bit r= elative offset, but this snippet seems to indicate that it's not relati= ve to the conditional itself. I also noticed in bytecode.c that the goto op= codes all use:
.....
stack.pc =3D stack.byte_string_start + op;
.....
where '= op' is a 16 bit int. This indicates that the jump is measured in relati= on to the top of the current byte_string, so if the conditional is in its o= wn byte_string, it should work:
;;--------------------
(defmacro many-forms ()
=A0 (let ((body '(= )))
=A0=A0=A0 (dotimes (i 20000)
=A0=A0=A0=A0=A0 (setq body (cons = 9;(message "more") body)))
=A0=A0=A0 `(progn ,@body)))

= (many-forms)

(defun bar ()
=A0 (if (eq 1 a)
=A0=A0=A0=A0=A0 (message "dude&qu= ot;)
=A0=A0=A0 (message "else")))

;; ------------------= ----------

Which does work. Wrapping in lambda also works.

So= now I'm thinking that as3_elispParser.el must have a single function b= ody whose bytecode overflows the 16bit limit. As removing chunks of code fr= om the top-level seems to alleviate the error (the lines with (a3el-parser-= bitset ..) for instance), I'm guessing that the problematic byte_string= corresponds to the top-level of the file.

I may be able to wrap this top-level setup code into smaller functions.= ...











On Tue, Aug 25, 2009 at 7:50 AM, Aidan= Kehoe <kehoea@parhasard.net> wrote:

=A0Ar an c=FAigi=FA l=E1 is fiche de m=ED L=FAnasa, scr=EDobh Miles Bader:<= br>

=A0> Aidan Kehoe <kehoea@parhasard.net> writes:
=A0> > GNU Emacs has (IIRC) 27-bit integers on 32-bit platforms
=A0>
=A0> ``The range of values for integers in Emacs Lisp is -268435456 to =A0> 268435455 (29 bits; i.e., -2**28 to 2**28 - 1) on most machines.= 9;'

Thanks for the correction. Aemon=92s problem remains, though, indepen= dent of
that. Is there a good reason you don=92t error on encountering integers the=
emacs binary can=92t represent? Portable code needs to be prepared for that=
anyway.

--
=BFD=F3nde estar=E1 ahora mi sobrino Yoghurtu Nghe, que tuvo que huir
precipitadamente de la aldea por culpa de la escasez de rinocerontes?


--00148539257add3d5b04728bd955--