Tangling is broken in git master

Tangling is broken in git master

[Bernt Hansen]

Hi Bastien,

Tangling doesn't work for me in git master anymore.  Git bisect
identifies the following commit as introducing the problem

--8<---------------cut here---------------start------------->8---
ba16c3c6f50738b070769040586945436439be76 is the first bad commit
commit ba16c3c6f50738b070769040586945436439be76
Author: Bastien Guerry <bzg@altern.org>
Date:   Sat Aug 11 10:43:56 2012 +0200

    Don't use `org-labels'
    
    * org-compat.el (org-labels): Remove.
    
    * org-bibtex.el (org-bibtex-headline): Don't use `org-labels'.
    
    * ob.el (org-babel-sha1-hash, org-babel-noweb-p): Ditto.

:040000 040000 bf962e4c7c93de4f2b5f71bf4fc1520ec0723a0e 43ecd68f89e1e856a4e65d84eff3cf6b84872494 M	lisp
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (void-function letrec)
  (letrec ((intersect ...)) (funcall intersect (case context ... ... ...) (split-string ...)))
  org-babel-noweb-p(((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:padline . "") (:noweb . "no") (:tangle . "yes") (:exports . "none") (:results . "replace") (:colnames . "no") (:hlines . "yes") (:padnewline . "yes") (:session . "none")) :tangle)
  (if (org-babel-noweb-p params :tangle) (org-babel-expand-noweb-references info) (nth 1 info))
  ((lambda (body) (if ... body ...)) (if (org-babel-noweb-p params :tangle) (org-babel-expand-noweb-references info) (nth 1 info)))
  ((lambda (body) (with-temp-buffer ... ... ...)) ((lambda ... ...) (if ... ... ...)))
  (let* ((info ...) (params ...) (link ...) (source-name ...) (expand-cmd ...) (assignments-cmd ...) (body ...) (comment ...) by-lang) (setq by-lang (cdr ...)) (setq blocks (delq ... blocks)) (setq blocks (cons ... blocks)))
  (if (and language (not ...)) nil (let* (... ... ... ... ... ... ... ... by-lang) (setq by-lang ...) (setq blocks ...) (setq blocks ...)))
  (unless (and language (not ...)) (let* (... ... ... ... ... ... ... ... by-lang) (setq by-lang ...) (setq blocks ...) (setq blocks ...)))
  (if (string= (cdr ...) "no") nil (unless (and language ...) (let* ... ... ... ...)))
  (unless (string= (cdr ...) "no") (unless (and language ...) (let* ... ... ... ...)))
  (let* ((start-line ...) (file ...) (info ...) (src-lang ...)) (unless (string= ... "no") (unless ... ...)))
  (let ((full-block ...) (beg-block ...) (end-block ...) (lang ...) (beg-lang ...) (end-lang ...) (switches ...) (beg-switches ...) (end-switches ...) (header-args ...) (beg-header-args ...) (end-header-args ...) (body ...) (beg-body ...) (end-body ...)) ((lambda ... ...) (replace-regexp-in-string "[ 	]" "-" ...)) (let* (... ... ... ...) (unless ... ...)) (goto-char end-block))
  (while (re-search-forward org-babel-src-block-regexp nil t) (goto-char (match-beginning 0)) (let (... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) (... ...) (let* ... ...) (goto-char end-block)))
  (save-window-excursion (when file (find-file file)) (setq to-be-removed (current-buffer)) (goto-char (point-min)) (while (re-search-forward org-babel-src-block-regexp nil t) (goto-char ...) (let ... ... ... ...)))
  (let* ((file ...) (visited-p ...) (point ...) to-be-removed) (save-window-excursion (when file ...) (setq to-be-removed ...) (goto-char ...) (while ... ... ...)) (unless visited-p (kill-buffer to-be-removed)) (goto-char point))
  (org-babel-map-src-blocks (buffer-file-name) ((lambda ... ...) (replace-regexp-in-string "[ 	]" "-" ...)) (let* (... ... ... ...) (unless ... ...)))
  (let ((block-counter 1) (current-heading "") blocks) (org-babel-map-src-blocks (buffer-file-name) (... ...) (let* ... ...)) (setq blocks (mapcar ... blocks)) blocks)
  org-babel-tangle-collect-blocks(nil)
  (mapc (lambda (by-lang) (let* ... ...)) (org-babel-tangle-collect-blocks lang))
  (let ((block-counter 0) (org-babel-default-header-args ...) path-collector) (mapc (lambda ... ...) (org-babel-tangle-collect-blocks lang)) (message "tangled %d code block%s from %s" block-counter (if ... "" "s") (file-name-nondirectory ...)) (when org-babel-post-tangle-hook (mapc ... path-collector)) path-collector)
  (save-excursion (let (... ... path-collector) (mapc ... ...) (message "tangled %d code block%s from %s" block-counter ... ...) (when org-babel-post-tangle-hook ...) path-collector))
  (save-restriction (when only-this-block (unless ... ...) (save-match-data ...) (narrow-to-region ... ...)) (save-excursion (let ... ... ... ... path-collector)))
  org-babel-tangle(nil)
  call-interactively(org-babel-tangle nil nil)
--8<---------------cut here---------------end--------------->8---

Regards,
Bernt

Re: Tangling is broken in git master

[Achim Gratz]

Bernt Hansen writes:
> Tangling doesn't work for me in git master anymore.  Git bisect
> identifies the following commit as introducing the problem
[...]
> Debugger entered--Lisp error: (void-function letrec)
>   (letrec ((intersect ...)) (funcall intersect (case context ... ... ...) (split-string ...)))

From the NEWS file in Emacs 24:
--8<---------------cut here---------------start------------->8---
* Lisp changes in Emacs 24.1
[...]
*** New macro `letrec' to define recursive local functions.
--8<---------------cut here---------------end--------------->8---


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: Tangling is broken in git master

[Bernt Hansen]

Achim Gratz <Stromeko@nexgo.de> writes:

> Bernt Hansen writes:
>> Tangling doesn't work for me in git master anymore.  Git bisect
>> identifies the following commit as introducing the problem
> [...]
>> Debugger entered--Lisp error: (void-function letrec)
>>   (letrec ((intersect ...)) (funcall intersect (case context ... ... ...) (split-string ...)))
>
> From the NEWS file in Emacs 24:
> * Lisp changes in Emacs 24.1
> [...]
> *** New macro `letrec' to define recursive local functions.
>
>
> Regards,
> Achim.

I'm using GNU Emacs 23.2.1 (i486-pc-linux-gnu, GTK+ Version 2.20.0) of
2010-12-11 on raven, modified by Debian

Regards,
Bernt

Re: Tangling is broken in git master

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 248 bytes --]

Hi Bernt,

Bernt Hansen <bernt@norang.ca> writes:

> Tangling doesn't work for me in git master anymore.  Git bisect
> identifies the following commit as introducing the problem

Please try the attached patch and let us know if it works.

Thanks,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-compat.el-New-alias-org-letrec-for-labels.patch --]
[-- Type: text/x-patch, Size: 4048 bytes --]

From 45c517919756b7af78b720e454e8ea8d969f6a43 Mon Sep 17 00:00:00 2001
From: Bastien Guerry <bzg@altern.org>
Date: Sun, 12 Aug 2012 10:41:28 +0200
Subject: [PATCH] org-compat.el: New alias 'org-letrec for 'labels

* org-compat.el: New alias 'org-letrec for 'labels.

* org-bibtex.el (org-compat): Require.
(org-bibtex-headline): Use the `org-letrec' alias.

* ob.el (org-compat): Require.
(org-babel-noweb-p): Use the `org-letrec' alias.
---
 lisp/ob.el         |   12 +++++++-----
 lisp/org-bibtex.el |   42 ++++++++++++++++++++++--------------------
 lisp/org-compat.el |    2 ++
 3 files changed, 31 insertions(+), 25 deletions(-)

diff --git a/lisp/ob.el b/lisp/ob.el
index a6d1359..79f12f7 100644
--- a/lisp/ob.el
+++ b/lisp/ob.el
@@ -27,6 +27,7 @@
   (require 'cl))
 (require 'ob-eval)
 (require 'org-macs)
+(require 'org-compat)
 
 (defconst org-babel-exeext
   (if (memq system-type '(windows-nt cygwin))
@@ -2223,11 +2224,12 @@ header argument from buffer or subtree wide properties.")
 (defun org-babel-noweb-p (params context)
   "Check if PARAMS require expansion in CONTEXT.
 CONTEXT may be one of :tangle, :export or :eval."
-  (letrec ((intersect (lambda (as bs)
-			(when as
-			  (if (member (car as) bs)
-			      (car as)
-			    (funcall intersect (cdr as) bs))))))
+  (org-letrec
+      ((intersect (lambda (as bs)
+		    (when as
+		      (if (member (car as) bs)
+			  (car as)
+			(funcall intersect (cdr as) bs))))))
     (funcall intersect (case context
 			 (:tangle '("yes" "tangle" "no-export" "strip-export"))
 			 (:eval   '("yes" "no-export" "strip-export" "eval"))
diff --git a/lisp/org-bibtex.el b/lisp/org-bibtex.el
index 43b3c41..f857459 100644
--- a/lisp/org-bibtex.el
+++ b/lisp/org-bibtex.el
@@ -111,6 +111,7 @@
 (require 'bibtex)
 (eval-when-compile
   (require 'cl))
+(require 'org-compat)
 
 (defvar org-bibtex-description nil) ; dynamically scoped from org.el
 (defvar org-id-locations)
@@ -309,26 +310,27 @@ This variable is relevant only if `org-bibtex-export-tags-as-keywords` is t."
 
 (defun org-bibtex-headline ()
   "Return a bibtex entry of the given headline as a string."
-  (letrec ((val (lambda (key lst) (cdr (assoc key lst))))
-	   (to (lambda (string) (intern (concat ":" string))))
-	   (from (lambda (key) (substring (symbol-name key) 1)))
-	   (flatten (lambda (&rest lsts)
-		      (apply #'append (mapcar
-				       (lambda (e)
-					 (if (listp e) (apply flatten e) (list e)))
-				       lsts))))
-	   (notes (buffer-string))
-	   (id (org-bibtex-get org-bibtex-key-property))
-	   (type (org-bibtex-get org-bibtex-type-property-name))
-	   (tags (when org-bibtex-tags-are-keywords
-		   (delq nil
-			 (mapcar
-			  (lambda (tag)
-			    (unless (member tag
-					    (append org-bibtex-tags
-						    org-bibtex-no-export-tags))
-			      tag))
-			  (org-get-local-tags-at))))))
+  (org-letrec
+      ((val (lambda (key lst) (cdr (assoc key lst))))
+       (to (lambda (string) (intern (concat ":" string))))
+       (from (lambda (key) (substring (symbol-name key) 1)))
+       (flatten (lambda (&rest lsts)
+		  (apply #'append (mapcar
+				   (lambda (e)
+				     (if (listp e) (apply flatten e) (list e)))
+				   lsts))))
+       (notes (buffer-string))
+       (id (org-bibtex-get org-bibtex-key-property))
+       (type (org-bibtex-get org-bibtex-type-property-name))
+       (tags (when org-bibtex-tags-are-keywords
+	       (delq nil
+		     (mapcar
+		      (lambda (tag)
+			(unless (member tag
+					(append org-bibtex-tags
+						org-bibtex-no-export-tags))
+			  tag))
+		      (org-get-local-tags-at))))))
     (when type
       (let ((entry (format
 		    "@%s{%s,\n%s\n}\n" type id
diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index b049ecc..5f410bd 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -110,6 +110,8 @@ any other entries, and any resulting duplicates will be removed entirely."
 	    t))
       t)))
 
+(defalias 'org-letrec (if (>= emacs-major-version 24) 'letrec 'labels)
+
 \f
 ;;;; Emacs/XEmacs compatibility
 
-- 
1.7.10.2


[-- Attachment #3: Type: text/plain, Size: 14 bytes --]


-- 
 Bastien

Re: Tangling is broken in git master

[Bernt Hansen]

Bastien <bzg@gnu.org> writes:

> Hi Bernt,
>
> Bernt Hansen <bernt@norang.ca> writes:
>
>> Tangling doesn't work for me in git master anymore.  Git bisect
>> identifies the following commit as introducing the problem
>
> Please try the attached patch and let us know if it works.
>
> Thanks,

That doesn't work.  There's a missing ) at the end of the defalias and
after I add that I get

progn: Symbol's value as variable is void: intersect

Regards,
Bernt

Re: Tangling is broken in git master

[Bastien]

Bernt Hansen <bernt@norang.ca> writes:

> That doesn't work.  There's a missing ) at the end of the defalias and
> after I add that I get

Er, sorry for the typo.

I've reverted this commit for now, I'll see if I can get rid of
cl-labels another way.

Thanks,

-- 
 Bastien

Re: Tangling is broken in git master

[Eric Schulte]

Bastien <bzg@gnu.org> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>
>> That doesn't work.  There's a missing ) at the end of the defalias and
>> after I add that I get
>
> Er, sorry for the typo.
>
> I've reverted this commit for now, I'll see if I can get rid of
> cl-labels another way.
>

I've just pushed up another version of this commit, which I believe
removes cl-labels while still preserving tangling behavior.  If you have
a chance please re-check tangling with the latest Org-mode.

Thanks,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Tangling is broken in git master

[Bastien]

Hi Eric,

Eric Schulte <eric.schulte@gmx.com> writes:

> I've just pushed up another version of this commit, which I believe
> removes cl-labels while still preserving tangling behavior.  If you have
> a chance please re-check tangling with the latest Org-mode.

`letrec' is not available on Emacs <24.1 

Your commit looks like the one I pushed here...
http://orgmode.org/w/?p=org-mode.git;a=commit;h=ba16c3

... and reverted, thanks to Bernt's report.

Apart from one replacement of org-labels with `let*' in ob.el,
I don't see how we can get rid of `org-labels' completely.

-- 
 Bastien

Re: Tangling is broken in git master

[Eric Schulte]

Bastien <bzg@gnu.org> writes:

> Hi Eric,
>
> Eric Schulte <eric.schulte@gmx.com> writes:
>
>> I've just pushed up another version of this commit, which I believe
>> removes cl-labels while still preserving tangling behavior.  If you have
>> a chance please re-check tangling with the latest Org-mode.
>
> `letrec' is not available on Emacs <24.1 
>
> Your commit looks like the one I pushed here...
> http://orgmode.org/w/?p=org-mode.git;a=commit;h=ba16c3
>
> ... and reverted, thanks to Bernt's report.
>
> Apart from one replacement of org-labels with `let*' in ob.el,
> I don't see how we can get rid of `org-labels' completely.

Oh, my apologies, I just reverted my commit.  I thought letrec was an
old elisp construct.  I'm happy to stick with using org-labels (the code
was much more readable using org-labels).

I'm surprised that elisp doesn't provide any mechanism for local
anonymous functions.  I can't imagine why this would be an intentional
design decision.

Thanks,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Tangling is broken in git master

[Bastien]

Hi Eric,

Eric Schulte <eric.schulte@gmx.com> writes:

> I'm surprised that elisp doesn't provide any mechanism for local
> anonymous functions.  

(let ((my-local-func (lambda (a) (message a))))
     (funcall my-local-func "Hello!"))

is fine.

It's just for recursive local function -- letrec provides it now, 
but apparently cl-labels was needed for that before.

-- 
 Bastien

Re: Tangling is broken in git master

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> Eric Schulte <eric.schulte@gmx.com> writes:
>
>> I'm surprised that elisp doesn't provide any mechanism for local
>> anonymous functions.  
>
> (let ((my-local-func (lambda (a) (message a))))
>      (funcall my-local-func "Hello!"))
>
> is fine.
>
> It's just for recursive local function -- letrec provides it now, 
> but apparently cl-labels was needed for that before.

You can have recursive local functions:

#+begin_src emacs-lisp
(let* (len                              ; For byte compiler.
       (len (lambda (l) (if (not l) 0
                     (1+ (funcall len (cdr l)))))))
  (funcall len '(1 2 3)))
#+end_src


Regards,

-- 
Nicolas Goaziou

Re: Tangling is broken in git master

[Bastien]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> You can have recursive local functions:
>
> #+begin_src emacs-lisp
> (let* (len                              ; For byte compiler.
>        (len (lambda (l) (if (not l) 0
>                      (1+ (funcall len (cdr l)))))))
>   (funcall len '(1 2 3)))
> #+end_src

Indeed, neat!

-- 
 Bastien

Re: Tangling is broken in git master

[Nick Dokos]

Eric Schulte <eric.schulte@gmx.com> wrote:

> Bastien <bzg@gnu.org> writes:
> 
> > Hi Eric,
> >
> > Eric Schulte <eric.schulte@gmx.com> writes:
> >
> >> I've just pushed up another version of this commit, which I believe
> >> removes cl-labels while still preserving tangling behavior.  If you have
> >> a chance please re-check tangling with the latest Org-mode.
> >
> > `letrec' is not available on Emacs <24.1 
> >
> > Your commit looks like the one I pushed here...
> > http://orgmode.org/w/?p=org-mode.git;a=commit;h=ba16c3
> >
> > ... and reverted, thanks to Bernt's report.
> >
> > Apart from one replacement of org-labels with `let*' in ob.el,
> > I don't see how we can get rid of `org-labels' completely.
> 
> Oh, my apologies, I just reverted my commit.  I thought letrec was an
> old elisp construct.  I'm happy to stick with using org-labels (the code
> was much more readable using org-labels).
> 
> I'm surprised that elisp doesn't provide any mechanism for local
> anonymous functions.  I can't imagine why this would be an intentional
> design decision.
> 

Can't the definition of letrec in emacs24 be lifted bodily into org-compat.el
(or whatever the correct place is) as a compatibility-with-emacs-23 macro?

Nick

Re: Tangling is broken in git master

[Bastien]

Hi Nick,

Nick Dokos <nicholas.dokos@hp.com> writes:

> Can't the definition of letrec in emacs24 be lifted bodily into org-compat.el
> (or whatever the correct place is) as a compatibility-with-emacs-23 macro?

I don't think it's worth the effort.

The current code works and compiles without warnings for the user.

Getting rid of org-flet was to make things a bit more "elispy", 
but I'm fine with `org-labels' and those four lines of warnings-
for-developers-only.

-- 
 Bastien

Re: Tangling is broken in git master

[Nick Dokos]

Bastien <bzg@gnu.org> wrote:

> Hi Nick,
> 
> Nick Dokos <nicholas.dokos@hp.com> writes:
> 
> > Can't the definition of letrec in emacs24 be lifted bodily into org-compat.el
> > (or whatever the correct place is) as a compatibility-with-emacs-23 macro?
> 
> I don't think it's worth the effort.
> 
> The current code works and compiles without warnings for the user.
> 

What about future code? IME, it's always worthwhile to be thinking about
ways to avoid future bugs.

> Getting rid of org-flet was to make things a bit more "elispy", 
> but I'm fine with `org-labels' and those four lines of warnings-
> for-developers-only.
> 

It's more the freedom that it gives to developers (i.e Eric S. :-) ) who
are used to writing code a certain way: the way it is now, they've got
to keep in mind that letrec is not valid for emacs23 and write the code
differently - I'd suggest that that can be a source of bugs that would be
avoided with a compatibility macro.

But maybe the compatibility macro would be a bigger problem - I don't
know for sure. In particular, the file would need a periodic cleanup to
get rid of old cruft, but if it's a once-a-year-or-two occurrence, that
might not be too bad.

Nick

Re: Tangling is broken in git master

[Bastien]

Hi Nick,

Nick Dokos <nicholas.dokos@hp.com> writes:

> I'd suggest that that can be a source of bugs that would be
> avoided with a compatibility macro.

`org-labels' is an alias for `cl-labels' (when available) or `labels'.

There is no need for a compatibility macro here, as the current code 
is compatible with both Emacs 23 and Emacs 24 -- and hopefully later
versions of Emacs.

The question is rather whether we want to be very strict and replace
as much cl-* constructs as possible.

Nicolas shown a way to let-bind functions recursively (which is simple
and neat, by the way), so yes, I'd favor replacing org-labels in this
case.  Especially because the cl-labels and labels macros do not work
exactly the same way.

See labels' docstring:

...
Like `cl-labels' except that the lexical scoping is handled via `lexical-let'
rather than relying on `lexical-binding'.

> But maybe the compatibility macro would be a bigger problem - I don't
> know for sure. In particular, the file would need a periodic cleanup to
> get rid of old cruft, but if it's a once-a-year-or-two occurrence, that
> might not be too bad.

Always good to know there are many people we can rely on to clean the
old cruft :)

-- 
 Bastien