How to use a variable from outer scope in the success function of a request call?

How to use a variable from outer scope in the success function of a request call?

[Iñigo Serna]

Hi,

; I can't use a variable inside the success lambda function of a web request call.
; As this is difficult to explain I'll show some things I've tried.

(require 'request)
(fnX "TEXT") ; this is the function call.

; In `fn1' execution fails because the c variable in the message calls is not found inside the success function scope.
(defun fn1 (c)
  (request "https://api.stackexchange.com/2.1/questions"
           :params '((order . "desc")
                     (sort . "activity")
                     (site . "stackoverflow"))
           :parser 'json-read
           :success (function*
                     (lambda (&key data &allow-other-keys)
                       (let* ((item (elt (assoc-default 'items data) 0))
                              (title (assoc-default 'title item))
                              (tags (assoc-default 'tags item)))
                         (message "PARAMS: %s" c)
                         (message "=> %s %S" title tags)
                         (message "LAST: %s" c))))))

; In `fn2' I've added c as parameter for lambda function, but I don't know how to pass the external c variable to the lambda function. Now it doesn't fail, but it shows nil.
(defun fn2 (c)
  (request "https://api.stackexchange.com/2.1/questions"
           :params '((order . "desc")
                     (sort . "activity")
                     (site . "stackoverflow"))
           :parser 'json-read
           :success (function*
                     (lambda (&key data c &allow-other-keys)
                       (let* ((item (elt (assoc-default 'items data) 0))
                              (title (assoc-default 'title item))
                              (tags (assoc-default 'tags item)))
                         (message "PARAMS: %s" c)
                         (message "=> %s %S" title tags)
                         (message "LAST: %s" c))))))

; So I'm trying to use defmacros to pass the variable.
(defmacro my-request2 (url func &optional more)
  "Call REQUEST URL and execute FUNC on success."
  `(request ,url
            :params '((order . "desc")
                      (sort . "activity")
                      (site . "stackoverflow"))
            :parser 'json-read
            :success (function* (lambda (&key data &allow-other-keys) (funcall ,func data ,more)))))

 ; Here, `fn3' does not crash but it shows c as symbol 'c', not the contents of c variable.
(defun fn3 (c)
  (my-request2 "https://api.stackexchange.com/2.1/questions"
               #'(lambda (data &optional d)
                   (let* ((item (elt (assoc-default 'items data) 0))
                          (title (assoc-default 'title item))
                          (tags (assoc-default 'tags item)))
                     (message "PARAMS: %S" d)
                     (message "=> %s %S" title tags)
                     (message "LAST: %S" (nth 2 d))))
               '("OOOO" "AAAA" c)))

; `fn4' directly fails, and don't know why. I expect this alternative to work.
(defun fn4 (c)
  (my-request2 "https://api.stackexchange.com/2.1/questions"
               #'(lambda (data &optional d)
                   (let* ((item (elt (assoc-default 'items data) 0))
                          (title (assoc-default 'title item))
                          (tags (assoc-default 'tags item)))
                     (message "PARAMS: %S" d)
                     (message "=> %s %S" title tags)
                     (message "LAST: %S" (nth 2 d))))
               (list "OOOO" "AAAA" c)))

; Finally, `fn5' works, but this adds a second defmacro
(defmacro fn5 (c)
  `(my-request2 "https://api.stackexchange.com/2.1/questions"
                #'(lambda (data &optional d)
                    (let* ((item (elt (assoc-default 'items data) 0))
                           (title (assoc-default 'title item))
                           (tags (assoc-default 'tags item)))
                      (message "PARAMS: %S" d)
                      (message "=> %s %S" title tags)
                      (message "LAST: %S" (nth 2 d))))
                '("OOOO" "AAAA" ,c)))

(fn5 "TEXT")

; Is there any simpler way I'm missing? Why `fn4' does not work?


Thanks,
-- 
Iñigo Serna

Re: How to use a variable from outer scope in the success function of a request call?

[Pascal J. Bourguignon]

Iñigo Serna <inigoserna@gmail.com> writes:

> ; I can't use a variable inside the success lambda function of a web request call.
> ; As this is difficult to explain I'll show some things I've tried.

Add:

  (setf lexical-binding t)

before fn1, and:

;; -*- mode:emacs-lis;lexical-binding:t -*-  

as the first line of all your .el files.


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: How to use a variable from outer scope in the success function of a request call?

[Iñigo Serna]

Hi,

Pascal J. Bourguignon writes:
>> ; I can't use a variable inside the success lambda function of a web request call.
>> ; As this is difficult to explain I'll show some things I've tried.
>
> Add:
>   (setf lexical-binding t)
> before fn1, and:
>   ;; -*- mode:emacs-lis;lexical-binding:t -*-  
> as the first line of all your .el files.

Thanks a lot Pascal, that makes fn1 `work'!
And to learn more myself, do you have any hint why `fn3' or `fn4' don't work and how to make
it work?

Thanks,
Iñigo Serna

Re: How to use a variable from outer scope in the success function of a request call?

[Iñigo Serna]

Answering myself...

Iñigo Serna writes:
> And to learn more myself, do you have any hint why `fn3' or `fn4' don't work and how to make
> it work?

Using `lexical-let' with a dummy variable works:

(setq lexical-binding t)
(defun fn4 (c2)
  (lexical-let ((c c2))
    (my-request2 "https://api.stackexchange.com/2.1/questions"
                 #'(lambda (data &optional d)
                     (let* ((item (elt (assoc-default 'items data) 0))
                            (title (assoc-default 'title item))
                            (tags (assoc-default 'tags item)))
                       (message "PARAMS: %S" d)
                       (message "=> %s %S" title tags)
                       (message "LAST: %S" (nth 2 d))))
                 (list "OOOO" "AAAA" c))))
(fn4 "TEXT")

Btw, any solution to mark that c2 function parameter as "lexical" to
avoid using the intermediate variable and 'lexical-let' ?


This link has been of much help to understand what happened,
specially the "Another example which shows the difference" section:
    http://emacswiki.org/emacs/DynamicBindingVsLexicalBinding


Thanks for all the help,
Iñigo Serna

Re: How to use a variable from outer scope in the success function of a request call?

[Pascal J. Bourguignon]

Iñigo Serna <inigoserna@gmail.com> writes:

> Hi,
>
> Pascal J. Bourguignon writes:
>>> ; I can't use a variable inside the success lambda function of a web request call.
>>> ; As this is difficult to explain I'll show some things I've tried.
>>
>> Add:
>>   (setf lexical-binding t)
>> before fn1, and:
>>   ;; -*- mode:emacs-lis;lexical-binding:t -*-  
>> as the first line of all your .el files.
>
> Thanks a lot Pascal, that makes fn1 `work'!
> And to learn more myself, do you have any hint why `fn3' or `fn4' don't work and how to make
> it work?
>
> Thanks,
> Iñigo Serna
>

Use pp macroexpand to see what your macro calls expand to:

(pp (macroexpand '(my-request2 "https://api.stackexchange.com/2.1/questions"
                   #'(lambda (data &optional d)
                       (let* ((item (elt (assoc-default 'items data) 0))
                              (title (assoc-default 'title item))
                              (tags (assoc-default 'tags item)))
                         (message "PARAMS: %S" d)
                         (message "=> %s %S" title tags)
                         (message "LAST: %S" (nth 2 d))))
                   '("OOOO" "AAAA" c))))

(request "https://api.stackexchange.com/2.1/questions" :params
         '((order . "desc")
           (sort . "activity")
           (site . "stackoverflow"))
                                                       :parser 'json-read :success
         (function*
          (lambda
              (&key data &allow-other-keys)
           (funcall
            #'(lambda
                  (data &optional d)
                (let*
                    ((item
                       (elt
                        (assoc-default 'items data)
                        0))
                     (title
                         (assoc-default 'title item))
                     (tags
                       (assoc-default 'tags item)))
                  (message "PARAMS: %S" d)
                  (message "=> %s %S" title tags)
                  (message "LAST: %S"
                           (nth 2 d))))
            data
            '("OOOO" "AAAA" c)))))

From this, you can see that the parameter c of fn3 is never used
(you just pass the symbol c to your callback).


Similarly, for fn4:

(pp (macroexpand '(my-request2 "https://api.stackexchange.com/2.1/questions"
                   #'(lambda (data &optional d)
                       (let* ((item (elt (assoc-default 'items data) 0))
                              (title (assoc-default 'title item))
                              (tags (assoc-default 'tags item)))
                         (message "PARAMS: %S" d)
                         (message "=> %s %S" title tags)
                         (message "LAST: %S" (nth 2 d))))
                   (list "OOOO" "AAAA" c))))

(request "https://api.stackexchange.com/2.1/questions" :params
         '((order . "desc")
           (sort . "activity")
           (site . "stackoverflow"))
                                                       :parser 'json-read :success
         (function*
          (lambda
              (&key data &allow-other-keys)
           (funcall
            #'(lambda
                  (data &optional d)
                (let*
                    ((item
                       (elt
                        (assoc-default 'items data)
                        0))
                     (title
                         (assoc-default 'title item))
                     (tags
                       (assoc-default 'tags item)))
                  (message "PARAMS: %S" d)
                  (message "=> %s %S" title tags)
                  (message "LAST: %S"
                           (nth 2 d))))
            data
            (list "OOOO" "AAAA" c)))))

You can see that your macro expands to a call to request with as
argument an anonymous function that refers a free variable named c.

However, without lexical-binding, variables are dynamic.  

    lexical = space = where
    dynamic = time  = when

Therefore WHEN this anonymous function is called, it may happen that no
variable named c exist anymore. Or worse, than another variable named c
THEN exist that is not the one that existed when your function fn4 was
executing!



In the context of Common Lisp, here is an explaination of
lexical/dynamic and global/local
https://groups.google.com/forum/#!msg/comp.lang.lisp/4VyopdWcFI4/1sDQU-3H8VgJ

Google also about the FUNARG problem.

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: How to use a variable from outer scope in the success function of a request call?

[Stefan Monnier]

>   (setf lexical-binding t)

Please never do that.

> ;; -*- lexical-binding:t -*-  
> as the first line of all your .el files.

Yes, this is the important setting.


        Stefan

Re: How to use a variable from outer scope in the success function of a request call?

[Iñigo Serna]

Hi again

Pascal J. Bourguignon writes:
> [...]
>
> However, without lexical-binding, variables are dynamic.  
>
>     lexical = space = where
>     dynamic = time  = when
>
> Therefore WHEN this anonymous function is called, it may happen that no
> variable named c exist anymore. Or worse, than another variable named c
> THEN exist that is not the one that existed when your function fn4 was
> executing!
>
> In the context of Common Lisp, here is an explaination of
> lexical/dynamic and global/local
> https://groups.google.com/forum/#!msg/comp.lang.lisp/4VyopdWcFI4/1sDQU-3H8VgJ
>
> Google also about the FUNARG problem.

I understand the differences now.

Thanks a lot Pascal for your kind explanations.

Best regards,
Iñigo Serna