A (probably silly) problem with request.el

A (probably silly) problem with request.el

[Marcin Borkowski]

Hi all,

I'd like to write a small library which needs to make HTTP requests.
Now I'm completely new to this whole networking thing, but I know I can
say e.g.

curl http://google.com

at the command line.  However,

(require 'request)
(request "http://google.com")

won't do anything good - it yields this:

--8<---------------cut here---------------start------------->8---
[cl-struct-request-response nil nil nil nil nil "http://google.com" nil
			    (:error
			     (closure
			      (t)
			      (&rest args)
			      (apply 'request-default-error-callback '"http://google.com" args))
			     :url "http://google.com" :response #0)
			    #<buffer  *request curl*> nil nil curl nil]
--8<---------------cut here---------------end--------------->8---

What am I doing wrong?

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: A (probably silly) problem with request.el

[Nicolas Richard]

Marcin Borkowski <mbork@mbork.pl> writes:

> Hi all,
>
> I'd like to write a small library which needs to make HTTP requests.
> Now I'm completely new to this whole networking thing, but I know I can
> say e.g.
>
> curl http://google.com
>
> at the command line.  However,
>
> (require 'request)
> (request "http://google.com")

Google provides examples of use:
http://tkf.github.io/emacs-request/

Also C-h f request RET provides documentation.

HTH,

-- 
Nico.

Re: A (probably silly) problem with request.el

[Marcin Borkowski]

On 2015-06-05, at 12:09, Nicolas Richard <youngfrog@members.fsf.org> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> Hi all,
>>
>> I'd like to write a small library which needs to make HTTP requests.
>> Now I'm completely new to this whole networking thing, but I know I can
>> say e.g.
>>
>> curl http://google.com
>>
>> at the command line.  However,
>>
>> (require 'request)
>> (request "http://google.com")
>
> Google provides examples of use:
> http://tkf.github.io/emacs-request/

I tried them, with similar results.

> Also C-h f request RET provides documentation.

I read it, but could not fathom what am I doing wrong (nor why the
attached examples seem not to work).

> HTH,

Thanks anyway,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: A (probably silly) problem with request.el

[Nicolas Richard]

Le 05/06/2015 16:02, Marcin Borkowski a écrit :
> On 2015-06-05, at 12:09, Nicolas Richard <youngfrog@members.fsf.org> wrote:
>> Marcin Borkowski <mbork@mbork.pl> writes:
>>> (require 'request)
>>> (request "http://google.com")
>>
>> Google provides examples of use:
>> http://tkf.github.io/emacs-request/
> 
> I tried them, with similar results.

What do you expect ?

The (request ...) form will always (I guess, I did not check the code)
return a structure like the one you pasted. The real job must be done by
a callback function. (Probably in the synchronous case it's different, I
don't know, but as the docstring says: don't use that except for testing
purposes.)

Nicolas.

Re: A (probably silly) problem with request.el

[Marcin Borkowski]

On 2015-06-05, at 17:40, Nicolas Richard <nrichard@members.fsf.org> wrote:

> Le 05/06/2015 16:02, Marcin Borkowski a écrit :
>> On 2015-06-05, at 12:09, Nicolas Richard <youngfrog@members.fsf.org> wrote:
>>> Marcin Borkowski <mbork@mbork.pl> writes:
>>>> (require 'request)
>>>> (request "http://google.com")
>>>
>>> Google provides examples of use:
>>> http://tkf.github.io/emacs-request/
>> 
>> I tried them, with similar results.
>
> What do you expect ?
>
> The (request ...) form will always (I guess, I did not check the code)
> return a structure like the one you pasted. The real job must be done by
> a callback function. (Probably in the synchronous case it's different, I
> don't know, but as the docstring says: don't use that except for testing
> purposes.)

OK.  As I said, my problem is (probably) silly.  Now the question is:
how do I do anything reasonable with what I got?  In my particular case
I expect json, so I'd like to use e.g. json.el to convert it to s-exps.
How do I do that?  I tried

(request "http://httpbin.org/get" :parser #'json-read),

expecting a sexp, but I still got a structure very much like
previously.

> Nicolas.

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: A (probably silly) problem with request.el

[Nicolas Richard]

Marcin Borkowski <mbork@mbork.pl> writes:
> Now the question is:
> how do I do anything reasonable with what I got?  In my particular case
> I expect json, so I'd like to use e.g. json.el to convert it to s-exps.
> How do I do that?  I tried
>
> (request "http://httpbin.org/get" :parser #'json-read),

The request form will *never*[1] return json, because it returns before the
request is complete. This is what asynchronous means, and that is why I
said "The real job must be done by a callback function".

The very first example in the homepage of the package uses httpbin.org.
Here's a simplified version of it :

(request
 "http://httpbin.org/get"
 :parser 'json-read
 :success (function*
           (lambda (&key data &allow-other-keys)
             (message "response: %S" data))))

Please try it and see what happens. The function mentionned as :success
takes over once the request is complete (well, unless the request fails
obviously...). That is a "callback" function.

HTH,

-- 
Nico.

[1] Except in the synchronous case, e.g.
(request-response-data
   (request "http://httpbin.org/get" :parser 'json-read :sync t))
But the doc explicitly says to not use that.

Re: A (probably silly) problem with request.el

[Marcin Borkowski]

On 2015-06-06, at 21:25, Nicolas Richard <youngfrog@members.fsf.org> wrote:

> HTH,

Yes it does.  Thanks a lot!

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: A (probably silly) problem with request.el

[Marcin Borkowski]

On 2015-06-06, at 21:25, Nicolas Richard <youngfrog@members.fsf.org> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>> Now the question is:
>> how do I do anything reasonable with what I got?  In my particular case
>> I expect json, so I'd like to use e.g. json.el to convert it to s-exps.
>> How do I do that?  I tried
>>
>> (request "http://httpbin.org/get" :parser #'json-read),
>
> The request form will *never*[1] return json, because it returns before the
> request is complete. This is what asynchronous means, and that is why I
> said "The real job must be done by a callback function".
>
> The very first example in the homepage of the package uses httpbin.org.
> Here's a simplified version of it :
>
> (request
>  "http://httpbin.org/get"
>  :parser 'json-read
>  :success (function*
>            (lambda (&key data &allow-other-keys)
>              (message "response: %S" data))))
>
> Please try it and see what happens. The function mentionned as :success
> takes over once the request is complete (well, unless the request fails
> obviously...). That is a "callback" function.

OK, so now that I played with it for a moment, let me ask more questions
(not very urgent ones, I'm just curious).  Do I get it correctly that
the "success" function will be called *at some later point* (assuming
that the request succeeded)?  What Emacs mechanism is used for that -
timers?  (I assume that the asynchronicity is more or less natural when
using curl, and then async processes are used, right?  Does url.el also
use some external tool?)

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: A (probably silly) problem with request.el

[Nicolas Richard]

Marcin Borkowski <mbork@mbork.pl> writes:
> OK, so now that I played with it for a moment, let me ask more questions
> (not very urgent ones, I'm just curious).  Do I get it correctly that
> the "success" function will be called *at some later point* (assuming
> that the request succeeded)?

Yes.

>                               What Emacs mechanism is used for that -
> timers?

I don't think so.

>          (I assume that the asynchronicity is more or less natural when
> using curl, and then async processes are used, right?  

I don't know the internals of request.el (I never used it, and
discovered it when reading this thread), but that would be my guess too.

>                                                        Does url.el also
> use some external tool?)

Network processes are also asynchronous in emacs. See
make-network-process for the low lever thing, and url-retrieve.

Best,

-- 
Nico.

Re: A (probably silly) problem with request.el

[Marcin Borkowski]

On 2015-06-06, at 21:25, Nicolas Richard <youngfrog@members.fsf.org> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>> Now the question is:
>> how do I do anything reasonable with what I got?  In my particular case
>> I expect json, so I'd like to use e.g. json.el to convert it to s-exps.
>> How do I do that?  I tried
>>
>> (request "http://httpbin.org/get" :parser #'json-read),
>
> The request form will *never*[1] return json, because it returns before the
> request is complete. This is what asynchronous means, and that is why I
> said "The real job must be done by a callback function".
>
> The very first example in the homepage of the package uses httpbin.org.
> Here's a simplified version of it :
>
> (request
>  "http://httpbin.org/get"
>  :parser 'json-read
>  :success (function*
>            (lambda (&key data &allow-other-keys)
>              (message "response: %S" data))))
>
> Please try it and see what happens. The function mentionned as :success
> takes over once the request is complete (well, unless the request fails
> obviously...). That is a "callback" function.

OK, so I have now another (though related) problem.  What I want is to
write a function which will accept the url, feed it to =request=, and
/return/ the json-translated-to-sexp result.  I tried this:

(require 'request-deferred)

(deferred:$
  (request-deferred "http://httpbin.org/get" :parser 'json-read)
  (deferred:nextc it
    (lambda (response)
      (request-response-data response))))

(slightly modified from http://tkf.github.io/emacs-request/manual.html),
but it didn't seem to work.  (It just returns the struct, not the sexp.)
Unfortunately, I do not really understand request's (nor deferred's)
docs well enough to be able to debug it myself.  Any hints?

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: A (probably silly) problem with request.el

[John Mastro]

Marcin Borkowski <mbork@mbork.pl> wrote:
> OK, so I have now another (though related) problem.  What I want is to
> write a function which will accept the url, feed it to =request=, and
> /return/ the json-translated-to-sexp result.  I tried this:
>
> (require 'request-deferred)
>
> (deferred:$
>   (request-deferred "http://httpbin.org/get" :parser 'json-read)
>   (deferred:nextc it
>     (lambda (response)
>       (request-response-data response))))
>
> (slightly modified from http://tkf.github.io/emacs-request/manual.html),
> but it didn't seem to work.  (It just returns the struct, not the sexp.)
> Unfortunately, I do not really understand request's (nor deferred's)
> docs well enough to be able to debug it myself.  Any hints?

Try this:

(defun my-request-get (url &optional params)
  (request-response-data (request
                          url
                          :params params
                          :parser #'json-read
                          :sync t)))

If you essentially don't want the request to be asyncronous, then I
think `:sync t' is the way to go.

-- 
john