Stripping HTTP headers from an HTTP response?

Stripping HTTP headers from an HTTP response?

[Elena]

Hello,

I'd like to get some HTML documents via HTTP for processing. I'm using
`url-retrieve-synchronously' to retrieve the HTTP response and it
works. However it returns the full HTTP response (that is: including
headers) and it seems there is not a function to extract the embedded
HTML document. Should I just strip lines until the first empty one, or
there is a better way?

I've tried the package `http-get.el' but it hangs Emacs for a while
before failing, maybe because I'm beyond a proxy.

Thanks.

Re: Stripping HTTP headers from an HTTP response?

[Klaus Straubinger]

Elena <egarrulo@gmail.com> wrote:

> I'd like to get some HTML documents via HTTP for processing.

url-insert-file-contents is a good function to use for that purpose,
probably in a temporary buffer.

For interactive use, you could also try browse-url-emacs.

> Should I just strip lines until the first empty one

This works, too.

-- 
Klaus Straubinger

Re: Stripping HTTP headers from an HTTP response?

[Elena]

On 15 Dic, 14:44, Klaus Straubinger <KSN...@UseNet.ArcorNews.DE>
wrote:
> Elena <egarr...@gmail.com> wrote:
> > I'd like to get some HTML documents via HTTP for processing.
>
> url-insert-file-contents is a good function to use for that purpose,

That's exactly what I was after. Thanks.

Re: Stripping HTTP headers from an HTTP response?

[Tim X]

Klaus Straubinger <KSNetz@UseNet.ArcorNews.DE> writes:

> Elena <egarrulo@gmail.com> wrote:
>
>> I'd like to get some HTML documents via HTTP for processing.
>
> url-insert-file-contents is a good function to use for that purpose,
> probably in a temporary buffer.
>
> For interactive use, you could also try browse-url-emacs.
>
>> Should I just strip lines until the first empty one
>
> This works, too.

There is also wget.el, which provides an interface to wget, which will
work through a proxy. However, from memory (which is a bit rusty), I
seem to remember there was either a function in url.el which would
either strip the headers or return the contents without headers. Check
for optional arguments on some of the functions. There is also an
interface to curl for emacs. 

Another place to get some ideas would be the sources to emacs w3.

Tim

-- 
tcross (at) rapttech dot com dot au

Re: Stripping HTTP headers from an HTTP response?

[Klaus Straubinger]

Tim X <timx@nospam.dev.null> wrote:

> Another place to get some ideas would be the sources to emacs w3.

And there we find

| (defun w3-download-callback (fname)
|   (let ((coding-system-for-write 'binary))
|     (goto-char (point-min))
|     (search-forward "\n\n" nil t)
|     (write-region (point) (point-max) fname))
|   (url-mark-buffer-as-dead (current-buffer))
|   (message "Download of %s complete." (url-view-url t))
|   (sit-for 3))

The simple solution to search for the first empty line seemed to be
good enough.

-- 
Klaus Straubinger