From: Noam Postavsky <npostavs@gmail.com>
To: stardiviner <numbchild@gmail.com>
Cc: 31427@debbugs.gnu.org
Subject: bug#31427: 27.0.50; libxml-parse-html-region stripped a big node
Date: Sun, 13 May 2018 10:06:54 -0400 [thread overview]
Message-ID: <87wow7k5tt.fsf@gmail.com> (raw)
In-Reply-To: <87k1s9e12y.fsf@gmail.com> (stardiviner's message of "Sat, 12 May 2018 16:21:25 +0800")
tags 31427 + notabug
quit
stardiviner <numbchild@gmail.com> writes:
> As the background here: https://github.com/AdamNiederer/elquery/issues/5
>
> ,----
> | (search "\"content" ; changing to "\"menu" reveals the <div id="menu">
> | (prin1-to-string (with-temp-buffer
> | (insert-string (with-current-buffer (url-retrieve-synchronously "https://nginx.org/en/docs/dirindex.html")
> | (buffer-string)))
> | (let ((tree (libxml-parse-html-region (point-min) (point-max))))
> | tree))))
> `----
>
> libxml-parse-html-region stipped the <div id="content"> node (which is big).
The problem is that the buffer given by `url-retrieve-synchronously' is
unibyte, then you insert the unibyte buffer-string into a multibyte
buffer (the temp buffer), which gives bad results.
The functions `search' and `insert-string' don't exist, so I
reformulated the code above:
(let ((case-fold-search nil)
(str (prin1-to-string
(with-temp-buffer
(insert (with-current-buffer (url-retrieve-synchronously "https://nginx.org/en/docs/dirindex.html")
(buffer-string)))
(let ((tree (libxml-parse-html-region (point-min) (point-max))))
tree)))))
(and (string-match "\"content" str)
(match-string 0 str)))
;=> nil
(let ((case-fold-search nil)
(str (prin1-to-string
(with-temp-buffer
(set-buffer-multibyte nil) ; <--------- set the temp buffer to unibyte
(insert (with-current-buffer (url-retrieve-synchronously "https://nginx.org/en/docs/dirindex.html")
(buffer-string)))
(let ((tree (libxml-parse-html-region (point-min) (point-max))))
tree)))))
(and (string-match "\"content" str)
(match-string 0 str)))
;=> "\"content"
Instead of setting the temp buffer to multibyte, it would probably be
more correct to check the "Content-Type: text/html; charset=utf-8" HTTP
header, and decode it accordingly. It looks like the libxml-parse-*
functions assume utf-8, so it works out either way in this particular
case.
next prev parent reply other threads:[~2018-05-13 14:06 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-12 8:21 bug#31427: 27.0.50; libxml-parse-html-region stripped a big node stardiviner
2018-05-13 14:06 ` Noam Postavsky [this message]
2018-05-14 1:04 ` stardiviner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87wow7k5tt.fsf@gmail.com \
--to=npostavs@gmail.com \
--cc=31427@debbugs.gnu.org \
--cc=numbchild@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.