extract URLs from an emacs (especially gnus) buffer

extract URLs from an emacs (especially gnus) buffer

[Sam Steingold]

Hi,
How do I extract all URLs from an emacs buffer?
(I am especially interested in gnus *Article* buffers).
E.g., the slashdot articles contain _buttons_, not actual urls, so what
I need to do is:

--8<---------------cut here---------------start------------->8---
    (let ((urls ()) (pos (point-min)))
      (while (setq pos (next-single-property-change pos 'follow-link))
        (push (get-char-property pos 'shr-url) urls)
        (setq pos (next-single-property-change pos 'follow-link)))
       urls)
--8<---------------cut here---------------end--------------->8---

On the other hand, raw URLs in other articles (e.g., in this group) have
to be collected by

--8<---------------cut here---------------start------------->8---
    (let ((urls ()) (pos (point-min)))
      (while (setq pos (next-single-property-change pos 'button))
        (let ((widget (get-char-property pos 'button)))
          (push (buffer-substring-no-properties
                  (widget-get widget :from)
                  (widget-get widget :to))
                urls))
        (setq pos (next-single-property-change pos 'button)))
      urls)
--8<---------------cut here---------------end--------------->8---

Maybe there are other ways for other modes?

So, how do I automate the following algorithm in elisp:

1. (goto-char (point-min))
2. until eobp
3.  find next clickable area
4.  if the click target is a url, click on it

Thanks.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1265
http://www.childpsy.net/ http://truepeace.org http://iris.org.il
http://palestinefacts.org http://memri.org http://ffii.org
Nothing is as important as you think it is when you are thinking about it.

Re: extract URLs from an emacs (especially gnus) buffer

[Sam Steingold]

> * Sam Steingold <fqf@tah.bet> [2014-02-20 16:18:40 -0500]:
>
> How do I extract all URLs from an emacs buffer?
> (I am especially interested in gnus *Article* buffers).

This is not what I originally wanted (without a filter it pushes all
buttons including e-mail addresses) but, I guess, it will do for now:

--8<---------------cut here---------------start------------->8---
(defun sds-push-button (buffer button)
  (with-current-buffer buffer
    (widget-apply button :action nil)))
(defun sds-push-all-buttons (buffer &optional filter)
  "Push all buttons in the buffer matching the filter."
  (interactive "bBuffer: \nP")
  (with-current-buffer buffer
    (setq filter (cond ((stringp filter) filter)
                       ((null filter) "^Link$")
                       (t (read-string "Filter (regexp): "))))
    (let ((delay 1) (count 0) (pos (point-min)))
      (while (setq pos (next-property-change pos))
        (let ((button (get-char-property pos 'button)))
          (when (and button (widgetp button)
                     (string-match filter
                                   (buffer-substring-no-properties
                                    (widget-get button :from)
                                    (widget-get button :to))))
            (run-with-timer delay nil 'sds-push-button buffer button)
            (incf delay 5)
            (incf count))))
      (message "Scheduled loading %d URLs" count))))
--8<---------------cut here---------------end--------------->8---


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1265
http://www.childpsy.net/ http://memri.org http://ffii.org
http://jihadwatch.org http://americancensorship.org http://mideasttruth.com
People with a good taste are especially appreciated by cannibals.