From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: [ELPA] New package: luwak Date: Tue, 25 Oct 2022 15:09:06 -0400 Message-ID: References: <877d0u7zqd.fsf@ypei.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18489"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Emacs Devel mailing list To: Yuchen Pei Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Oct 25 21:12:00 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1onPLE-0004Yn-38 for ged-emacs-devel@m.gmane-mx.org; Tue, 25 Oct 2022 21:12:00 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1onPIb-0001WX-Os; Tue, 25 Oct 2022 15:09:17 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1onPIY-0001LU-OA for emacs-devel@gnu.org; Tue, 25 Oct 2022 15:09:14 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1onPIW-0007bg-8c for emacs-devel@gnu.org; Tue, 25 Oct 2022 15:09:14 -0400 Original-Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 85B41802B3; Tue, 25 Oct 2022 15:09:10 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 0A6AF80091; Tue, 25 Oct 2022 15:09:08 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1666724948; bh=sclnKVuImSKeke6MeTApgQbp0SpKIA89lAACuhsp0e8=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From; b=SgxZQ3tJy8KKnU1zbYeI7g64qwIXzZ/qn9R0oKbum76XZfu/cnjF/yw1kG9ua9CL+ JQFQprdvtGyRDBEw8YkPJDRV5+j0XFTSVX4N2ZYtg6IAEjyNx3o3bNAdhMq/XoEZp5 18oFEBd2mviy8106CMDMsacH0Q/1Wal48kpoDenSMLk6xWU0HF1yueaUp9yH2/kHFL yfNw8ijNxyzs1fys1SFHJ8fbu8P92wgSRgOYFYy9ThMDXq8eUhRrSrPZTG5QaEb+Yk /fAYdll8mb17QiooEd6+sWPglGG9BXH5NjTE6AwUaUqWdbJkgv4f1LAJjpc1fP3Z9T YKwUgcnHO2qEg== Original-Received: from alfajor (unknown [45.44.229.252]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id D6ECC120F20; Tue, 25 Oct 2022 15:09:07 -0400 (EDT) In-Reply-To: <877d0u7zqd.fsf@ypei.org> (Yuchen Pei's message of "Fri, 21 Oct 2022 12:05:30 +1100") Received-SPF: pass client-ip=132.204.25.50; envelope-from=monnier@iro.umontreal.ca; helo=mailscanner.iro.umontreal.ca X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: "Emacs-devel" Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:298491 Archived-At: > It has been a while since Emacs won the Editor War^[Citations needed]. > Do you, like me, sometimes wake up in the middle of the night, feeling > something is amiss, only to realise after M-x list-packages that there > are not enough web browsers in ELPA for a Browser War (inside Emacs)? It shouldn't take much work to resuscitate W3, if needed. > luwak[1] is a simple web browser in Emacs, utilising the power of lynx > -dump[2]. It is currently text-only and GET-only. Sounds good. Except I'd much prefer it tried to improve on EWW rather than reinvent a wheel that's just as square but in a different way. > Features: > > - Asynchronous loading > - Some usual browser features: open, reload, search with a search > engine, follow links, go forward / backward in history, copy url of > the current page or link at point > - Completion from persistent history in prompt to open a url > - Multiple ways of rendering links: numbered, forward-sexp or hide > altogether > - Quickly open a link on the page with completion for url / link id > - imenu support, from all unindented strings (which look like > headings) > - Support of storing and capturing for org mode, guessing the title > (first imenu item) > - Write the dump of the current page to a file > - Render a buffer containing a lynx dump in the luwak mode > - Browse with or without torsocks Many of those features seem independent from whether the rendering is done by w3m, lynx, shr, ... So could you try and merge this code with that of EWW. More specifically: - Divide your code into a "backend" part that deals with calling lynx, and a frontend part which provides features by analysing the rendered text. - Add your frontend features to EWW. - change EWW so it can call your backend instead of shr. This last part may be more difficult depending on how easy it is to align the details of shr's rendering with those of lynx. Admittedly, we'd then still suffer from "not enough web browsers", but at least users wouldn't have to choose between incomparable sets of features. WDYT? In the mean time I've added `luwak` to `elpa.git`. You might want to install and/or look at the patch below as well. Stefan diff --git a/.gitignore b/.gitignore index 628aad4cc8..b3b5372679 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,4 @@ *~ -*.elc \ No newline at end of file +*.elc +/luwak-autoloads.el +/luwak-pkg.el diff --git a/luwak.el b/luwak.el index 236a25f42e..a283e13536 100644 --- a/luwak.el +++ b/luwak.el @@ -37,34 +37,39 @@ (defvar luwak-history-file "~/.emacs.d/luwak-history") (defun luwak-lynx-buffer (url) (format "*luwak-lynx %s*" url)) + +(defgroup luwak () + "Web browser based on lynx -dump." + :group 'web) + (defcustom luwak-search-engine "https://html.duckduckgo.com/html?q=%s" "Default search engine for use in 'luwak-search'." - :group 'luwak :type '(string)) + :type '(string)) (defcustom luwak-url-rewrite-function 'identity "Function to rewrite url before loading." - :group 'luwak :type '(function)) + :type '(function)) (defcustom luwak-tor-switch t "Switch behaviour of prefix arg concerning the use of tor. When nil, use tor by default (requires a tor daemon having been started in the system), and not use it with a prefix arg. When non-nill, swap the tor-switch in prefix-arg effect." - :group 'luwak :type '(boolean)) + :type '(boolean)) (defcustom luwak-max-history-length 100 "Maximum history length." - :group 'luwak :type '(natnum)) + :type '(natnum)) (defcustom luwak-render-link-function 'luwak-render-link-id "Function to render a link." - :group 'luwak :type '(choice (const luwak-render-link-id) - (const luwak-render-link-forward-sexp) - (const luwak-render-link-hide-link))) + :type '(choice (const luwak-render-link-id) + (const luwak-render-link-forward-sexp) + (const luwak-render-link-hide-link))) (defcustom luwak-keep-history t "If non-nil, will keep history in 'luwak-history-file'." - :group 'luwak :type '(boolean)) + :type '(boolean)) (defcustom luwak-use-history t "If non-nil, will use history from the 'luwak-history-file' when invoking 'luwak-open'." - :group 'luwak :type '(boolean)) + :type '(boolean)) (put luwak-history 'history-length luwak-max-history-length) @@ -81,18 +86,18 @@ non-nill, swap the tor-switch in prefix-arg effect." (defvar luwak-mode-map (let ((kmap (make-sparse-keymap))) - (define-key kmap "\t" 'forward-button) - (define-key kmap [backtab] 'backward-button) - (define-key kmap "g" 'luwak-reload) - (define-key kmap "l" 'luwak-history-backward) - (define-key kmap "r" 'luwak-history-forward) - (define-key kmap "w" 'luwak-copy-url) - (define-key kmap "o" 'luwak-open) - (define-key kmap "s" 'luwak-search) - (define-key kmap "d" 'luwak-save-dump) - (define-key kmap "j" 'imenu) - (define-key kmap "t" 'luwak-toggle-links) - (define-key kmap "a" 'luwak-follow-numbered-link) + (define-key kmap "\t" #'forward-button) + (define-key kmap [backtab] #'backward-button) + (define-key kmap "g" #'luwak-reload) + (define-key kmap "l" #'luwak-history-backward) + (define-key kmap "r" #'luwak-history-forward) + (define-key kmap "w" #'luwak-copy-url) + (define-key kmap "o" #'luwak-open) + (define-key kmap "s" #'luwak-search) + (define-key kmap "d" #'luwak-save-dump) + (define-key kmap "j" #'imenu) + (define-key kmap "t" #'luwak-toggle-links) + (define-key kmap "a" #'luwak-follow-numbered-link) kmap)) (define-derived-mode luwak-mode special-mode (luwak-mode-name) @@ -120,14 +125,15 @@ non-nill, swap the tor-switch in prefix-arg effect." (buffer-substring-no-properties (1- (point)) (progn (end-of-line 1) (point)))))) -(when (require 'org nil t) - (defun luwak-org-store-link () - (when (derived-mode-p 'luwak-mode) - (org-link-store-props - :type "luwak" - :link (plist-get luwak-data :url) - :description (luwak-guess-title)))) +(defun luwak-org-store-link () + (when (derived-mode-p 'luwak-mode) + (org-link-store-props + :type "luwak" + :link (plist-get luwak-data :url) + :description (luwak-guess-title)))) +;; FIXME: `org' is always available, so this should never fail! +(when (require 'org nil t) (org-link-set-parameters "luwak" :follow #'luwak-open :store #'luwak-org-store-link)) @@ -138,13 +144,14 @@ non-nill, swap the tor-switch in prefix-arg effect." (interactive (list (if luwak-use-history - (car + (car ;FIXME: Why throw away everything after space? (split-string - (completing-read "Url to open: " (luwak-history-collection-from-file)))) + (completing-read "Url to open: " + (luwak-history-collection-from-file)))) (read-string "Url to open: ")))) (luwak-open-url (url-encode-url url) - (xor luwak-tor-switch current-prefix-arg) 'luwak-add-to-history)) + (xor luwak-tor-switch current-prefix-arg) #'luwak-add-to-history)) (defun luwak-history-collection-from-file () (split-string @@ -162,7 +169,7 @@ non-nill, swap the tor-switch in prefix-arg effect." ;;;###autoload (defun luwak-search (query) - "Search QUERY using 'luwak-search-engine'." + "Search QUERY using `luwak-search-engine'." (interactive "sLuwak search query: ") (luwak-open (format luwak-search-engine query))) @@ -265,7 +272,7 @@ non-nill, swap the tor-switch in prefix-arg effect." (defun luwak-follow-link (marker) (let ((url (get-text-property marker 'url))) (luwak-open-url - url (plist-get luwak-data :no-tor) 'luwak-add-to-history))) + url (plist-get luwak-data :no-tor) #'luwak-add-to-history))) (defun luwak-render-links (urls) (with-current-buffer luwak-buffer @@ -335,7 +342,7 @@ non-nill, swap the tor-switch in prefix-arg effect." (goto-char (point-min)) (re-search-forward "^References\n\n\\(\\ *Visible links:\n\\)?" nil t) (delete-region (point-min) (match-end 0)) - (seq-filter 'identity + (seq-filter #'identity ;`delq nil' ? (mapcar (lambda (s) (when (string-match "^\\ *\\([0-9]+\\)\\. \\(.*\\)" s) (concat (match-string 1 s) " " (match-string 2 s)))) @@ -347,11 +354,9 @@ non-nill, swap the tor-switch in prefix-arg effect." (list (completing-read "Select link to open: " (luwak-collect-links) nil t))) (luwak-open (cadr (split-string link)))) -(defun luwak-start-process-with-torsocks (no-tor name buffer program &rest program-args) - (if no-tor - (apply 'start-process (append (list name buffer program) program-args)) - (apply 'start-process - (append (list name buffer "torsocks" program) program-args)))) +(defun luwak-start-process-with-torsocks (no-tor name buffer &rest cmd) + (apply #'start-process name buffer + (if no-tor cmd `("torsocks" ,@cmd)))) (defun luwak-save-dump (file-name) "Write dump of the current luwak buffer to FILE-NAME."