From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Adam Porter Newsgroups: gmane.emacs.devel Subject: Re: [ELPA/elpa-admin] Render README.org as HTML with ox-html Date: Tue, 07 Sep 2021 05:26:31 -0500 Message-ID: <87ilzc65rc.fsf@alphapapa.net> References: <87h7f7zww5.fsf@alphapapa.net> <87zgszy91h.fsf_-_@alphapapa.net> <87a6ku2z8x.fsf@alphapapa.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24494"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Sep 07 12:28:20 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mNYKy-00068B-Qr for ged-emacs-devel@m.gmane-mx.org; Tue, 07 Sep 2021 12:28:20 +0200 Original-Received: from localhost ([::1]:34958 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mNYKx-0002Nf-Rr for ged-emacs-devel@m.gmane-mx.org; Tue, 07 Sep 2021 06:28:19 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:42202) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mNYJQ-0000yI-Nb for emacs-devel@gnu.org; Tue, 07 Sep 2021 06:26:44 -0400 Original-Received: from ciao.gmane.io ([116.202.254.214]:36084) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mNYJO-0006YO-D2 for emacs-devel@gnu.org; Tue, 07 Sep 2021 06:26:44 -0400 Original-Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1mNYJM-0003mo-FN for emacs-devel@gnu.org; Tue, 07 Sep 2021 12:26:40 +0200 X-Injected-Via-Gmane: http://gmane.org/ Received-SPF: pass client-ip=116.202.254.214; envelope-from=ged-emacs-devel@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:274223 Archived-At: --=-=-= Content-Type: text/plain Stefan Monnier writes: >> After some more thinking, the ASCII-only patch seemed redundant, so here >> is a new, simpler patch that also accounts for a few other details I had >> overlooked. It exports Org readmes to HTML in the ELPA Web page and to >> plain-text for the PACKAGE-readme.txt file. > > Sorry for not getting back to you sooner. > I still haven't been able to devote the attention that this deserves, > but here's some comments for now: No problem. Thanks. >> +(cl-defun elpaa--export-org (file backend &key body-only ext-plist) >> + "Return Org FILE as an exported string. >> +BACKEND and EXT-PLIST are passed to `org-export-as', which see. >> +Uses `elpaa--call-sandboxed', since exporting with Org may run >> +arbitrary code." >> + (declare (indent defun)) >> + (cl-check-type backend symbol) >> + (cl-assert (memq body-only '(nil t)) t >> + "BODY-ONLY may only be nil or t") >> + (with-temp-buffer >> + (unless (zerop (elpaa--call-sandboxed >> + t "emacs" "--batch" "-l" (format "ox-%s" backend) >> + file >> + "--eval" (format "(message \"%%s\" (org-export-as '%s nil nil %S '%S))" >> + backend body-only ext-plist))) >> + (error "Unable to export Org file: %S" file)) >> + (buffer-string))) > > `elpaa--call-sandboxed` doesn't return the exit status of the process. Oops, I wrongly assumed that it was just like `elpaa--call', but sandboxed. :) I removed the check for the exit code, so `elpaa--call-sandboxed' can signal an error when it checks the code itself. > `emacs --batch` will load the site-init files which may emit messages, > and those will "pollute" your temp buffer. > > It's not hypothetical, because the Emacs that's in Debian does exactly > that, so the Emacs on elpa.gnu.org (which is arguably the most important > Emacs for this code) emits such warnings (that's how I know ;-). > So you probably want to pass the result of `org-export-as` to > `write-region` to save it explicitly to some file of your choice. Apologies for not noticing that. I've changed it to write to and read from a temp file. > I'd also use `%S` rather than `%s` for `backend` (basically I follow > the rule that we should always use `%S` except for those cases where the > arg is a string and we want to plug its contents). Done. > [ Also, I haven't yet tried the code on elpa.gnu.org but that's still > running Emacs-26, so there might be compatibility issues for Org. > It should be upgraded to Emacs-27 "real soon now", tho, so maybe it's > not worth the trouble. ] Sometimes Org changes can be annoying to compensate for. Hopefully the org-export infrastructure won't change too much. I tested this on Emacs 26.3 and it seems to work. >> @@ -1313,11 +1326,33 @@ return section under HEADER in package's main file." >> (insert (format "

To install this package, run in Emacs:

>>
M-x package-install RET %s RET
" >> name)) >> - (let ((rm (elpaa--get-README pkg-spec srcdir))) >> - (when rm >> - (write-region rm nil (concat name "-readme.txt")) >> - (insert "

Full description

\n" (elpaa--html-quote rm)
>> -                  "\n
\n"))) >> + (let ((readme-file-name >> + (elpaa--spec-get pkg-spec :readme >> + '("README" "README.rst" >> + ;; Most README.md files seem to be currently >> + ;; worse than the Commentary: section :-( >> + ;; "README.md" >> + "README.org"))) >> + (output-filename (concat name "-readme.txt")) >> + readme-content page-content) >> + (pcase readme-file-name >> + ("README.org" >> + (setf readme-content (elpaa--export-org readme-file-name 'ascii >> + :ext-plist (append '(:ascii-charset utf-8) >> + elpaa--org-export-options)) >> + page-content (elpaa--export-org readme-file-name 'html >> + :body-only t :ext-plist elpaa--org-export-options))) >> + (_ ;; Non-Org readme. >> + (setf readme-content (elpaa--get-section >> + "Commentary" readme-file-name >> + dir pkg-spec) > > The byte-compiler tells me `dir` is an unknown variable. Oops, fixed. >> + page-content (when readme-content >> + (concat "
\n"
>> +                                        (elpaa--html-quote readme-content)
>> +                                        "\n
\n"))))) >> + (when readme-content >> + (write-region readme-content nil output-filename) >> + (insert "

Full description

\n" page-content))) >> ;; (message "latest=%S; files=%S" latest files) >> (unless (< (length files) (if (zerop (length latest)) 1 2)) >> (insert (format "

Old versions

\n")) > > Hmm... this function is already too long, so please move this to > a separate function. I don't disagree, but I'm not sure what you have in mind. Since this code also writes to the temp buffer made by the containing function, `elpaa--html-make-pkg', it would seem awkward to me to move that code as-is to another function that would both write to a file and insert into the current buffer, but maybe that's not what you meant, or maybe I just haven't learned your preferred style. :) > After tweaking your code a bit to get it to work (see patch below), I'm > still not getting the HTML I expect (the "HTML" doc only contains > Debian Emacs's extraneous init messages but not the actual HTML from > org-export-as, even though running the Emacs command by hand does > output it, I haven't had time to dig deeper into the problem). I guess you forgot to attach the patch, but maybe the one I'm attaching fixes that problem, anyway. :) > One other thing. I think it would make sense to make > `elpaa--get-section` return a pair of the string content and some "type" > (org, markdown, plaintext, ...) so as to automatically use your HTML > renderer for any .org file we find instead of only when the org file is > specified explicitly via `:readme "README.org"`. WDYT? I'm not sure what you have in mind. I felt confused by that function's signature and it's "overloading", shall we say, in that it may return a file's contents or a section's contents. In the patched version of `elpaa--html-make-pkg', I tried to label the logic to make it easier for me to follow. Anyway, here's an updated patch that I hope will basically work correctly on your end. --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-elpa-admin.el-Export-Org-readmes-to-ASCII-and-HTML.patch >From 05ccce75900f42d28aaf8dff241e22d146918bbf Mon Sep 17 00:00:00 2001 From: Adam Porter Date: Sun, 29 Aug 2021 17:45:22 -0500 Subject: [PATCH] * elpa-admin.el: Export Org readmes to ASCII and HTML (elpaa--export-org): New function. (elpaa--org-export-options): New variable. (elpaa--html-make-pkg): Export Org readmes to HTML and plain-text, and other readme formats to plain-text. (elpaa--get-README): Remove function. --- elpa-admin.el | 70 +++++++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 56 insertions(+), 14 deletions(-) diff --git a/elpa-admin.el b/elpa-admin.el index ac72f2f..c38d69a 100644 --- a/elpa-admin.el +++ b/elpa-admin.el @@ -69,6 +69,11 @@ to be installed and has only been tested on some Debian systems.") (defvar elpaa--debug nil) +(defvar elpaa--org-export-options + '(:with-author nil :with-creator nil :with-broken-links t) + "Options used common to all Org export backends. +See variable `org-export-options-alist'.") + (unless (fboundp 'ignore-error) (defmacro ignore-error (condition &rest body) `(condition-case nil (progn ,@body) (,condition nil)))) @@ -1209,15 +1214,30 @@ return section under HEADER in package's main file." (delete-region (point) (point-max)) (buffer-string))))))) -(defun elpaa--get-README (pkg-spec dir) - (elpaa--get-section - "Commentary" (elpaa--spec-get pkg-spec :readme - '("README" "README.rst" - ;; Most README.md files seem to be currently - ;; worse than the Commentary: section :-( - ;; "README.md" - "README.org")) - dir pkg-spec)) +(cl-defun elpaa--export-org (file backend &key body-only ext-plist) + "Return Org FILE as an exported string. +BACKEND and EXT-PLIST are passed to `org-export-as', which see. +Uses `elpaa--call-sandboxed', since exporting with Org may run +arbitrary code." + (declare (indent defun)) + (cl-check-type backend symbol) + (cl-assert (memq body-only '(nil t)) t + "BODY-ONLY may only be nil or t") + ;; "emacs --batch" loads site-init files, which may pollute output, + ;; so we write it to a temp file. + (let ((output-filename (make-temp-file "elpaa--export-org-"))) + (unwind-protect + (progn + (with-temp-buffer + (elpaa--call-sandboxed + t "emacs" "--batch" "-l" (format "ox-%S" backend) + file + "--eval" (format "(write-region (org-export-as '%s nil nil %S '%S) nil %S)" + backend body-only ext-plist output-filename))) + (with-temp-buffer + (insert-file-contents output-filename) + (buffer-string))) + (delete-file output-filename)))) (defun elpaa--get-NEWS (pkg-spec dir) (let ((text @@ -1313,11 +1333,33 @@ return section under HEADER in package's main file." (insert (format "

To install this package, run in Emacs:

M-x package-install RET %s RET
" name)) - (let ((rm (elpaa--get-README pkg-spec srcdir))) - (when rm - (write-region rm nil (concat name "-readme.txt")) - (insert "

Full description

\n" (elpaa--html-quote rm)
-                  "\n
\n"))) + (let ((readme-file-name + (elpaa--spec-get pkg-spec :readme + '("README" "README.rst" + ;; Most README.md files seem to be currently + ;; worse than the Commentary: section :-( + ;; "README.md" + "README.org"))) + (output-filename (concat name "-readme.txt")) + readme-content page-content) + (pcase readme-file-name + ("README.org" + (setf readme-content (elpaa--export-org readme-file-name 'ascii + :ext-plist (append '(:ascii-charset utf-8) + elpaa--org-export-options)) + page-content (elpaa--export-org readme-file-name 'html + :body-only t :ext-plist elpaa--org-export-options))) + (_ ;; Non-Org readme. + (setf readme-content (elpaa--get-section + "Commentary" readme-file-name + srcdir pkg-spec) + page-content (when readme-content + (concat "
\n"
+                                        (elpaa--html-quote readme-content)
+                                        "\n
\n"))))) + (when readme-content + (write-region readme-content nil output-filename) + (insert "

Full description

\n" page-content))) ;; (message "latest=%S; files=%S" latest files) (unless (< (length files) (if (zerop (length latest)) 1 2)) (insert (format "

Old versions

\n")) -- 2.7.4 --=-=-=--