From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id +JT6BLLkoGYIzwAA62LTzQ:P1 (envelope-from ) for ; Wed, 24 Jul 2024 11:25:38 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id +JT6BLLkoGYIzwAA62LTzQ (envelope-from ) for ; Wed, 24 Jul 2024 13:25:38 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=hfmdk-frankfurt.de (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1721820338; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=SiMNahj69jnkRmqBAy5IkP7MBa+C0Fwyj+cn+2v25HQ=; b=gYu/vZ5yiNURZO2JwdDClhXU0DUj3zFCxDrng+VwSprvbC07IRtYE0bjvuzBH+Ktzppa9/ gKl8ExufhFDc7Nt4pviMOLCRkQ3A/cnp9xCqlyrWx00+C3pDIW2L6Yp+y4Y6gS8IY3yPBi 0jxRUbTsWMhFZBeAjN4zId/TppRTHzAfECowDyWSFhDrpKElOZQvBMfHm8ej6IERzMjKzA b1j8evJjy/B6C1HhybPjcSXptbWKI/Frl9r9S5oxqXvFVtHiDAw/108x1nLhJw5Tf7Qnne VyZh54cqJd5orlZ4i5+jEjtpx4onScaw6v83jrmN2P87mTg9ip8ZSox+oVNY7Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=hfmdk-frankfurt.de (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1721820338; a=rsa-sha256; cv=none; b=mZ+fr1RQ6IXG5cIAoXEmc1BIGTnBSZnkpRzGonHWecbwFzbUfk7Hev/RG0vW1+HluJVlZP 9c1b3yfGqHrfWGse/ZpBJ40cyFxSUNxPn8wWk78BtXbasMsF2KSMRNN4k/HFgASFQkf8RZ RDpfvtY0On7/7bBTdKdpIF+m48KZl51GDtqnik52t4iFCeAGi2JiIt7k185i1/ZewhJBQM IqEZvj9BGxL/5NAxctPawLT7cE4g1AdpJ6RmLqhGlKsHUN+iDfgENW3Ot5pFJ895i2Xttj neDo2SZpSHBGW7MnzXjEb6zU/QkaC/iw3FWCu2/MFv7BZlQwPv6zWyTSMcm2fg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 62437DF6C for ; Wed, 24 Jul 2024 13:25:37 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sWa6r-0005Ya-Kk; Wed, 24 Jul 2024 07:24:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sWa6q-0005YI-2o for emacs-orgmode@gnu.org; Wed, 24 Jul 2024 07:24:40 -0400 Received: from www.selma.hfmdk-frankfurt.de ([46.4.92.145] helo=mail.selma.hfmdk-frankfurt.de) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sWa6m-0007hj-M5 for emacs-orgmode@gnu.org; Wed, 24 Jul 2024 07:24:39 -0400 Received: by mail.selma.hfmdk-frankfurt.de (Postfix, from userid 113) id 85DE2F60D3B; Wed, 24 Jul 2024 13:24:32 +0200 (CEST) Received: from selma.hfmdk-frankfurt.de (ip-037-201-128-004.um10.pools.vodafone-ip.de [37.201.128.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (Client did not present a certificate) by mail.selma.hfmdk-frankfurt.de (Postfix) with ESMTPSA id 47379F60D3B; Wed, 24 Jul 2024 13:24:30 +0200 (CEST) Received: by selma.hfmdk-frankfurt.de (Postfix, from userid 1000) id AAD633960515; Wed, 24 Jul 2024 13:24:29 +0200 (CEST) Date: Wed, 24 Jul 2024 13:24:29 +0200 From: Orm Finnendahl To: Ihor Radchenko Cc: Org mailing list Subject: Re: multipage html output Message-ID: Mail-Followup-To: Ihor Radchenko , Org mailing list References: <87wmlp38gr.fsf@localhost> <874j8gz9qh.fsf@localhost> <87bk2o2o2m.fsf@localhost> <87sew011c6.fsf@localhost> <87frrzdrbz.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <87frrzdrbz.fsf@localhost> X-Disclaimer: Why are you listening to me? X-Operating-System: GNU/Linux Organization: Hochschule =?utf-8?B?ZsO8?= =?utf-8?Q?r?= Musik und Darstellende Kunst Frankfurt, Frankfurt, Germany Received-SPF: pass client-ip=46.4.92.145; envelope-from=orm.finnendahl@selma.hfmdk-frankfurt.de; helo=mail.selma.hfmdk-frankfurt.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -0.92 X-Migadu-Queue-Id: 62437DF6C X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -0.92 X-TUID: ZAImmcmr7eDv Hi Ihor, thanks a lot for the patches and explanations. Your assessment makes sense and I agree that we should stick to the current design as much as possible. I will go ahead and adapt my ox-html.el code the way you propose and use your patched ox.el for this. It shouldn't take too much effort and I will get back as soon as I have results (or questions ;-). -- Orm Am Mittwoch, den 24. Juli 2024 um 10:20:16 Uhr (+0000) schrieb Ihor Radchenko: > Orm Finnendahl writes: > > > To recapitulate: In my code, org-export-as calls process-multipage in > > the backend. This function: > > > > - collects and adds information necessary for org-multipage to do its > > job, splitting the document into different parts, etc. and > > > > - then calls org-export-data on the subtrees and exports each returned > > string to an individual file. > > > > - It finally issues a done string and executes a browser open/visit > > file or simply exits nil. > > Currently, org-export-as does the following: > > 1. Compute global export attributes, according to the selected export backend > 2. Copy original buffer into working copy > 3. Process and parse the copy, generating AST > 4. Do the actual export > > You plugged your multipage processing into (4), but what it actually > does involves (3), (4), and also a new kind of post-processing. > I do not think that it is a good design from the point of view of ox.el. > I prefer to reuse or extend the existing mechanisms if at all possible - > this makes new features less confusing for users and backend developers. > > > - collects and adds information necessary for org-multipage to do its > > job, splitting the document into different parts, etc. and > > What you describe here is more or less what :filter-parse-tree filters > do - they can rearrange the parse tree before passing it to the > transcoders. Why not reusing it for multipage export? > > > - then calls org-export-data on the subtrees and exports each returned > > string to an individual file. > > And you simply call `org-export-transcode-page' for this, followed by > writing the returned string to file. > > The first part can fit within `org-export-as', but writing to file is > going a step beyond, duplicating what `org-export-to-file' does. > > > - It finally issues a done string and executes a browser open/visit > > file or simply exits nil. > > ... which again steps beyond `org-export-as' scope - post-processing is > currently done as a part of `org-export-to-file'/`org-export-to-buffer'. > > ---- > > Let me propose the following changes to ox.el: > > 1. org-data will be transcoded using `org-export-transcode-org-data', > which can be overridden by setting org-data transcoders in the > individual backends. > > 2. org-export-as will understand transcoded output to be a list of > strings and will transfer INFO plist as text property in the return > values > > 3. org-export-to-file will make use of the text properties to retrieve > the file name to write. This way, export backend itself can assign > the file names where each exporter string should go. > > I believe that my changes should allow you to implement multipage export > in the following way: > > 1. You can use :filter-parse-tree in ox-html backend to replace the > original (org-data ...) AST with a list of > ((org-page ...) (org-page ...) ...) pseudo-elements and populate INFO > channel with auxiliary information you now compute in `org-html-process-multipage' > > 2. You can define org-page transcoder to render individual pages as > needed > > 3. You can assign :output-file text property to the returned org-page > strings and use org-export-to-file to generate the multipage output > on disk > > 4. You can handle opening exported files by augmenting POST-PROCESS > argument in `org-html-export-to-multipage-html' and calling > `org-export-file' instead of `org-export-as'. > > The tentative patches (against Org mode main branch) implementing my > changes are attached. > > From 540c8ef21c26df79cf48f58afb4e88130985e2f7 Mon Sep 17 00:00:00 2001 > Message-ID: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> > From: Ihor Radchenko > Date: Wed, 24 Jul 2024 11:40:57 +0200 > Subject: [PATCH 1/3] ox: Factor out org-data transcoding into dedicated > overrideable transcoder > > * lisp/ox.el (org-export-transcode-org-data): New function serving as > the default transcoder for org-data export. > (org-export-transcoder): Use `org-export-transcode-org-data' when no > org-data transcoder is defined. > (org-export-as): Rely upon org-data transcoder to do its job. > --- > lisp/ox.el | 55 +++++++++++++++++++++++++++++------------------------- > 1 file changed, 30 insertions(+), 25 deletions(-) > > diff --git a/lisp/ox.el b/lisp/ox.el > index fbd9bb0df..bdee71082 100644 > --- a/lisp/ox.el > +++ b/lisp/ox.el > @@ -1883,9 +1883,11 @@ (defun org-export-transcoder (blob info) > INFO is a plist containing export directives." > (let ((type (org-element-type blob))) > ;; Return contents only for complete parse trees. > - (if (eq type 'org-data) (lambda (_datum contents _info) contents) > - (let ((transcoder (cdr (assq type (plist-get info :translate-alist))))) > - (and (functionp transcoder) transcoder))))) > + (let ((transcoder (cdr (assq type (plist-get info :translate-alist))))) > + (cond > + ((functionp transcoder) transcoder) > + ;; Use default org-data transcoder unless specified. > + ((eq type 'org-data) #'org-export-transcode-org-data))))) > > (defun org-export--keep-spaces (data info) > "Non-nil, when post-blank spaces after removing DATA should be preserved. > @@ -3004,31 +3006,34 @@ (defun org-export-as > backend info subtreep visible-only ext-plist)) > ;; Eventually transcode TREE. Wrap the resulting string into > ;; a template. > - (let* ((body (org-element-normalize-string > - (or (org-export-data (plist-get info :parse-tree) info) > - ""))) > - (inner-template (cdr (assq 'inner-template > - (plist-get info :translate-alist)))) > - (full-body (org-export-filter-apply-functions > - (plist-get info :filter-body) > - (if (not (functionp inner-template)) body > - (funcall inner-template body info)) > - info)) > - (template (cdr (assq 'template > - (plist-get info :translate-alist)))) > - (output > - (if (or (not (functionp template)) body-only) full-body > - (funcall template full-body info)))) > + (let ((output > + (or (org-export-data (plist-get info :parse-tree) info) > + ""))) > ;; Call citation export finalizer. > (when (plist-get info :with-cite-processors) > (setq output (org-cite-finalize-export output info))) > - ;; Remove all text properties since they cannot be > - ;; retrieved from an external process. Finally call > - ;; final-output filter and return result. > - (org-no-properties > - (org-export-filter-apply-functions > - (plist-get info :filter-final-output) > - output info))))))))) > + (let ((filters (plist-get info :filter-final-output))) > + ;; Remove all text properties since they cannot be > + ;; retrieved from an external process. Finally call > + ;; final-output filter and return result. > + (org-no-properties > + (org-export-filter-apply-functions filters output info)))))))))) > + > +(defun org-export-transcode-org-data (_ body info) > + "Transcode `org-data' node with BODY. Return transcoded string. > +INFO is the communication channel plist." > + (let* ((inner-template (cdr (assq 'inner-template > + (plist-get info :translate-alist)))) > + (full-body (org-export-filter-apply-functions > + (plist-get info :filter-body) > + (if (not (functionp inner-template)) body > + (funcall inner-template body info)) > + info)) > + (template (cdr (assq 'template > + (plist-get info :translate-alist)))) > + (body-only (memq 'body-only (plist-get info :export-options)))) > + (if (or (not (functionp template)) body-only) full-body > + (funcall template full-body info)))) > > (defun org-export--annotate-info (backend info &optional subtreep visible-only ext-plist) > "Annotate the INFO plist according to the BACKEND. > -- > 2.45.2 > > From 1b0b331f92abc1ca7e04f71fe7ff60da57c719b8 Mon Sep 17 00:00:00 2001 > Message-ID: <1b0b331f92abc1ca7e04f71fe7ff60da57c719b8.1721815865.git.yantar92@posteo.net> > In-Reply-To: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> > References: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> > From: Ihor Radchenko > Date: Wed, 24 Jul 2024 11:51:21 +0200 > Subject: [PATCH 2/3] org-export-as: Allow the return value to be a list of > strings; add INFO > > * lisp/ox.el (org-export-as): Allow the transcoders to return list of > strings and return it. When returning a string, put INFO plist as > text property. Do not remove text properties assigned by the > transcoders. > (org-export-data): Document that list of strings may be returned. > --- > lisp/ox.el | 28 ++++++++++++++++++++-------- > 1 file changed, 20 insertions(+), 8 deletions(-) > > diff --git a/lisp/ox.el b/lisp/ox.el > index bdee71082..a76b3b353 100644 > --- a/lisp/ox.el > +++ b/lisp/ox.el > @@ -1930,7 +1930,7 @@ (defun org-export-data (data info) > > The `:filter-parse-tree' filters are not applied. > > -Return a string." > +Return a string or a list of strings." > (or (gethash data (plist-get info :exported-data)) > ;; Handle broken links according to > ;; `org-export-with-broken-links'. > @@ -2969,7 +2969,9 @@ (defun org-export-as > with external parameters overriding Org default settings, but > still inferior to file-local settings. > > -Return code as a string." > +Return code as a string or a list of strings. > +The returned strings will have their `org-export-info' property set to > +export information channel." > (when (symbolp backend) (setq backend (org-export-get-backend backend))) > (org-export-barf-if-invalid-backend backend) > (org-fold-core-ignore-modifications > @@ -3009,15 +3011,25 @@ (defun org-export-as > (let ((output > (or (org-export-data (plist-get info :parse-tree) info) > ""))) > + (setq output (ensure-list output)) > ;; Call citation export finalizer. > (when (plist-get info :with-cite-processors) > - (setq output (org-cite-finalize-export output info))) > + (setq output > + (mapcar > + (lambda (o) (org-cite-finalize-export o info)) > + output))) > (let ((filters (plist-get info :filter-final-output))) > - ;; Remove all text properties since they cannot be > - ;; retrieved from an external process. Finally call > - ;; final-output filter and return result. > - (org-no-properties > - (org-export-filter-apply-functions filters output info)))))))))) > + ;; Call final-output filter and return result. > + (setq output > + (mapcar > + (lambda (o) (org-export-filter-apply-functions filters o info)) > + output))) > + ;; Apply org-export-info property. > + (setq output > + (mapcar > + (lambda (o) (org-add-props o nil 'org-export-info info)) > + output)) > + (if (length= output 1) (car output) output)))))))) > > (defun org-export-transcode-org-data (_ body info) > "Transcode `org-data' node with BODY. Return transcoded string. > -- > 2.45.2 > > From 6fa2efadd229a667fba1b18aecc9d1ead5f284ac Mon Sep 17 00:00:00 2001 > Message-ID: <6fa2efadd229a667fba1b18aecc9d1ead5f284ac.1721815865.git.yantar92@posteo.net> > In-Reply-To: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> > References: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> > From: Ihor Radchenko > Date: Wed, 24 Jul 2024 12:09:36 +0200 > Subject: [PATCH 3/3] org-export-to-file: Derive file name to write from export > output > > * lisp/ox.el (org-export--write-output): New helper function > performing writing an export output or a list of outputs to file. It > derives the file name from :output-file property in the output string > or INFO plist stored in the output string. > (org-export-to-file): Handle export output being a list of strings. > Use `org-export--write-output'. > --- > lisp/ox.el | 61 ++++++++++++++++++++++++++++++++++-------------------- > 1 file changed, 38 insertions(+), 23 deletions(-) > > diff --git a/lisp/ox.el b/lisp/ox.el > index a76b3b353..d78c04998 100644 > --- a/lisp/ox.el > +++ b/lisp/ox.el > @@ -6830,6 +6830,31 @@ (defun org-latex-export-as-latex > (switch-to-buffer-other-window buffer)) > buffer))) > > +(defun org-export--write-output (output encoding) > + "Write OUTPUT to file with ENCODING. > +OUTPUT may be a string or a list of strings. > +The target file is retrieved from :output-file OUTPUT property or > +:output-file property in plist stored in `org-export-info' property of > +each string. > + > +Return the file name or a list of file names." > + (if (listp output) (mapcar #'org-export--write-output output) > + (let ((file (or > + (get-text-property 0 :output-file output) > + (plist-get > + (get-text-property 0 'org-export-info output) > + :output-file)))) > + (with-temp-buffer > + (insert output) > + ;; Ensure final newline. This is what was done > + ;; historically, when we used `write-file'. > + ;; Note that adding a newline is only safe for > + ;; non-binary data. > + (unless (bolp) (insert "\n")) > + (let ((coding-system-for-write encoding)) > + (write-region nil nil file)) > + file)))) > + > ;;;###autoload > (defun org-export-to-file > (backend file &optional async subtreep visible-only body-only ext-plist > @@ -6878,33 +6903,23 @@ (defun org-latex-export-to-latex > `(let ((output > (org-export-as > ',backend ,subtreep ,visible-only ,body-only > - ',ext-plist))) > - (with-temp-buffer > - (insert output) > - ;; Ensure final newline. This is what was done > - ;; historically, when we used `write-file'. > - ;; Note that adding a newline is only safe for > - ;; non-binary data. > - (unless (bolp) (insert "\n")) > - (let ((coding-system-for-write ',encoding)) > - (write-region nil nil ,file))) > - (or (ignore-errors (funcall ',post-process ,file)) ,file))) > + ',ext-plist)) > + file) > + (setq file (org-export--write-output output ',encoding)) > + (let ((post (lambda (f) (or (ignore-errors (funcall ',post-process f)) f)))) > + (if (listp file) (mapcar post file) (funcall post file))))) > (let ((output (org-export-as > - backend subtreep visible-only body-only ext-plist))) > - (with-temp-buffer > - (insert output) > - ;; Ensure final newline. This is what was done > - ;; historically, when we used `write-file'. > - ;; Note that adding a newline is only safe for > - ;; non-binary data. > - (unless (bolp) (insert "\n")) > - (let ((coding-system-for-write encoding)) > - (write-region nil nil file))) > + backend subtreep visible-only body-only ext-plist)) > + file) > + (setq file (org-export--write-output output encoding)) > (when (and (org-export--copy-to-kill-ring-p) (org-string-nw-p output)) > (org-kill-new output)) > ;; Get proper return value. > - (or (and (functionp post-process) (funcall post-process file)) > - file)))))) > + (let ((post (lambda (f) > + (or (and (functionp post-process) > + (funcall post-process f)) > + f)))) > + (if (listp file) (mapcar post file) (funcall post file)))))))) > > (defun org-export-output-file-name (extension &optional subtreep pub-dir) > "Return output file's name according to buffer specifications. > -- > 2.45.2 > > > -- > Ihor Radchenko // yantar92, > Org mode contributor, > Learn more about Org mode at . > Support Org development at , > or support my work at