Multiple bibliography files with ox-bibtex and html export

Multiple bibliography files with ox-bibtex and html export

[Thibault Marin]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]


Hi list,

I would like to use ox-bibtex to export a bibliography to html with
multiple bibliography files, as follows:

#+BIBLIOGRAPHY: bibtex_1.bib,bibtex_2.bib plain option:-d option:-noabstract limit:t

This works with latex export but not with html (I get a "Executing
bibtex2html failed").  It appears that bibtex2html can only process a
single file.

I am attaching a patch which allows me to use multiple files with html
export.  It creates a combined bibliography file and call bibtex2html on
it.  I am not sure this is the best way to address this, so any
suggestion would be welcome.

Does this look like something that could be merged?

Thanks,
thibault




[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-contrib-lisp-ox-bibtex.el-org-bibtex-process-bib-fil.patch --]
[-- Type: text/x-diff, Size: 2053 bytes --]

From c8a97a2d79d349a5d7c55ce052daa0794bde49ad Mon Sep 17 00:00:00 2001
From: thibault <thibault.marin@gmx.com>
Date: Tue, 23 Aug 2016 22:57:19 -0500
Subject: [PATCH] * contrib/lisp/ox-bibtex.el (org-bibtex-process-bib-files):
 Add support for multiple bibliography files with html export.

Combine comma-separated bibliography files into a single one and process it
using bibtex2html.  This matches the behavior already present for latex export.
---
 contrib/lisp/ox-bibtex.el | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/contrib/lisp/ox-bibtex.el b/contrib/lisp/ox-bibtex.el
index 56dec38..7e266ab 100644
--- a/contrib/lisp/ox-bibtex.el
+++ b/contrib/lisp/ox-bibtex.el
@@ -191,13 +191,34 @@ Return new parse tree."
 		(file (org-bibtex-get-file keyword))
 		temp-file
 		out-file)
+	    (let ((files (org-split-string file ",")))
+	      (when (< 1 (length files))
+		(let ((combined-bib-file
+		       (concat
+			(file-name-sans-extension
+			 (file-name-nondirectory
+			  (buffer-file-name))) "-combined.bib")))
+		  (with-temp-file combined-bib-file
+		    (dolist (bib files)
+		      (insert-file-contents
+		       (if (equal (file-name-extension bib) "bib")
+			   bib
+			 (concat bib ".bib")
+			 )
+		       )
+		      )
+		    )
+		  (setq file combined-bib-file)
+		  )
+		)
+	      )
 	    ;; Test if filename is given with .bib-extension and strip
     	    ;; it off. Filenames with another extensions will be
 	    ;; untouched and will finally rise an error in bibtex2html.
 	    (setq file (if (equal (file-name-extension file) "bib")
 			   (file-name-sans-extension file) file))
-	    ;; Outpufiles of bibtex2html will be put into current working directory
-	    ;; so define a variable for this.
+	    ;; Output files of bibtex2html will be put into current
+	    ;; working directory so define a variable for this.
 	    (setq out-file (file-name-sans-extension
 			    (file-name-nondirectory file)))
 	    ;; limit is set: collect citations throughout the document
-- 
2.8.1

Re: Multiple bibliography files with ox-bibtex and html export

[Nicolas Goaziou]

Hello,

Thibault Marin <thibault.marin@gmx.com> writes:

> I would like to use ox-bibtex to export a bibliography to html with
> multiple bibliography files, as follows:
>
> #+BIBLIOGRAPHY: bibtex_1.bib,bibtex_2.bib plain option:-d option:-noabstract limit:t
>
> This works with latex export but not with html (I get a "Executing
> bibtex2html failed").  It appears that bibtex2html can only process a
> single file.
>
> I am attaching a patch which allows me to use multiple files with html
> export.  It creates a combined bibliography file and call bibtex2html on
> it.  I am not sure this is the best way to address this, so any
> suggestion would be welcome.
>
> Does this look like something that could be merged?

Apparently no objection was raised, so I think this can be merged. Some
minor comments below.

> +	    (let ((files (org-split-string file ",")))

I think `split-string' is sufficient here.

> +	      (when (< 1 (length files))
> +		(let ((combined-bib-file
> +		       (concat
> +			(file-name-sans-extension
> +			 (file-name-nondirectory
> +			  (buffer-file-name))) "-combined.bib")))
> +		  (with-temp-file combined-bib-file
> +		    (dolist (bib files)
> +		      (insert-file-contents
> +		       (if (equal (file-name-extension bib) "bib")
> +			   bib
> +			 (concat bib ".bib")
> +			 )
> +		       )
> +		      )
> +		    )
> +		  (setq file combined-bib-file)
> +		  )
> +		)
> +	      )

There should be no dangling parenthesis in Lisp.

Could you send an updated patch?

Thank you.


Regards,

-- 
Nicolas Goaziou

Re: Multiple bibliography files with ox-bibtex and html export

[Thibault Marin]

[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]


Nicolas Goaziou writes:

> Hello,
>
> Thibault Marin <thibault.marin@gmx.com> writes:
>
>> I would like to use ox-bibtex to export a bibliography to html with
>> multiple bibliography files, as follows:
>>
>> #+BIBLIOGRAPHY: bibtex_1.bib,bibtex_2.bib plain option:-d option:-noabstract limit:t
>>
>> This works with latex export but not with html (I get a "Executing
>> bibtex2html failed").  It appears that bibtex2html can only process a
>> single file.
>>
>> I am attaching a patch which allows me to use multiple files with html
>> export.  It creates a combined bibliography file and call bibtex2html on
>> it.  I am not sure this is the best way to address this, so any
>> suggestion would be welcome.
>>
>> Does this look like something that could be merged?
>
> Apparently no objection was raised, so I think this can be merged. Some
> minor comments below.
>
>> +	    (let ((files (org-split-string file ",")))
>
> I think `split-string' is sufficient here.
>
>> +	      (when (< 1 (length files))
>> +		(let ((combined-bib-file
>> +		       (concat
>> +			(file-name-sans-extension
>> +			 (file-name-nondirectory
>> +			  (buffer-file-name))) "-combined.bib")))
>> +		  (with-temp-file combined-bib-file
>> +		    (dolist (bib files)
>> +		      (insert-file-contents
>> +		       (if (equal (file-name-extension bib) "bib")
>> +			   bib
>> +			 (concat bib ".bib")
>> +			 )
>> +		       )
>> +		      )
>> +		    )
>> +		  (setq file combined-bib-file)
>> +		  )
>> +		)
>> +	      )
>
> There should be no dangling parenthesis in Lisp.
>
> Could you send an updated patch?
>
> Thank you.
>
>
> Regards,

Thanks for the review, here is an updated patch.

Best,

thibault

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-contrib-lisp-ox-bibtex.el-org-bibtex-process-bib-fil.patch --]
[-- Type: text/x-diff, Size: 1987 bytes --]

From cb07ff936587a456f1e6599d216efe9463431d3f Mon Sep 17 00:00:00 2001
From: thibault <thibault.marin@gmx.com>
Date: Tue, 6 Sep 2016 22:42:39 -0500
Subject: [PATCH] * contrib/lisp/ox-bibtex.el (org-bibtex-process-bib-files):
 Add support for multiple bibliography files with html export.

Combine comma-separated bibliography files into a single one and process
it using bibtex2html.  This matches the behavior already present for
latex export.
---
 contrib/lisp/ox-bibtex.el | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/contrib/lisp/ox-bibtex.el b/contrib/lisp/ox-bibtex.el
index 56dec38..b46cb76 100644
--- a/contrib/lisp/ox-bibtex.el
+++ b/contrib/lisp/ox-bibtex.el
@@ -191,13 +191,27 @@ Return new parse tree."
 		(file (org-bibtex-get-file keyword))
 		temp-file
 		out-file)
+	    (let ((files (split-string file ",")))
+	      (when (< 1 (length files))
+		(let ((combined-bib-file
+		       (concat
+			(file-name-sans-extension
+			 (file-name-nondirectory
+			  (buffer-file-name))) "-combined.bib")))
+		  (with-temp-file combined-bib-file
+		    (dolist (bib files)
+		      (insert-file-contents
+		       (if (equal (file-name-extension bib) "bib")
+			   bib
+			 (concat bib ".bib")))))
+		  (setq file combined-bib-file))))
 	    ;; Test if filename is given with .bib-extension and strip
     	    ;; it off. Filenames with another extensions will be
 	    ;; untouched and will finally rise an error in bibtex2html.
 	    (setq file (if (equal (file-name-extension file) "bib")
 			   (file-name-sans-extension file) file))
-	    ;; Outpufiles of bibtex2html will be put into current working directory
-	    ;; so define a variable for this.
+	    ;; Output files of bibtex2html will be put into current
+	    ;; working directory so define a variable for this.
 	    (setq out-file (file-name-sans-extension
 			    (file-name-nondirectory file)))
 	    ;; limit is set: collect citations throughout the document
-- 
2.8.1

Re: Multiple bibliography files with ox-bibtex and html export

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 418 bytes --]

On 2016-09-06 23:46, Thibault Marin wrote:
>>> I am attaching a patch which allows me to use multiple files with html
>>> export.  It creates a combined bibliography file and call bibtex2html on
>>> it.  I am not sure this is the best way to address this, so any
>>> suggestion would be welcome.

Sorry for the late comment.  bibtex2html can read from standard input; maybe that would be cleaner?

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple bibliography files with ox-bibtex and html export

[Thibault Marin]

[-- Attachment #1: Type: text/plain, Size: 1549 bytes --]


Clément Pit--Claudel writes:

> On 2016-09-06 23:46, Thibault Marin wrote:
>>>> I am attaching a patch which allows me to use multiple files with html
>>>> export.  It creates a combined bibliography file and call bibtex2html on
>>>> it.  I am not sure this is the best way to address this, so any
>>>> suggestion would be welcome.
>
> Sorry for the late comment.  bibtex2html can read from standard input; maybe that would be cleaner?
>
> Clément.

That may be a good idea, it would prevent potential name clashing with
the created bib file.  Currently, the function creates a
<name-of-org-file>-combined.bib file with the content of all
bibliography files, then bibtex2html creates
<name-of-org-file>-combined.html and
<name-of-org-file>-combined_bib.html.  Passing the contents via stdin
would skip the <name-of-org-file>-combined.bib.  We could achieve the
same by simply deleting <name-of-org-file>-combined.bib after calling
bibtex2html.  I personally don't mind leaving the .bib file after
processing, but if there is a consensus to limit the side effect, we can
do that.

As far as the implementation goes, I am not sure what is the best way to
get this to work with stdin.  In the attached patch (which does *not*
work) I tried to use `call-process-region' and dump the bibliography
files into a temporary buffer.  This complicates the code a little.
Alternatively, we could use the `INFILE' parameter from `call-process',
but it looks that this would require a file, so it would not change much
from the previous patch.

Any thoughts?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-contrib-lisp-ox-bibtex.el-org-bibtex-process-bib-fil.patch --]
[-- Type: text/x-diff, Size: 4049 bytes --]

From 85369923cdd7540467a615ca92cf486fd6d08708 Mon Sep 17 00:00:00 2001
From: thibault <thibault@dell-desktop.WORKGROUP>
Date: Thu, 8 Sep 2016 22:06:21 -0500
Subject: [PATCH 2/2] * contrib/lisp/ox-bibtex.el
 (org-bibtex-process-bib-files): (WIP)  Add support for multiple bibliography
 files with html export.

Pass multiple bibliography files (comma separated) to bibtex2html.
---
 contrib/lisp/ox-bibtex.el | 82 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 53 insertions(+), 29 deletions(-)

diff --git a/contrib/lisp/ox-bibtex.el b/contrib/lisp/ox-bibtex.el
index b46cb76..38c0957 100644
--- a/contrib/lisp/ox-bibtex.el
+++ b/contrib/lisp/ox-bibtex.el
@@ -187,24 +187,20 @@ Return new parse tree."
     (org-element-map tree 'keyword
       (lambda (keyword)
 	(when (equal (org-element-property :key keyword) "BIBLIOGRAPHY")
-	  (let ((arguments (org-bibtex-get-arguments keyword))
-		(file (org-bibtex-get-file keyword))
-		temp-file
-		out-file)
-	    (let ((files (split-string file ",")))
-	      (when (< 1 (length files))
-		(let ((combined-bib-file
-		       (concat
-			(file-name-sans-extension
-			 (file-name-nondirectory
-			  (buffer-file-name))) "-combined.bib")))
-		  (with-temp-file combined-bib-file
-		    (dolist (bib files)
-		      (insert-file-contents
-		       (if (equal (file-name-extension bib) "bib")
-			   bib
-			 (concat bib ".bib")))))
-		  (setq file combined-bib-file))))
+	  (let* ((arguments (org-bibtex-get-arguments keyword))
+		 (file (org-bibtex-get-file keyword))
+		 temp-file
+		 out-file
+		 (multiple-bib-files (split-string file ","))
+		 (multiple-bib-p (< 1 (length multiple-bib-files)))
+		 multiple-bib-file)
+	    (when multiple-bib-p
+	      (setq multiple-bib-file
+		    (concat
+		     (file-name-sans-extension
+		      (file-name-nondirectory
+		       (buffer-file-name))) "-combined.bib"))
+	      (setq file multiple-bib-file))
 	    ;; Test if filename is given with .bib-extension and strip
     	    ;; it off. Filenames with another extensions will be
 	    ;; untouched and will finally rise an error in bibtex2html.
@@ -231,17 +227,45 @@ Return new parse tree."
 				 (append (plist-get arguments :options)
 					 (list "-citefile" temp-file))))))
 	    ;; Call "bibtex2html" on specified file.
-	    (unless (eq 0 (apply
-			   'call-process
-			   (append '("bibtex2html" nil nil nil)
-				   '("-a" "-nodoc" "-noheader" "-nofooter")
-				   (let ((style
-					  (org-not-nil
-					   (org-bibtex-get-style keyword))))
-				     (and style (list "--style" style)))
-				   (plist-get arguments :options)
-				   (list (concat file ".bib")))))
-	      (error "Executing bibtex2html failed"))
+	    (let* ((bibtex2html-cmd '("bibtex2html" nil nil nil))
+		   (bibtex2html-args-default '("-a" "-nodoc" "-noheader"
+					       "-nofooter"))
+		   (bibtex2html-style
+		    (let ((style
+			   (org-not-nil
+			    (org-bibtex-get-style keyword))))
+		      (and style (list "--style" style))))
+		   (bibtex2html-opts (plist-get arguments :options)))
+	      (message "mf=%s file=%s" multiple-bib-file multiple-bib-files)
+	      (if multiple-bib-p
+		  (with-temp-buffer
+		    multiple-bib-file
+		    (dolist (bib multiple-bib-files)
+		      (insert-file-contents
+		       (if (equal (file-name-extension bib) "bib")
+			   bib
+			 (concat bib ".bib"))))
+		    (unless
+			(eq 0
+			    (apply
+			     'call-process-region
+			     (append `(,(point-min) ,(point-max))
+				     bibtex2html-cmd
+				     bibtex2html-args-default
+				     bibtex2html-style
+				     bibtex2html-opts
+				     `("-o" ,file)))))
+		      (error "Executing bibtex2html failed"))
+		(unless
+		    (eq 0
+			(apply
+			 'call-process
+			 (append bibtex2html-cmd
+				 bibtex2html-args-default
+				 bibtex2html-style
+				 bibtex2html-opts
+				 (list (concat file ".bib")))))
+		  (error "Executing bibtex2html failed"))))
 	    (and temp-file (delete-file temp-file))
 	    ;; Open produced HTML file, and collect Bibtex key names
 	    (with-temp-buffer
-- 
2.8.1

Re: Multiple bibliography files with ox-bibtex and html export

[Thibault Marin]

Do you mean:
1) Using `call-process' for cases where a single bibliography file is
passed and `process-send-string' when multiple files are used?
2) Using `process-send-string' regardless of the number of bibliography
files?  In this case, can we still unify the code between single and
multiple files?
3) Something else?

In my opinion 1) makes the code more error-prone and harder to
maintain. If there are other reasons to replace the existing behavior
(for single bibliography files) by `process-send-string', then 2) may
make sense, otherwise it sounds to me that it may not be worth it: the
existing code is apparently working fine for single files, I would feel
a little uncomfortable changing it based only on my test case,
especially since there isn't (as far as I know) a battery of tests for
it.

- Is having a temporary file unacceptable?

The first patch creates and keeps the combined bibliography around, but
this created file could easily be deleted if preferred.  If the problem
is just the extra file, the first patch can fix it and seems less
intrusive to me.

- Is the main concern performance?

I think that the main argument for using standard input may be to skip
the disk access required by the temporary file.  I do not know if the
potential savings for files of size around a few MB (or more?) justify
the more intrusive change in the code.  Maybe others would have a better
feel for this than I do.

Thanks for the comments on this.  Once a consensus is reached, I can
work towards an updated patch.

thibault

> I'd suggest starting the process and then using process-send-string.
> 
> Clément.
> 
> On 2016-09-08 23:55, Thibault Marin wrote:
> > 
> > Clément Pit--Claudel writes:
> > 
> >> On 2016-09-06 23:46, Thibault Marin wrote:
> >>>>> I am attaching a patch which allows me to use multiple files with html
> >>>>> export.  It creates a combined bibliography file and call bibtex2html on
> >>>>> it.  I am not sure this is the best way to address this, so any
> >>>>> suggestion would be welcome.
> >>
> >> Sorry for the late comment.  bibtex2html can read from standard input; maybe that would be cleaner?
> >>
> >> Clément.
> > 
> > That may be a good idea, it would prevent potential name clashing with
> > the created bib file.  Currently, the function creates a
> > <name-of-org-file>-combined.bib file with the content of all
> > bibliography files, then bibtex2html creates
> > <name-of-org-file>-combined.html and
> > <name-of-org-file>-combined_bib.html.  Passing the contents via stdin
> > would skip the <name-of-org-file>-combined.bib.  We could achieve the
> > same by simply deleting <name-of-org-file>-combined.bib after calling
> > bibtex2html.  I personally don't mind leaving the .bib file after
> > processing, but if there is a consensus to limit the side effect, we can
> > do that.
> > 
> > As far as the implementation goes, I am not sure what is the best way to
> > get this to work with stdin.  In the attached patch (which does *not*
> > work) I tried to use `call-process-region' and dump the bibliography
> > files into a temporary buffer.  This complicates the code a little.
> > Alternatively, we could use the `INFILE' parameter from `call-process',
> > but it looks that this would require a file, so it would not change much
> > from the previous patch.
> > 
> > Any thoughts?
> > 

Re: Multiple bibliography files with ox-bibtex and html export

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 3826 bytes --]

I'd suggest 2 :) But not that I don't use this feature.

It should be easy to unify the code: something along the lines of starting the process, and then looping over bibtex files and sending them one by one to bibtex2html's standard input.

Cheers,
Clément.

On 2016-09-10 02:07, Thibault Marin wrote:
> 
> 
> Do you mean:
> 1) Using `call-process' for cases where a single bibliography file is
> passed and `process-send-string' when multiple files are used?
> 2) Using `process-send-string' regardless of the number of bibliography
> files?  In this case, can we still unify the code between single and
> multiple files?
> 3) Something else?
> 
> In my opinion 1) makes the code more error-prone and harder to
> maintain. If there are other reasons to replace the existing behavior
> (for single bibliography files) by `process-send-string', then 2) may
> make sense, otherwise it sounds to me that it may not be worth it: the
> existing code is apparently working fine for single files, I would feel
> a little uncomfortable changing it based only on my test case,
> especially since there isn't (as far as I know) a battery of tests for
> it.
> 
> - Is having a temporary file unacceptable?
> 
> The first patch creates and keeps the combined bibliography around, but
> this created file could easily be deleted if preferred.  If the problem
> is just the extra file, the first patch can fix it and seems less
> intrusive to me.
> 
> - Is the main concern performance?
> 
> I think that the main argument for using standard input may be to skip
> the disk access required by the temporary file.  I do not know if the
> potential savings for files of size around a few MB (or more?) justify
> the more intrusive change in the code.  Maybe others would have a better
> feel for this than I do.
> 
> Thanks for the comments on this.  Once a consensus is reached, I can
> work towards an updated patch.
> 
> thibault
> 
>> I'd suggest starting the process and then using process-send-string.
>>
>> Clément.
>>
>> On 2016-09-08 23:55, Thibault Marin wrote:
>>>
>>> Clément Pit--Claudel writes:
>>>
>>>> On 2016-09-06 23:46, Thibault Marin wrote:
>>>>>>> I am attaching a patch which allows me to use multiple files with html
>>>>>>> export.  It creates a combined bibliography file and call bibtex2html on
>>>>>>> it.  I am not sure this is the best way to address this, so any
>>>>>>> suggestion would be welcome.
>>>>
>>>> Sorry for the late comment.  bibtex2html can read from standard input; maybe that would be cleaner?
>>>>
>>>> Clément.
>>>
>>> That may be a good idea, it would prevent potential name clashing with
>>> the created bib file.  Currently, the function creates a
>>> <name-of-org-file>-combined.bib file with the content of all
>>> bibliography files, then bibtex2html creates
>>> <name-of-org-file>-combined.html and
>>> <name-of-org-file>-combined_bib.html.  Passing the contents via stdin
>>> would skip the <name-of-org-file>-combined.bib.  We could achieve the
>>> same by simply deleting <name-of-org-file>-combined.bib after calling
>>> bibtex2html.  I personally don't mind leaving the .bib file after
>>> processing, but if there is a consensus to limit the side effect, we can
>>> do that.
>>>
>>> As far as the implementation goes, I am not sure what is the best way to
>>> get this to work with stdin.  In the attached patch (which does *not*
>>> work) I tried to use `call-process-region' and dump the bibliography
>>> files into a temporary buffer.  This complicates the code a little.
>>> Alternatively, we could use the `INFILE' parameter from `call-process',
>>> but it looks that this would require a file, so it would not change much
>>> from the previous patch.
>>>
>>> Any thoughts?
>>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Multiple bibliography files with ox-bibtex and html export

[Thibault Marin]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]


Hi, sorry for the delay, I was away for a while.

> I'd suggest 2 :) But not that I don't use this feature.
>
> It should be easy to unify the code: something along the lines of starting the process, and then looping over bibtex files and sending them one by one to bibtex2html's standard input.
>
> Cheers,
> Clément.

Please find attached a tentative patch using `process-send-string'.  It seems to
work on my test cases, but I don't know how to wait for the bibtex2html process
to finish before processing the output.  I am currently using a while loop (see
l. 96 of the patch or l. 256 of the patched ox-bibtex.el) combined with a
sentinel changing the while condition upon completion.  This seems suboptimal
but I don't know how to achieve that more elegantly.

I still have a few questions:
1. How can I wait on the subprocess to complete after all the bib files have
   been passed via stdin?
2. Why is this approach preferred over concatenating bib files and delegating
   processing to the bibtex2html executable?

Thanks for the help,
thibault


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-bibtex.el-Support-multiple-bib-files-in-HTML-expo.patch --]
[-- Type: text/x-diff, Size: 4023 bytes --]

From 66edb29f79ddcdf90a47cd8626fb9f04167f5997 Mon Sep 17 00:00:00 2001
From: thibault <thibault.marin@gmx.com>
Date: Tue, 27 Sep 2016 22:36:57 -0500
Subject: [PATCH] ox-bibtex.el: Support multiple bib files in HTML export

* contrib/lisp/ox-bibtex.el (org-bibtex-process-bib-files): Pass
input bibliography files to asynchronous bibtex2html process.
---
 contrib/lisp/ox-bibtex.el | 69 +++++++++++++++++++++++++++++++++--------------
 1 file changed, 49 insertions(+), 20 deletions(-)

diff --git a/contrib/lisp/ox-bibtex.el b/contrib/lisp/ox-bibtex.el
index 56dec38..ca7839f 100644
--- a/contrib/lisp/ox-bibtex.el
+++ b/contrib/lisp/ox-bibtex.el
@@ -188,18 +188,26 @@ Return new parse tree."
       (lambda (keyword)
 	(when (equal (org-element-property :key keyword) "BIBLIOGRAPHY")
 	  (let ((arguments (org-bibtex-get-arguments keyword))
-		(file (org-bibtex-get-file keyword))
+		(files (split-string (org-bibtex-get-file keyword) ","))
 		temp-file
 		out-file)
 	    ;; Test if filename is given with .bib-extension and strip
-    	    ;; it off. Filenames with another extensions will be
+	    ;; it off. Filenames with another extensions will be
 	    ;; untouched and will finally rise an error in bibtex2html.
-	    (setq file (if (equal (file-name-extension file) "bib")
-			   (file-name-sans-extension file) file))
-	    ;; Outpufiles of bibtex2html will be put into current working directory
-	    ;; so define a variable for this.
-	    (setq out-file (file-name-sans-extension
-			    (file-name-nondirectory file)))
+	    (setq files
+		  (mapcar
+		   (lambda (file)
+		     (if (equal (file-name-extension file) "bib")
+			 (file-name-sans-extension file)
+		       file))
+		   files))
+	    ;; Output files of bibtex2html will be put into current
+	    ;; working directory so define a variable for this.
+	    (setq out-file
+		  (if (> (length files) 1)
+		      (concat (buffer-file-name) "-combined")
+		      (file-name-sans-extension
+		       (file-name-nondirectory (car files)))))
 	    ;; limit is set: collect citations throughout the document
 	    ;; in TEMP-FILE and pass it to "bibtex2html" as "-citefile"
 	    ;; argument.
@@ -216,18 +224,39 @@ Return new parse tree."
 				 :options
 				 (append (plist-get arguments :options)
 					 (list "-citefile" temp-file))))))
-	    ;; Call "bibtex2html" on specified file.
-	    (unless (eq 0 (apply
-			   'call-process
-			   (append '("bibtex2html" nil nil nil)
-				   '("-a" "-nodoc" "-noheader" "-nofooter")
-				   (let ((style
-					  (org-not-nil
-					   (org-bibtex-get-style keyword))))
-				     (and style (list "--style" style)))
-				   (plist-get arguments :options)
-				   (list (concat file ".bib")))))
-	      (error "Executing bibtex2html failed"))
+	    ;; Call "bibtex2html" on specified files.
+	    (let ((process-complete nil)
+		  (bibtex2html-proc
+		   (or
+		    (apply
+		     'start-process
+		     (append '("bibtex2html" "*bibtex2html-proc*")
+			     '("bibtex2html" "-a" "-nodoc"
+			       "-noheader" "-nofooter")
+			     (let ((style
+				    (org-not-nil
+				     (org-bibtex-get-style keyword))))
+			       (and style (list "--style" style)))
+			     (plist-get arguments :options)
+			     `("-o" ,out-file)))
+		    (error "Unable to start bibtex2html process"))))
+	      (when bibtex2html-proc
+		(set-process-sentinel
+		 bibtex2html-proc
+		 (lambda (process event)
+		   (when (equal event "finished\n")
+		     (setq process-complete t))))
+		(dolist (file files)
+		  (let ((file-content
+			 (with-temp-buffer
+			   (insert-file-contents (concat file ".bib"))
+			   (buffer-string))))
+		    (process-send-string bibtex2html-proc file-content)))
+		(process-send-eof bibtex2html-proc))
+	      ;; FIXME: How to wait for process to finish?
+	      (while (not process-complete)
+		(accept-process-output bibtex2html-proc)
+		(sit-for 1)))
 	    (and temp-file (delete-file temp-file))
 	    ;; Open produced HTML file, and collect Bibtex key names
 	    (with-temp-buffer
-- 
2.9.3

Re: Multiple bibliography files with ox-bibtex and html export

[Thibault Marin]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]


Hi,

I am trying to get back to the multiple bibliography issue discussed
some time ago.

I have two patches proposing two slightly different approaches (from
earlier suggestions):

- 0001-contrib-lisp-ox-bibtex.el-org-bibtex-process-bib-fil.patch creates a
  new bibliography file, a concatenation of all the input files, before calling
  bibtex2html.

- 0001-ox-bibtex.el-Support-multiple-bib-files-in-HTML-expo.patch use
  `start-process' to start the bibtex2html process, then feeds it the input
  bibliography files via stdin.  Once all files are sent it uses a while loop to
  wait for the process to complete (I don't know if there is a better way to do
  that without busy waiting).
  #+BEGIN_SRC emacs-lisp
;; FIXME: How to wait for process to finish?
(while (not process-complete)
  (accept-process-output bibtex2html-proc)
  (sit-for 1)))
  #+END_SRC

Could you please let me know which direction would be preferred, and if changes
should be made to the patch?

Thanks

thibault


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-contrib-lisp-ox-bibtex.el-org-bibtex-process-bib-fil.patch --]
[-- Type: text/x-diff, Size: 1987 bytes --]

From f093490a3631d4e9de0b18dc5e129eb8049975bc Mon Sep 17 00:00:00 2001
From: thibault <thibault.marin@gmx.com>
Date: Tue, 6 Sep 2016 22:42:39 -0500
Subject: [PATCH] * contrib/lisp/ox-bibtex.el (org-bibtex-process-bib-files):
 Add support for multiple bibliography files with html export.

Combine comma-separated bibliography files into a single one and process
it using bibtex2html.  This matches the behavior already present for
latex export.
---
 contrib/lisp/ox-bibtex.el | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/contrib/lisp/ox-bibtex.el b/contrib/lisp/ox-bibtex.el
index 56dec38..b46cb76 100644
--- a/contrib/lisp/ox-bibtex.el
+++ b/contrib/lisp/ox-bibtex.el
@@ -191,13 +191,27 @@ Return new parse tree."
 		(file (org-bibtex-get-file keyword))
 		temp-file
 		out-file)
+	    (let ((files (split-string file ",")))
+	      (when (< 1 (length files))
+		(let ((combined-bib-file
+		       (concat
+			(file-name-sans-extension
+			 (file-name-nondirectory
+			  (buffer-file-name))) "-combined.bib")))
+		  (with-temp-file combined-bib-file
+		    (dolist (bib files)
+		      (insert-file-contents
+		       (if (equal (file-name-extension bib) "bib")
+			   bib
+			 (concat bib ".bib")))))
+		  (setq file combined-bib-file))))
 	    ;; Test if filename is given with .bib-extension and strip
     	    ;; it off. Filenames with another extensions will be
 	    ;; untouched and will finally rise an error in bibtex2html.
 	    (setq file (if (equal (file-name-extension file) "bib")
 			   (file-name-sans-extension file) file))
-	    ;; Outpufiles of bibtex2html will be put into current working directory
-	    ;; so define a variable for this.
+	    ;; Output files of bibtex2html will be put into current
+	    ;; working directory so define a variable for this.
 	    (setq out-file (file-name-sans-extension
 			    (file-name-nondirectory file)))
 	    ;; limit is set: collect citations throughout the document
-- 
2.9.3


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-ox-bibtex.el-Support-multiple-bib-files-in-HTML-expo.patch --]
[-- Type: text/x-diff, Size: 4023 bytes --]

From c3056a453efb8bdd39e1dfd6fe4f75090cd1a584 Mon Sep 17 00:00:00 2001
From: thibault <thibault.marin@gmx.com>
Date: Tue, 27 Sep 2016 22:36:57 -0500
Subject: [PATCH] ox-bibtex.el: Support multiple bib files in HTML export

* contrib/lisp/ox-bibtex.el (org-bibtex-process-bib-files): Pass
input bibliography files to asynchronous bibtex2html process.
---
 contrib/lisp/ox-bibtex.el | 69 +++++++++++++++++++++++++++++++++--------------
 1 file changed, 49 insertions(+), 20 deletions(-)

diff --git a/contrib/lisp/ox-bibtex.el b/contrib/lisp/ox-bibtex.el
index 56dec38..ca7839f 100644
--- a/contrib/lisp/ox-bibtex.el
+++ b/contrib/lisp/ox-bibtex.el
@@ -188,18 +188,26 @@ Return new parse tree."
       (lambda (keyword)
 	(when (equal (org-element-property :key keyword) "BIBLIOGRAPHY")
 	  (let ((arguments (org-bibtex-get-arguments keyword))
-		(file (org-bibtex-get-file keyword))
+		(files (split-string (org-bibtex-get-file keyword) ","))
 		temp-file
 		out-file)
 	    ;; Test if filename is given with .bib-extension and strip
-    	    ;; it off. Filenames with another extensions will be
+	    ;; it off. Filenames with another extensions will be
 	    ;; untouched and will finally rise an error in bibtex2html.
-	    (setq file (if (equal (file-name-extension file) "bib")
-			   (file-name-sans-extension file) file))
-	    ;; Outpufiles of bibtex2html will be put into current working directory
-	    ;; so define a variable for this.
-	    (setq out-file (file-name-sans-extension
-			    (file-name-nondirectory file)))
+	    (setq files
+		  (mapcar
+		   (lambda (file)
+		     (if (equal (file-name-extension file) "bib")
+			 (file-name-sans-extension file)
+		       file))
+		   files))
+	    ;; Output files of bibtex2html will be put into current
+	    ;; working directory so define a variable for this.
+	    (setq out-file
+		  (if (> (length files) 1)
+		      (concat (buffer-file-name) "-combined")
+		      (file-name-sans-extension
+		       (file-name-nondirectory (car files)))))
 	    ;; limit is set: collect citations throughout the document
 	    ;; in TEMP-FILE and pass it to "bibtex2html" as "-citefile"
 	    ;; argument.
@@ -216,18 +224,39 @@ Return new parse tree."
 				 :options
 				 (append (plist-get arguments :options)
 					 (list "-citefile" temp-file))))))
-	    ;; Call "bibtex2html" on specified file.
-	    (unless (eq 0 (apply
-			   'call-process
-			   (append '("bibtex2html" nil nil nil)
-				   '("-a" "-nodoc" "-noheader" "-nofooter")
-				   (let ((style
-					  (org-not-nil
-					   (org-bibtex-get-style keyword))))
-				     (and style (list "--style" style)))
-				   (plist-get arguments :options)
-				   (list (concat file ".bib")))))
-	      (error "Executing bibtex2html failed"))
+	    ;; Call "bibtex2html" on specified files.
+	    (let ((process-complete nil)
+		  (bibtex2html-proc
+		   (or
+		    (apply
+		     'start-process
+		     (append '("bibtex2html" "*bibtex2html-proc*")
+			     '("bibtex2html" "-a" "-nodoc"
+			       "-noheader" "-nofooter")
+			     (let ((style
+				    (org-not-nil
+				     (org-bibtex-get-style keyword))))
+			       (and style (list "--style" style)))
+			     (plist-get arguments :options)
+			     `("-o" ,out-file)))
+		    (error "Unable to start bibtex2html process"))))
+	      (when bibtex2html-proc
+		(set-process-sentinel
+		 bibtex2html-proc
+		 (lambda (process event)
+		   (when (equal event "finished\n")
+		     (setq process-complete t))))
+		(dolist (file files)
+		  (let ((file-content
+			 (with-temp-buffer
+			   (insert-file-contents (concat file ".bib"))
+			   (buffer-string))))
+		    (process-send-string bibtex2html-proc file-content)))
+		(process-send-eof bibtex2html-proc))
+	      ;; FIXME: How to wait for process to finish?
+	      (while (not process-complete)
+		(accept-process-output bibtex2html-proc)
+		(sit-for 1)))
 	    (and temp-file (delete-file temp-file))
 	    ;; Open produced HTML file, and collect Bibtex key names
 	    (with-temp-buffer
-- 
2.9.3