latex exporting to different directory with v9.0

latex exporting to different directory with v9.0

[Alex Fenton]

Firstly, congratulations and thanks to all the contributors to the new 
release.

One thing that has unfortunately stopped working is my setup for 
exporting to pdf, putting all the intermediary  (.tex, .aux, .bbl, etc) 
and output files in a different directory to the source .org files. My 
setup is basically

project/
- images/
- out/

with all the .org files in the project base directory. A simple 
publishing set-up exported .tex to out/ then called org-latex-compile.

With 9.0, org-latex-compile hands off to org-compile-file, in which (as 
documented) "`default-directory' is set to SOURCE directory during the whole
process." This means, however, that all the image and bibliograpy links 
in the .org files are now broken, since they point to ./img/foo.png and 
not ../img/foo.png

It is easy enough to continue to compile from the base directory with 
latexmk and --output-directory, but how is it now intended to do this 
within org's export/publishing framework?

symlinks are not an option since I'm sometimes working on Windows.

thanks
alex

Re: latex exporting to different directory with v9.0

[Nicolas Goaziou]

Hello,

Alex Fenton <alex.fenton@pressure.to> writes:

> One thing that has unfortunately stopped working is my setup for
> exporting to pdf, putting all the intermediary  (.tex, .aux, .bbl,
> etc) and output files in a different directory to the source .org
> files. My setup is basically
>
> project/
> - images/
> - out/
>
> with all the .org files in the project base directory. A simple
> publishing set-up exported .tex to out/ then called org-latex-compile.
>
> With 9.0, org-latex-compile hands off to org-compile-file, in which
> (as documented) "`default-directory' is set to SOURCE directory during
> the whole
> process." This means, however, that all the image and bibliograpy
> links in the .org files are now broken, since they point to
> ./img/foo.png and not ../img/foo.png

Honestly, I'm surprised it worked. I'm also surprised it could be
related to `default-directory' set-up, since links are created during
Org -> LaTeX conversion, whereas `org-compile-file' handles LaTeX ->
PDF. What is that "simple publishing set-up" you are talking about?

Not that there is not something fishy in `org-compile-file' at the
moment (see, e.g.,
<http://permalink.gmane.org/gmane.emacs.orgmode/110030>), but I cannot
see a way out without breaking some eggs.

Basically, there are three directories to consider: source (".tex")
directory, output (".pdf") directory, and working directory, i.e.,
probably ".org" file directory.

The assumption for `org-compile-file', and before it,
`org-latex-compile', is that source and output directories are the same.
Hence `org-compile-file' returns an error when

  (concat out-dir tex-file.pdf)

cannot be found.

Now, according to `org-latex-pdf-process', some values do not care much
about the working directory, e.g.,

  "pdflatex -interaction nonstopmode -output-directory %o %f"

but others clearly require the working directory to be the output
directory (note the absence of output directory in the command below)

  "texi2dvi -p -b -V %f"

As a consequence, if we do not set `default-directory' to the output
directory, the latter is broken. Note that if we do, "%o" place-holder
becomes useless as it is always "./".

In a nutshell we can either set default-directory to source/output
directory or leave it as-is. In all cases, it seems to break something
anyway.

I'm Cc'ing Joon Ro since he reported a related issue with
`org-compile-file'.

Comments welcome.


Regards,

-- 
Nicolas Goaziou

Re: latex exporting to different directory with v9.0

[Alex Fenton]

Hello,

On 05/11/16 23:54, Nicolas Goaziou wrote:
> Honestly, I'm surprised it worked. I'm also surprised it could be
> related to `default-directory' set-up, since links are created during
> Org -> LaTeX conversion, whereas `org-compile-file' handles LaTeX ->
> PDF. What is that "simple publishing set-up" you are talking about?

Thanks for the reply. The publishing set up defines 
:publishing-directory as "out", and then the publishing function for 
each org file was:

(defun thesis-publish-chapter-to-pdf (plist filename pub-dir)
   "Export a chapter to a LaTeX file in output dir and compile"
   (let ((outfile
           (org-publish-org-to 'latex filename "-chapter.tex" plist 
pub-dir)))
     (org-latex-compile (file-relative-name outfile))))

With org-latex-pdf-process being a standard "latexmk -xelatex 
-interaction=nonstopmode -f -outdir=%o -pdf %f"

With 8.3, the latex process launches in the base (org) directory. Image 
& bibtex links in the tex file (in ./out) are resolved by latex relative 
to the base (working) directory.

With 9.0, the enforced switch of default-directory to "./out" means the 
latex process is launched there instead, and relative links are no 
longer correctly resolved.
> Basically, there are three directories to consider: source (".tex")
> directory, output (".pdf") directory, and working directory, i.e.,
> probably ".org" file directory.
>
> The assumption for `org-compile-file', and before it,
> `org-latex-compile', is that source and output directories are the same.
That seems reasonable. All that I'd ideally like to continue to be able 
to do is keep a clean "working" (org) directory with correct relative 
links in the org files, and use a single other directory as "source" 
(tex) and "output" (pdf).
> but others clearly require the working directory to be the output
> directory (note the absence of output directory in the command below)
>
>    "texi2dvi -p -b -V %f"

I'm conscious that org-mode has to work with a lot of different backends 
and compilers, but that example should still work without switching 
default-directory; only the pdf file would end up not in ./out but ./.

> As a consequence, if we do not set `default-directory' to the output
> directory, the latter is broken. Note that if we do, "%o" place-holder
> becomes useless as it is always "./".
>
> In a nutshell we can either set default-directory to source/output
> directory or leave it as-is. In all cases, it seems to break something
> anyway.

I imagine the case for the vast majority of the time is that working, 
source and output directories are the same. I'm just wondering whether, 
for the cases where that's not true (calling a compiler from one 
directory to compile a file somewhere else), the responsibility for 
setting the correct default-directory, if necessary, could be left to 
the calling function, rather than having an enforced switch hard-coded 
in the core org-compile-file function.

Thanks again,
alex

Re: latex exporting to different directory with v9.0

[Nicolas Goaziou]

Hello,

Alex Fenton <alex.fenton@pressure.to> writes:

> (defun thesis-publish-chapter-to-pdf (plist filename pub-dir)
>   "Export a chapter to a LaTeX file in output dir and compile"
>   (let ((outfile
>           (org-publish-org-to 'latex filename "-chapter.tex" plist
> pub-dir)))
>     (org-latex-compile (file-relative-name outfile))))

See `org-latex-publish-to-pdf' for a possibly better way to handle this.

> With org-latex-pdf-process being a standard
> "latexmk -xelatex -interaction=nonstopmode -f -outdir=%o -pdf %f"
>

[...]

> That seems reasonable. All that I'd ideally like to continue to be
> able to do is keep a clean "working" (org) directory with correct
> relative links in the org files, and use a single other directory as
> "source" (tex) and "output" (pdf).

Per above, you could use latexmk "-C" option and move both ".tex" and
".pdf" file to appropriate directories.

>> but others clearly require the working directory to be the output
>> directory (note the absence of output directory in the command below)
>>
>>    "texi2dvi -p -b -V %f"
>
> I'm conscious that org-mode has to work with a lot of different
> backends and compilers, but that example should still work without
> switching default-directory; only the pdf file would end up not in
> ./out but ./.

This breaks the export process, because `org-compile-file' can no longer
find, and return, the produced file. `org-compile-file' has to know
where the output is.

> I imagine the case for the vast majority of the time is that working,
> source and output directories are the same. I'm just wondering
> whether, for the cases where that's not true (calling a compiler from
> one directory to compile a file somewhere else), the responsibility
> for setting the correct default-directory, if necessary, could be left
> to the calling function, rather than having an enforced switch
> hard-coded in the core org-compile-file function.

One problem is that we already provide compilation commands that do not
set it (e.g., the "texi2dvi" command above). As I wrote, if we leave
default-directory alone, something else has to be fixed.

In any case, I removed default-directory setting and, hopefully, applied
the necessary changes to commands.

Feedback welcome.


Regards,

-- 
Nicolas Goaziou

Re: latex exporting to different directory with v9.0

[Alex Fenton]

Hello,

On 08/11/16 18:49, Nicolas Goaziou wrote:
> Per above, you could use latexmk "-C" option and move both ".tex" and
> ".pdf" file to appropriate directories.

I wasn't aware of the "-C" option, it looks like it could - with some 
settings - blow away the intermediate files that latexmk uses to 
minimise compile passes.

> This breaks the export process, because `org-compile-file' can no longer
> find, and return, the produced file. `org-compile-file' has to know
> where the output is.

Ah, OK. I'm curious, however, how a compiler (eg tex2dvi) cd'd to a 
different working directory can ever resolve links to image files.


> One problem is that we already provide compilation commands that do not
> set it (e.g., the "texi2dvi" command above). As I wrote, if we leave
> default-directory alone, something else has to be fixed.
>
> In any case, I removed default-directory setting and, hopefully, applied
> the necessary changes to commands.

I'm very grateful for your help with this and for trying this 
change/reversion. I'll try it with the next release.

Best,
alex

Re: latex exporting to different directory with v9.0

[Nicolas Goaziou]

Hello,

Alex Fenton <alex.fenton@pressure.to> writes:

> Ah, OK. I'm curious, however, how a compiler (eg tex2dvi) cd'd to
> a different working directory can ever resolve links to image files.

It cannot.

Another alternative is:

  "texi2dvi -p -b -V -o %o%b.pdf %f"

However this litters current directory with auxiliary files instead of
output directory. So `org-latex-remove-logfiles' is ignored (it looks
for such files in the output directory) when using this process, which
is arguably a bug.

What do you suggest, then?

Regards,

-- 
Nicolas Goaziou