org export outcomes differ from one attempt to the next

org export outcomes differ from one attempt to the next

[Eric S Fraga]

Hello all,

I'm pulling my hair out in frustration...

I have a very large org file that comprises my lectures for a whole
course.  This org file makes extensive use of LaTeX and therefore has
quite a number of latex_header directives.

I have a very strange situation: if I start emacs, visit the file and
export to beamer LaTeX and then export again to beamer LaTeX, the result
of both exports differs. The difference is that some of the latex_header
lines do not get exported in the second attempt.

I have just verified this with "emacs -Q", loading in ox-latex and
ob-octave, then exporting, renaming the target out of the way and
exporting again.  A diff shows 3 lines in the first file that are not in
the second and 4 lines in the second that are not in the first.

What I've noticed, and which surprised me, is that the order of the
lines written out in the LaTeX preamble does not match the order of the
latex_header lines in my org file.

I cannot post a minimal example as any time I try to create one, the
problem disappears.  What I would like is some suggestion of how I can
track the problem down.  This problem has existed for quite some time
without me realising that there was a pattern to it.

Thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.1-234-g8c85c9

Re: org export outcomes differ from one attempt to the next

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 182 bytes --]

Following up on my own post, I thought I would illustrate what I get.

I've attached three screenshots:

1. a snippet from the org file where I have a number of latex_header
lines.


[-- Attachment #2: screendump-20150921163358.png --]
[-- Type: image/png, Size: 64265 bytes --]

[-- Attachment #3: Type: text/plain, Size: 118 bytes --]


2. one of the sections of the resulting (from C-c C-e l b) LaTeX files
which differ from one invocation to another.


[-- Attachment #4: screendump-20150921163429.png --]
[-- Type: image/png, Size: 93936 bytes --]

[-- Attachment #5: Type: text/plain, Size: 43 bytes --]


3. another strange section of the diffs.


[-- Attachment #6: screendump-20150921163508.png --]
[-- Type: image/png, Size: 9217 bytes --]

[-- Attachment #7: Type: text/plain, Size: 219 bytes --]


Maybe this will help somebody point me in the right direction!

Org is up to date from git as of half an hour ago...

Thanks again,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.1-234-g8c85c9

Re: org export outcomes differ from one attempt to the next

[Robert Klein]

Hi,

Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> Following up on my own post, I thought I would illustrate what I get.
> 
> I've attached three screenshots:
> 
> 1. a snippet from the org file where I have a number of latex_header
> lines.

two things:

- if your org version is the one stated in your mail footer
  (release_8.3.1-234-g8c85c9), then it is 11 days old :)

- anyway, this sounds similar to my issues in thread
  http://article.gmane.org/gmane.emacs.orgmode/98428  (resolved, but
  it wasn't really clear, /why/ the patch worked...)  Does your
  issue also happen, when you take only the beginning of your file up
  to the first paragraph of the first heading (not marked
  as :noexport:)?  This is not a /minimal/ minimal example, but if
  you can post this plus the relevant parts of your setup, there is a
  chance to reproduce and solve the issue.

Best regards
Robert

Re: org export outcomes differ from one attempt to the next

[Eric S Fraga]

On Monday, 21 Sep 2015 at 20:21, Robert Klein wrote:
> Hi,
>
> Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
>
>> Following up on my own post, I thought I would illustrate what I get.
>> 
>> I've attached three screenshots:
>> 
>> 1. a snippet from the org file where I have a number of latex_header
>> lines.
>
> two things:
>
> - if your org version is the one stated in your mail footer
>   (release_8.3.1-234-g8c85c9), then it is 11 days old :)

Hi Robert,

thanks for your response.

I run multiple instances of emacs simultaneously, one of them
specifically for email, i.e. gnus, and that emacs instance was running a
previous version of org...  the tests were done with the latest version
(-252-).  sorry for the confusion.

> - anyway, this sounds similar to my issues in thread
>   http://article.gmane.org/gmane.emacs.orgmode/98428  (resolved, but
>   it wasn't really clear, /why/ the patch worked...)

I went through that thread and I'm not entirely sure if or how it
relates to my issue.  However, I tried going back to June (well before
the patch mentioned) and that version shows the same behaviour I am
seeing now but worse (more lines changing from one export to the next)
so maybe it is related.

> Does your
>   issue also happen, when you take only the beginning of your file up
>   to the first paragraph of the first heading (not marked
>   as :noexport:)?  This is not a /minimal/ minimal example, but if
>   you can post this plus the relevant parts of your setup, there is a
>   chance to reproduce and solve the issue.

Actually, my settings are at the end of the whole 4000+ line file so I
am not sure what you are suggesting I try.  If I delete the whole
settings subtree, the exports all result in the same LaTeX.  However,
that is of no use to me, obviously.  I guess I can try to add one line
at a time to see when the export breaks.  I may try that tomorrow.

I guess I would like some insight into how the latex_header lines are
parsed by org, maybe in org-element?  There is something strange going
on because every or most evaluations of the export dispatcher to LaTeX
lead to slightly different results.

Thanks again,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.1-234-g8c85c9

Re: org export outcomes differ from one attempt to the next

[Nicolas Goaziou]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Actually, my settings are at the end of the whole 4000+ line file so I
> am not sure what you are suggesting I try.  If I delete the whole
> settings subtree, the exports all result in the same LaTeX.  However,
> that is of no use to me, obviously.  I guess I can try to add one line
> at a time to see when the export breaks.  I may try that tomorrow.

Providing a shorter test file would be helpful, no matter how much
settings you have.

> I guess I would like some insight into how the latex_header lines are
> parsed by org, maybe in org-element?

They are collected in `org-export--get-inbuffer-options' which is called
by `org-export-get-environment'.

IIRC, Robert Klein's problem was due to an explicit call to
`org-export-get-environment' at Babel evaluation time, before the buffer
was correctly prepared for export. If you call
`org-export-get-environment' within your settings, it may be related.


Regards,

-- 
Nicolas Goaziou

Re: org export outcomes differ from one attempt to the next

[Robert Klein]

Hi,

Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Hello,
> 
> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
> 
> > Actually, my settings are at the end of the whole 4000+ line file
> > so I am not sure what you are suggesting I try.  If I delete the
> > whole settings subtree, the exports all result in the same LaTeX.
> > However, that is of no use to me, obviously.  I guess I can try to
> > add one line at a time to see when the export breaks.  I may try
> > that tomorrow.
> 
> Providing a shorter test file would be helpful, no matter how much
> settings you have.
> 
> > I guess I would like some insight into how the latex_header lines
> > are parsed by org, maybe in org-element?
> 
> They are collected in `org-export--get-inbuffer-options' which is
> called by `org-export-get-environment'.
> 
> IIRC, Robert Klein's problem was due to an explicit call to
> `org-export-get-environment' at Babel evaluation time, before the
> buffer was correctly prepared for export. If you call
> `org-export-get-environment' within your settings, it may be related.

is there another way to get property values during export (i use them
in #+LaTeX_Header: lines).

Thanks and best regards
Robert

Re: org export outcomes differ from one attempt to the next

[Nicolas Goaziou]

Hello,

Robert Klein <roklein@roklein.de> writes:

> is there another way to get property values during export (i use them
> in #+LaTeX_Header: lines).

During export, all property values are accessible through communication
channel (usually INFO argument), e.g.,

  (plist-get info :latex-header)
  (plist-get info :latex-header-extra)

Regards,


-- 
Nicolas Goaziou

Re: org export outcomes differ from one attempt to the next

[Eric S Fraga]

On Monday, 21 Sep 2015 at 22:25, Nicolas Goaziou wrote:
> Providing a shorter test file would be helpful, no matter how much
> settings you have.

Hi Nicolas,

Sorry for the delay in getting back to you.  I have been working on this
document over the past 10 days, getting ready for the new academic
year.  This particular file is the complete set of slides for a whole
course.  It is very long and brings in all kinds of bits from org,
primarily beamer export, babel for a number of languages and has a lot
of embedded LaTeX.

I have not been able to create smaller example that demonstrates the
problem.

I *have* however solved (to use the term loosely) the problem by moving
all of the #+latex_header settings to the start of the file instead of
the end.  They remain within a separate :noexport: section.  Only their
actual location in the file has changed.

I usually put these at the end because I also use emacs file variables
for settings specific to the document and these are always at the end of
the file (for me, anyway).  These emacs settings remain at the end.

I wish I could provide more insight into the problem but, at least for
now, I am able to get my ~700 slides ready for teaching this term!  My
gut feeling, nonetheless, is that I have something in one of the 700
slides that confuses the org parser somehow so that by the time the end
of the file is reached, some data structures are inconsistent.  But who
knows?

Thanks for your help,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.1-270-g256fef

Re: org export outcomes differ from one attempt to the next

[Nicolas Goaziou]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> I wish I could provide more insight into the problem but, at least for
> now, I am able to get my ~700 slides ready for teaching this term!  My
> gut feeling, nonetheless, is that I have something in one of the 700
> slides that confuses the org parser somehow so that by the time the end
> of the file is reached, some data structures are inconsistent.  But who
> knows?

It doesn't ring a bell. Wild guess: what if you set
`org-element-use-cache' to nil prior to export?

Regards,

Re: org export outcomes differ from one attempt to the next

[Eric S Fraga]

On Thursday,  1 Oct 2015 at 22:02, Nicolas Goaziou wrote:
> It doesn't ring a bell. Wild guess: what if you set
> `org-element-use-cache' to nil prior to export?

Ah ha!  If I set this to nil, the exported file looks good!  And it is
the same the first time as well as the second time which is definitely a
change in behaviour from before.  And this is with my settings all at
the end of the file.

Thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.5.1, Org release_8.3.1-203-g4ca3d3