Efficiency of Org v. LaTeX v. Word

Efficiency of Org v. LaTeX v. Word

[Ken Mankoff]

People here might be interested in a publication from [2014-12-19 Fri]
available at http://dx.doi.org/10.1371/journal.pone.0115069

Title: An Efficiency Comparison of Document Preparation Systems Used
in Academic Research and Development

Summary: Word users are more efficient and have less errors than even
experienced LaTeX users.

Someone here should repeat experiment and add Org into the mix, perhaps
Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
pandoc) beat straight Word?

  -k.

Re: Efficiency of Org v. LaTeX v. Word

[Thomas S. Dye]

,----------------------------------------------------------------------
| "One may also argue that given a well-designed LaTeX document class  
| file, document development speed and text and formatting accuracy are
| significantly improved."                                             
`----------------------------------------------------------------------

Apparently, the LaTeX users didn't have the benefit of a document
class.  Hard to take a "study" like this seriously.

,-----------------------------------------------------------------------
| "preventing researchers from producing documents in LaTeX would save  
| time and money to maximize the benefit of research and development for
| both the research team and the public"                                
`-----------------------------------------------------------------------

All you have to lose is your freedom.

All the best,
Tom

Ken Mankoff <mankoff@gmail.com> writes:

> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.
>
> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?
>
>   -k.
>
>
>

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Efficiency of Org v. LaTeX v. Word

[briangpowell .]

[-- Attachment #1: Type: text/plain, Size: 2726 bytes --]

Word is a desktop publishing system.

LaTeX is a macro language which lays on top of TeX=Tau-Epsilon-Chi~Art in
Greek

TeX is computerized typesetting that enables vector graphics--you can get
TeX to draw anything you want--you can even create your own font.

More Math journals and books you'd find in the library are created using
TeX than any other software system.

The poor kerning and severe limitations of Word are too many to number here.

Word is in a different class of software, the 2 aren't comparable at all.

Word is a poor WYSIWYG software package that is good for low quality
desktop publishing, team collaboration but can be programmed and interacted
with through VB--its useful to the general public.

LaTeX provides precision and expression; there are things you can do with
TeX that aren't possible with Word.

Members of the Free Software community (which TeX has always been a part
of) will never bow down to the Micro$oft tyranny which is so evil words
can't express the depths of their corruption--the comparison is absurd.




On Fri, Dec 26, 2014 at 6:36 PM, Thomas S. Dye <tsd@tsdye.com> wrote:

>
> ,----------------------------------------------------------------------
> | "One may also argue that given a well-designed LaTeX document class
> | file, document development speed and text and formatting accuracy are
> | significantly improved."
> `----------------------------------------------------------------------
>
> Apparently, the LaTeX users didn't have the benefit of a document
> class.  Hard to take a "study" like this seriously.
>
> ,-----------------------------------------------------------------------
> | "preventing researchers from producing documents in LaTeX would save
> | time and money to maximize the benefit of research and development for
> | both the research team and the public"
> `-----------------------------------------------------------------------
>
> All you have to lose is your freedom.
>
> All the best,
> Tom
>
> Ken Mankoff <mankoff@gmail.com> writes:
>
> > People here might be interested in a publication from [2014-12-19 Fri]
> > available at http://dx.doi.org/10.1371/journal.pone.0115069
> >
> > Title: An Efficiency Comparison of Document Preparation Systems Used
> > in Academic Research and Development
> >
> > Summary: Word users are more efficient and have less errors than even
> > experienced LaTeX users.
> >
> > Someone here should repeat experiment and add Org into the mix, perhaps
> > Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> > Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> > pandoc) beat straight Word?
> >
> >   -k.
> >
> >
> >
>
> --
> Thomas S. Dye
> http://www.tsdye.com
>
>

[-- Attachment #2: Type: text/html, Size: 3793 bytes --]

Re: Efficiency of Org v. LaTeX v. Word

[Christopher W. Ryan]

This seems like more of a typing contest than anything else. Reproducing
a single page of an already-typeset document is not what LaTeX is
designed for, nor is it what scientists do for a living. The test
selections were absurdly short relative to the typical scientic
manuscript. Long and complex documents are where LaTeX excels. And this
did not call upon some of the most important (IMHO) capabilities of
LaTeX: managing citations with BibTex; changing the style to suit
different journals; storing, revisiting, and reusing your document years
later.

--Chris Ryan

Ken Mankoff wrote:
> 
> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
> 
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
> 
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.
> 
> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?
> 
>   -k.
> 
> 

Re: Efficiency of Org v. LaTeX v. Word

[Nick Dokos]

Ken Mankoff <mankoff@gmail.com> writes:

> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.
>
> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?
>

The "study" is deeply flawed: Word users typed more text in 30 minutes
than LaTeX users? 20% more? For straight text? I don't believe it: I
think it's much more likely that the Word users were better typists on
average and I didn't see any mention of normalizing the results by
taking that into account. And for LaTeX, the editing environment is
of paramount importance: did they mostly use vi, emacs, emacs+auctex?
Other than a couple of vague sentences in the "Discussion" section,
there is no mention of how this variable was (or was not) controlled.
And did the LaTeX users have to type the preamble or were they allowed
to use a template? It just seems unbelievable that there is such
a big difference for straight text.

I can believe perhaps that typing a table by hand into LaTeX is more
error prone than typing it into Word (although to be honest, I have
never done the latter, so I don't really know). With org and radio
tables, this would be a non-issue on the LaTeX side.

As for equations, even the authors admit that LaTeX is better, although
they tend to minimize the differences as statistically insignificant (at
least between the expert classes), which strikes me as somewhat suspect
as well: there seems to be a 10% difference between the expert user
averages and a bigger one for novices, although the error bars might
overlap in the first case (although they don't look it). I don't think
that even that would make the difference insignificant, but we'd have to
analyze their raw results to make sure (which they do provide and which
I took a look at, but afaict they don't provide answers to the questions
I raised above; maybe we should suggest that the authors use org and
reproducible results methods).

Anyway, color me deeply suspicious of the "study". 

-- 
Nick

Re: Efficiency of Org v. LaTeX v. Word

[Peter Neilson]

On Fri, 26 Dec 2014 23:27:37 -0500, Nick Dokos <ndokos@gmail.com> wrote:

> Anyway, color me deeply suspicious of the "study".

Indeed!

The study touches only a few of the inherent difficulties in document  
production. Its major flaw is that it draws any conclusions at all  
recommending that authors produce documents one way or another. Personally  
I am always disappointed when someone requests a document in MS Word  
format, because that means I'll have to fire up Libre Office and shove my  
text through it, rather than using whatever other system I happen to have  
been using. I do not believe that I currently own a system with genuine MS  
Word.

As well as having insufficient control of variables, and a flawed  
understanding of what is involved in "document preparation," the study  
also has a marginally small sample size. Any study for any purpose that  
presents "statistics" with sample sizes smaller than 30 is immediately  
suspect. I won't even begin to address the misinterpretation of  
correlation as causation that appears in the "softer" sciences, nor their  
necessity for sample sizes far larger than 100, nor the tendency in some  
fields to mistake a time series as a set of samples.

MS Word works extremely well for "one-off" small papers. Little investment  
of effort is required for a naive person to produce adequate results, and  
as every user of emacs knows, that's pretty much the opposite of emacs.

On the other hand, MS Word has historically been a terrible tool for  
producing large documents, or documents that are to be maintained by a  
group of people, or over several years or decades. Handling Word's "Master  
Document" provision without being crippled by corrupted documents is an  
art form unto itself. The standard advice among experienced users of Word  
has always been, "Don't Use Master Documents!" When a group of people are  
all editing versions of a document, any attempt to use standard formatting  
in Word requires substantial effort to prevent naive contributers from  
reformatting outside the established styles, or even breaking all the  
styles. Furthermore, Word documents are in general not amenable to  
incremental version control as commonly used by coding teams.

My conclusions? If your paper is trivial and you are under pressure to  
produce it quickly, then MS Word might be the best tool. Established  
journals should attempt to allow contributions in more than one format,  
and restriction to MS Word format is a bad idea, no matter how much some  
people like the apparent ease-of-use that MS Word provides. Attempting to  
extend the "study" to include org mode would be a waste of effort.

Re: Efficiency of Org v. LaTeX v. Word

[Achim Gratz]

Am 26.12.2014 um 23:47 schrieb Ken Mankoff:
> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.

The way researcher efficiency is defined in that "study" completely 
misses the purpose of scientific publishing and it goes downhill from 
there.  The statistics are pseudo-scientific smokes and mirrors, no 
control groups, no normalization and not a single hint of why it should 
be acceptable to use normal distributions for something that clearly 
isn't normally distributed other than the obvious convenience of drawing 
wild conclusions from a small sample size.

I'm still not sure if this isn't an elaborate joke, but I'm afraid not.

> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?

Repeating a deeply flawed "study" that seems designed to support some 
pre-conceived notion or preference of the authors isn't going to produce 
any new insights and I'm quite certain that there is better research 
into the differences of WYSIWIG vs. non-WYSIWIG publication systems 
and/or researcher efficiency.  If a reasearcher is nothing more than a 
typist that needs to produce pages of texts, tables and equations in a 
prescribed format in the least amount of time motivated by a monetary 
prize, we wouldn't need researchers at all.  That would incidentally 
save much more money than having them all switch from LaTeX to Word, so 
let's stop funding research.


-- 
Achim.

(on the road :-)

Re: Efficiency of Org v. LaTeX v. Word

[Paul Rudin]

Ken Mankoff <mankoff-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.
>
> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?
>

No mention of emacs... who uses anything else to prepare their LaTeX?

Re: Efficiency of Org v. LaTeX v. Word

[M]

> Von: Paul Rudin <paul@rudin.co.uk>
> Datum: Sat, 27 Dec 2014 10:05:19 +0000
> An: <emacs-orgmode@gnu.org>
> Betreff: Re: [O] Efficiency of Org v. LaTeX v. Word
> 
> Ken Mankoff <mankoff@gmail.com> writes:
> 
>> People here might be interested in a publication from [2014-12-19 Fri]
>> available at http://dx.doi.org/10.1371/journal.pone.0115069
>> 
>> Title: An Efficiency Comparison of Document Preparation Systems Used
>> in Academic Research and Development
>> 
>> Summary: Word users are more efficient and have less errors than even
>> experienced LaTeX users.
>> 
>> Someone here should repeat experiment and add Org into the mix, perhaps
>> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
>> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
>> pandoc) beat straight Word?
>> 
> 
> No mention of emacs... who uses anything else to prepare their LaTeX?
> 
Did you forget the " ;-)" or are you serious?

Emacs is for sure a very good one, but there are a lot of popular
alternatives, if you have a look at the (for sure not representative) voting
on the answers of this discussion here:

http://tex.stackexchange.com/questions/339/latex-editors-ides

(It's clear, that people may have voted for several of those editors, so
that no valid statistics at all, but at least an idea...)

Is there any real survey result about which editors LaTeX users use?

Martin

Re: Efficiency of Org v. LaTeX v. Word

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 3744 bytes --]

I agree that this study is certainly not large enough to draw strong
conclusions, but it raises a couple of questions
and some points may require attention.

I have spent many years in the TeX world. I see how lots of people use TeX
: students, professionals, researchers etc...
and I would easily draw 2 categories of people :
- those who are programmers "in their soul" (DEK once said that 2% or so of
the whole human race is gifted with programming, the same way some people
are gifted to play music etc.)
- those who use LaTeX "because it is the best typesetting system"
People who belong to the intersection of those 2 categories will certainly
be very efficient in producing documents with LaTeX, much more than what
this study shows.
But people from the first category may also be efficient in producing
documents with Word (Word is programmable too and the typesetting engine is
fancier than most people would believe).
The real problem is the guys from the second category who stick to use a
tool they are not comfortable with but they don't want to admit it.
Over the last years, I have seen more and more students come with LaTeX
documents which had a very poor appearance.
There has been a lot of pressure with the rise of Linux to use LaTeX.
Unfortunately the results of using LaTeX may not be up to the expectations.
The tool is too complex. It can produce beautiful documents when used
right, but it can also easily produce awful documents.
You can also spend a lot of time in fixing details, and it happens more
frequently than even proficient LaTeX users would admit.
In the end, I think the tendency is to a growing number of LaTeX users who
use it poorly.

Finally, today, my experience is that publishers charge much more for LaTeX
documents than for Word (or similar tools) documents and they are reluctant
to use LaTeX because of its complexity.

That was my $0.02

Fabrice

2014-12-27 11:36 GMT+01:00 M <Elwood151@web.de>:

> > Von: Paul Rudin <paul@rudin.co.uk>
> > Datum: Sat, 27 Dec 2014 10:05:19 +0000
> > An: <emacs-orgmode@gnu.org>
> > Betreff: Re: [O] Efficiency of Org v. LaTeX v. Word
> >
> > Ken Mankoff <mankoff@gmail.com> writes:
> >
> >> People here might be interested in a publication from [2014-12-19 Fri]
> >> available at http://dx.doi.org/10.1371/journal.pone.0115069
> >>
> >> Title: An Efficiency Comparison of Document Preparation Systems Used
> >> in Academic Research and Development
> >>
> >> Summary: Word users are more efficient and have less errors than even
> >> experienced LaTeX users.
> >>
> >> Someone here should repeat experiment and add Org into the mix, perhaps
> >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> >> pandoc) beat straight Word?
> >>
> >
> > No mention of emacs... who uses anything else to prepare their LaTeX?
> >
> Did you forget the " ;-)" or are you serious?
>
> Emacs is for sure a very good one, but there are a lot of popular
> alternatives, if you have a look at the (for sure not representative)
> voting
> on the answers of this discussion here:
>
> http://tex.stackexchange.com/questions/339/latex-editors-ides
>
> (It's clear, that people may have voted for several of those editors, so
> that no valid statistics at all, but at least an idea...)
>
> Is there any real survey result about which editors LaTeX users use?
>
> Martin
>
>
>
>


-- 
Fabrice Popineau
-----------------------------
SUPELEC
Département Informatique
3, rue Joliot Curie
91192 Gif/Yvette Cedex
Tel direct : +33 (0) 169851950
Standard : +33 (0) 169851212
------------------------------

[-- Attachment #2: Type: text/html, Size: 4980 bytes --]

Re: Efficiency of Org v. LaTeX v. Word

[Daniele Pizzolli]

Hello,

Ken Mankoff writes:

> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.

As other said, the efficiency in the paper is about the manual copy of a
small portion of text, tables...  This is a little bit different to
publish a research, maybe a reproducible one, with the help of a team.

> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?

Repeating a flawed experiment do not add a lot of value...

Best,
Daniele

Re: Efficiency of Org v. LaTeX v. Word

[Eric S Fraga]

On Friday, 26 Dec 2014 at 21:21, briangpowell . wrote:

[...]

> Word is in a different class of software, the 2 aren't comparable at all.

Indeed.  Nonsense article, in my opinion.  Comparing apples and oranges.

In any case, as my writing is very highly equation based, I think I'll
stick to LaTeX...

(using org, of course)

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-581-g0e52f0

Re: Efficiency of Org v. LaTeX v. Word

[Eric S Fraga]

On Saturday, 27 Dec 2014 at 04:06, Peter Neilson wrote:

[...]

> My conclusions? If your paper is trivial and you are under pressure to
> produce it quickly, then MS Word might be the best tool.

Actually, I don't think I can get any faster than using org for a
trivial paper needed quickly...  I'm in emacs already ;-)
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-581-g0e52f0

Re: Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA!

[Christophe Pouzat]

[-- Attachment #1: Type: text/plain, Size: 13854 bytes --]

Hi all,

After seeing Ken's mail:

Le 26/12/2014 23:47, Ken Mankoff a écrit :
> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.
>
> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?
>
>    -k.
>
>
and some of replies it triggered on the list, I went to check the paper. 
As many of you guys I found some "results" puzzling in particular:
1. the use of bar graphs when the data would better be displayed 
directly (that qualifies immediately the paper as "low quality" for me).
2. the larger error bars observed for LaTeX when compared to Word.
3. the systematic inverse relationship between the blue and pink bars 
heights.

So I went to figshare to download the data and looked at them. A quick 
and dirty "analysis" is attached to this mail in PDF format (generated 
with org, of course, and this awful software called LaTeX!) and the 
source org file can be found at the bottom of this mail. I used R to do 
the figures (and I'm sure the authors of the paper will then criticize 
me for not using Excel with which everyone knows errors are generated 
much more efficiently).

I managed to understand the inverse relationship in point 3 above: the 
authors considered 3 types of mistakes / errors:
1. Formatting and typos error.
2. Orthographic and grammatical errors.
3. Missing words and signs.
Clearly, following the mail of Tom (Dye) on the list and on the Plos web 
site, I would argue that formatting errors in LaTeX are bona fide bugs. 
But the point I want to make is that the third source accounts for 80% 
of the total errors (what's shown in pink bars in the paper) and clearly 
the authors counted what the subjects did not have time to type as an 
error of this type. Said differently, the blue and pink bars are showing 
systematically the same thing by construction! The second type of error 
in not a LaTeX issue (and in fact does not differ significantly from the 
Word case) but an "environment" issue (what spelling corrector had the 
LaTeX users access to?).

There is another strange thing in the table copy case. For both the 
expert and novice group in LaTeX, there is one among 10 subjects that 
did produce 0% of the table but still manage to produce 22 typographic 
errors!

The overall worst performance of LaTeX users remains to be explained and 
as mentioned in on the mails in the list, that does not make sense at 
least for the continuous text exercise. The method section of the paper 
is too vague but my guess is that some LaTeX users did attempt to 
reproduce the exact layout of the text they had to copy, something LaTeX 
is definitely not design to provide quickly.

One more point: how many of you guys could specify their total number of 
hours of experience with LaTeX (or any other software you are currently 
using)? That what the subjects of this study had to specify...

Let me know what you think,

Christophe

My org buffer:

#+TITLE: An Efficiency Comparison of Document Preparation Systems Used 
in Academic Research and Development: A Re-analysis.
#+DATE: <2014-12-28 dim.>
#+AUTHOR: Christophe Pouzat
#+EMAIL: christophe.pouzat@gmail.com
#+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline
#+OPTIONS: author:t c:nil creator:comment d:(not "LOGBOOK") date:t
#+OPTIONS: e:t email:nil f:t inline:t num:t p:nil pri:nil stat:t
#+OPTIONS: tags:t tasks:t tex:t timestamp:t toc:nil todo:t |:t
#+CREATOR: Emacs 24.4.1 (Org mode 8.2.10)
#+DESCRIPTION:
#+EXCLUDE_TAGS: noexport
#+KEYWORDS:
#+LANGUAGE: en
#+SELECT_TAGS: export
#+LaTeX_HEADER: \usepackage{alltt}
#+LaTeX_HEADER: \usepackage[usenames,dvipsnames]{xcolor}
#+LaTeX_HEADER: \renewenvironment{verbatim}{\begin{alltt} \scriptsize 
\color{Bittersweet} \vspace{0.2cm} }{\vspace{0.2cm} \end{alltt} 
\normalsize \color{black}}
#+LaTeX_HEADER: \definecolor{lightcolor}{gray}{.55}
#+LaTeX_HEADER: \definecolor{shadecolor}{gray}{.85}
#+LaTeX_HEADER: \usepackage{minted}
#+LaTeX_HEADER: \hypersetup{colorlinks=true}

#+NAME: org-latex-set-up
#+BEGIN_SRC emacs-lisp :results silent :exports none
(setq org-latex-listings 'minted)
(setq org-latex-minted-options
       '(("bgcolor" "shadecolor")
     ("fontsize" "\\scriptsize")))
(setq org-latex-pdf-process
       '("pdflatex -shell-escape -interaction nonstopmode 
-output-directory %o %f"
     "biber %b"
     "pdflatex -shell-escape -interaction nonstopmode -output-directory 
%o %f"
     "pdflatex -shell-escape -interaction nonstopmode -output-directory 
%o %f"))
#+END_SRC

* Introduction
This is a re-analysis of the data presented in 
[[http://dx.doi.org/10.1371/journal.pone.0115069][An Efficiency 
Comparison of Document Preparation Systems Used in Academic Research and 
Development]]. My "interest" in this paper was triggered by a discussion 
on the [[http://article.gmane.org/gmane.emacs.orgmode/93655][emacs org 
mode mailing list]]. Ignoring the "message" of the paper, what stroke me 
was the systematic use of bar graphs: a way of displaying data that 
*should never be used* since when many observations are considered, a 
box plot is going to do a much better job and when, like in the present 
paper, few (10 in each of the 4 categories) observations are available, 
a direct display or even a simple table is going to do a *much better* 
job. Since it turns out that the data are available both on the Plos web 
site and on 
[[http://figshare.com/articles/_An_Efficiency_Comparison_of_Document_Preparation_Systems_Used_in_Academic_Research_and_Development_/1275631][figshare]], 
I decided to re-analyze them.

* Getting the data, etc.

We get the data with:

#+BEGIN_SRC sh
wget http://files.figshare.com/1849394/S1_Materials.xlsx
#+END_SRC

#+RESULTS:
Using for instance [[http://dag.wiee.rs/home-made/unoconv/][unoconv]], 
we can convert the =Excel= file into a friendlier =csv= file:

#+BEGIN_SRC sh
unoconv -f csv S1_Materials.xlsx
#+END_SRC

#+RESULTS:
We then get the data with =R= =read.csv= function:

#+NAME: data-table
#+BEGIN_SRC R :session *R* :results silent
efficiency <- read.csv("S1_Materials.csv",header=TRUE,dec=",")
#+END_SRC
The description of this table is obtained with:

#+BEGIN_SRC sh :exports both :results output
wget http://files.figshare.com/1849395/S2_Materials.txt
cat "S2_Materials.txt"
#+END_SRC

* Making some figures
We can now make a figure out of the same data as figures 4, 5 and 6 of 
the paper but showing the actual data. We start with the "continuous 
text" exercise. We represent, in each of the four categories, each of 
the 10 individuals by a number between 0 and 9. Some horizontal jitter 
has been added to avoid overlaps. Category 1 corresponds to expert 
=Word= users; 2 to novice =Word= users; 3 to expert \LaTeX{} users; 4 to 
novice \LaTeX{} users:

#+HEADER: :file continuous.png :width 1000 :height 1000
#+BEGIN_SRC R :session *R* :exports both :results output graphics
layout(matrix(1:4,nc=2,byrow=TRUE))
par(cex=2)
plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100),
      xlab="User category",ylab="",main="Fraction of text")
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                PROZENT1[Kenntnisse==k],
                                pch = paste(0:9))))

with(efficiency,
      plot(c(1,4),c(0,100),type="n",
           xlim=c(0.5,4.5),ylim=range(FEHLERSFT),xlab="User category",
           ylab="",main="Formatting errors and typos"))
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                FEHLERSFT[Kenntnisse==k],
                                pch = paste(0:9))))

with(efficiency,
      plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
           ylim=range(FEHLEROFT),xlab="User category",ylab="",
           main="Orthographic and grammatical mistakes"))
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                FEHLEROFT[Kenntnisse==k],
                                pch = paste(0:9))))

with(efficiency,
plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLENDFT),
           xlab="User category",ylab="",main="Missing words and signs"))
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                FEHLENDFT[Kenntnisse==k],
                                pch = paste(0:9))))
#+END_SRC


Notice that the number of "missing words and signs" exactly mirrors the 
fraction of written text. We will see that this observation holds for 
the two following exercises. This "missing words and signs" is always 
roughly ten times as large as the two other sources of mistakes. This 
explains the inverse relationship between the blue and pink bars on each 
of the 3 figures.

Let's keep going with the "table exercise":

#+HEADER: :file table.png :width 1000 :height 1000
#+BEGIN_SRC R :session *R* :exports both :results output graphics
layout(matrix(1:4,nc=2,byrow=TRUE))
par(cex=2)
plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100),
      xlab="User category",ylab="",main="Fraction of text")
with(efficiency,sapply(1:4,
                        function(k) points(runif(10,k-0.2,k+0.2),
                                           PROZENT2[Kenntnisse==k],
                                           pch = paste(0:9))))

with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
                      ylim=range(FEHLERST),xlab="User category",
                      ylab="",main="Formatting errors and typos"))
with(efficiency,sapply(1:4,
                        function(k) points(runif(10,k-0.2,k+0.2),
                                           FEHLERST[Kenntnisse==k],
                                           pch = paste(0:9))))

with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
                      ylim=range(FEHLEROT),xlab="User category",
                      ylab="",main="Orthographic and grammatical mistakes"))
with(efficiency,sapply(1:4,
                        function(k) points(runif(10,k-0.2,k+0.2),
                                           FEHLEROT[Kenntnisse==k],
                                           pch = paste(0:9))))

with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
                      ylim=range(FEHLENDT),xlab="User category",ylab="",
                      main="Missing words and signs"))
with(efficiency,sapply(1:4,
                        function(k) points(runif(10,k-0.2,k+0.2),
                                           FEHLENDT[Kenntnisse==k],
                                           pch = paste(0:9))))
#+END_SRC

We also see a strange thing here: in each of the expert \LaTeX{} and the 
novice \LaTeX{} users we have one individual who did not right anything 
but still manage to produce 22 "formatting errors and typos" (!) but 
luckily no orthographic or grammatical error...

#+BEGIN_SRC R :session *R* :exports both
with(efficiency,cbind(c(PROZENT2[Kenntnisse==3][10],
                         FEHLERST[Kenntnisse==3][10],
                         FEHLEROT[Kenntnisse==3][10],
                         FEHLENDT[Kenntnisse==3][10]),
                       c(PROZENT2[Kenntnisse==4][7],
                         FEHLERST[Kenntnisse==4][7],
                         FEHLEROT[Kenntnisse==4][7],
                         FEHLENDT[Kenntnisse==4][7])))
#+END_SRC


Now for the "equations" exercise:

#+HEADER: :file equation.png :width 1000 :height 1000
#+BEGIN_SRC R :session *R* :exports both :results output graphics
layout(matrix(1:4,nc=2,byrow=TRUE))
par(cex=2)
plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100),
      xlab="User category",ylab="",main="Fraction of text")
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                PROZENT3[Kenntnisse==k],
                                pch = paste(0:9))))

with(efficiency,
      plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
           ylim=range(FEHLERSFOR),xlab="User category",ylab="",
           main="Formatting errors and typos"))
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                FEHLERSFOR[Kenntnisse==k],
                                pch = paste(0:9))))

with(efficiency,
plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLEROFOR),
           xlab="User category",ylab="",
           main="Orthographic and grammatical mistakes"))
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                FEHLEROFOR[Kenntnisse==k],
                                pch = paste(0:9))))

with(efficiency,
      plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
           ylim=range(FEHLENDFOR),xlab="User category",ylab="",
           main="Missing words and signs"))
with(efficiency,
      sapply(1:4,
             function(k) points(runif(10,k-0.2,k+0.2),
                                FEHLENDFOR[Kenntnisse==k],
                                pch = paste(0:9))))
#+END_SRC



-- 
A Master Carpenter has many tools and is expert with most of them. If you only know how to use a hammer, every problem starts to look like a nail. Stay away from that trap.

Richard B Johnson.

--

Christophe Pouzat
MAP5 - Mathématiques Appliquées à Paris 5
CNRS UMR 8145
45, rue des Saints-Pères
75006 PARIS
France

tel: +33142863828
mobile: +33662941034
web: http://xtof.disque.math.cnrs.fr


[-- Attachment #2: EfficiencyComparison.pdf --]
[-- Type: application/pdf, Size: 218418 bytes --]

Re: Efficiency of Org v. LaTeX v. Word

[Pascal Fleury]

[-- Attachment #1: Type: text/plain, Size: 4685 bytes --]

On Sat, Dec 27, 2014 at 12:36 PM, Fabrice Popineau <
fabrice.popineau@supelec.fr> wrote:

> I agree that this study is certainly not large enough to draw strong
> conclusions, but it raises a couple of questions
> and some points may require attention.
>
> I have spent many years in the TeX world. I see how lots of people use TeX
> : students, professionals, researchers etc...
> and I would easily draw 2 categories of people :
> - those who are programmers "in their soul" (DEK once said that 2% or so
> of the whole human race is gifted with programming, the same way some
> people are gifted to play music etc.)
> - those who use LaTeX "because it is the best typesetting system"
> People who belong to the intersection of those 2 categories will certainly
> be very efficient in producing documents with LaTeX, much more than what
> this study shows.
> But people from the first category may also be efficient in producing
> documents with Word (Word is programmable too and the typesetting engine is
> fancier than most people would believe).
>

That is funny, as I still face regularly Word typeset documents that do not
handle orphan lines properly, and have at least 2 fonts as "body text".
Easy to fix, but a non-issue in Latex.

As a researcher, handling references and cross-references is not something
that is "amortized" on a one-off paper, it's something that pays off over a
few documents. And in a publish-or-perish world, this does usually not take
long.

As a programmer, I like to be able to run one command (call it 'make' if
you wish...) that will run some analysis and recompute both the figures and
the document into a new version, possibly versionned.

And now you know why I use orgmode too...

--paf




> The real problem is the guys from the second category who stick to use a
> tool they are not comfortable with but they don't want to admit it.
> Over the last years, I have seen more and more students come with LaTeX
> documents which had a very poor appearance.
> There has been a lot of pressure with the rise of Linux to use LaTeX.
> Unfortunately the results of using LaTeX may not be up to the expectations.
> The tool is too complex. It can produce beautiful documents when used
> right, but it can also easily produce awful documents.
> You can also spend a lot of time in fixing details, and it happens more
> frequently than even proficient LaTeX users would admit.
> In the end, I think the tendency is to a growing number of LaTeX users who
> use it poorly.
>
> Finally, today, my experience is that publishers charge much more for
> LaTeX documents than for Word (or similar tools) documents and they are
> reluctant to use LaTeX because of its complexity.
>
> That was my $0.02
>
> Fabrice
>
> 2014-12-27 11:36 GMT+01:00 M <Elwood151@web.de>:
>
>> > Von: Paul Rudin <paul@rudin.co.uk>
>> > Datum: Sat, 27 Dec 2014 10:05:19 +0000
>> > An: <emacs-orgmode@gnu.org>
>> > Betreff: Re: [O] Efficiency of Org v. LaTeX v. Word
>> >
>> > Ken Mankoff <mankoff@gmail.com> writes:
>> >
>> >> People here might be interested in a publication from [2014-12-19 Fri]
>> >> available at http://dx.doi.org/10.1371/journal.pone.0115069
>> >>
>> >> Title: An Efficiency Comparison of Document Preparation Systems Used
>> >> in Academic Research and Development
>> >>
>> >> Summary: Word users are more efficient and have less errors than even
>> >> experienced LaTeX users.
>> >>
>> >> Someone here should repeat experiment and add Org into the mix, perhaps
>> >> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
>> >> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
>> >> pandoc) beat straight Word?
>> >>
>> >
>> > No mention of emacs... who uses anything else to prepare their LaTeX?
>> >
>> Did you forget the " ;-)" or are you serious?
>>
>> Emacs is for sure a very good one, but there are a lot of popular
>> alternatives, if you have a look at the (for sure not representative)
>> voting
>> on the answers of this discussion here:
>>
>> http://tex.stackexchange.com/questions/339/latex-editors-ides
>>
>> (It's clear, that people may have voted for several of those editors, so
>> that no valid statistics at all, but at least an idea...)
>>
>> Is there any real survey result about which editors LaTeX users use?
>>
>> Martin
>>
>>
>>
>>
>
>
> --
> Fabrice Popineau
> -----------------------------
> SUPELEC
> Département Informatique
> 3, rue Joliot Curie
> 91192 Gif/Yvette Cedex
> Tel direct : +33 (0) 169851950
> Standard : +33 (0) 169851212
> ------------------------------
>
>

[-- Attachment #2: Type: text/html, Size: 6655 bytes --]

Re: Efficiency of Org v. LaTeX v. Word

[Bob Newell]

The "study" is an obvious diatribe couched in (poorly done) scientific
method. It almost seems like these researchers have at some time been
required to use LaTeX and are angry over it.

I will agree that LaTeX is slower and less efficient than LibreOffice (I
don't have Word on any of my computers but the argument I suppose is the
same), if your only interest is pounding out text of a first draft. But
the "study" didn't allow for the effects of proofreading, etc., and all
the things that are always done when producing something for
publication. I expect that the differences in grammar and orthographic
errors will be insignificant.

A 30-minute test is ridiculous. More meaningful would have been
end-to-end time to complete a given document. Maybe then Word would have
still been faster; I don't know. But that doesn't tell the whole story
by any means, including the very important matter of long-term storage
in a non-proprietary format.

-- 
Bob Newell
Honolulu, Hawai`i
* Sent via Ma Gnus 0.12-Emacs 24.3-Linux Mint 17 *

Re: Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA!

[Thomas S. Dye]

Aloha Christophe,

I think you make a good case for the authors' poor choice of metrics.
These aren't well defined in the paper, so it is enlightening to see
your graphics and learn how their metrics were ineptly designed.

I hope you'll make your findings known to the PLOS audience.  This looks
to me like a clear case of peer review failure (see
http://www.sciencemag.org/content/342/6154/60.summary for an eye-opening
look at what's become of peer review.)

All the best,
Tom

Christophe Pouzat <christophe.pouzat@gmail.com> writes:

> Hi all,
>
> After seeing Ken's mail:
>
> Le 26/12/2014 23:47, Ken Mankoff a écrit :
>> People here might be interested in a publication from [2014-12-19 Fri]
>> available at http://dx.doi.org/10.1371/journal.pone.0115069
>>
>> Title: An Efficiency Comparison of Document Preparation Systems Used
>> in Academic Research and Development
>>
>> Summary: Word users are more efficient and have less errors than even
>> experienced LaTeX users.
>>
>> Someone here should repeat experiment and add Org into the mix, perhaps
>> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
>> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
>> pandoc) beat straight Word?
>>
>>    -k.
>>
>>
> and some of replies it triggered on the list, I went to check the paper. 
> As many of you guys I found some "results" puzzling in particular:
> 1. the use of bar graphs when the data would better be displayed 
> directly (that qualifies immediately the paper as "low quality" for me).
> 2. the larger error bars observed for LaTeX when compared to Word.
> 3. the systematic inverse relationship between the blue and pink bars 
> heights.
>
> So I went to figshare to download the data and looked at them. A quick 
> and dirty "analysis" is attached to this mail in PDF format (generated 
> with org, of course, and this awful software called LaTeX!) and the 
> source org file can be found at the bottom of this mail. I used R to do 
> the figures (and I'm sure the authors of the paper will then criticize 
> me for not using Excel with which everyone knows errors are generated 
> much more efficiently).
>
> I managed to understand the inverse relationship in point 3 above: the 
> authors considered 3 types of mistakes / errors:
> 1. Formatting and typos error.
> 2. Orthographic and grammatical errors.
> 3. Missing words and signs.
> Clearly, following the mail of Tom (Dye) on the list and on the Plos web 
> site, I would argue that formatting errors in LaTeX are bona fide bugs. 
> But the point I want to make is that the third source accounts for 80% 
> of the total errors (what's shown in pink bars in the paper) and clearly 
> the authors counted what the subjects did not have time to type as an 
> error of this type. Said differently, the blue and pink bars are showing 
> systematically the same thing by construction! The second type of error 
> in not a LaTeX issue (and in fact does not differ significantly from the 
> Word case) but an "environment" issue (what spelling corrector had the 
> LaTeX users access to?).
>
> There is another strange thing in the table copy case. For both the 
> expert and novice group in LaTeX, there is one among 10 subjects that 
> did produce 0% of the table but still manage to produce 22 typographic 
> errors!
>
> The overall worst performance of LaTeX users remains to be explained and 
> as mentioned in on the mails in the list, that does not make sense at 
> least for the continuous text exercise. The method section of the paper 
> is too vague but my guess is that some LaTeX users did attempt to 
> reproduce the exact layout of the text they had to copy, something LaTeX 
> is definitely not design to provide quickly.
>
> One more point: how many of you guys could specify their total number of 
> hours of experience with LaTeX (or any other software you are currently 
> using)? That what the subjects of this study had to specify...
>
> Let me know what you think,
>
> Christophe
>
> My org buffer:
>
> #+TITLE: An Efficiency Comparison of Document Preparation Systems Used 
> in Academic Research and Development: A Re-analysis.
> #+DATE: <2014-12-28 dim.>
> #+AUTHOR: Christophe Pouzat
> #+EMAIL: christophe.pouzat@gmail.com
> #+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline
> #+OPTIONS: author:t c:nil creator:comment d:(not "LOGBOOK") date:t
> #+OPTIONS: e:t email:nil f:t inline:t num:t p:nil pri:nil stat:t
> #+OPTIONS: tags:t tasks:t tex:t timestamp:t toc:nil todo:t |:t
> #+CREATOR: Emacs 24.4.1 (Org mode 8.2.10)
> #+DESCRIPTION:
> #+EXCLUDE_TAGS: noexport
> #+KEYWORDS:
> #+LANGUAGE: en
> #+SELECT_TAGS: export
> #+LaTeX_HEADER: \usepackage{alltt}
> #+LaTeX_HEADER: \usepackage[usenames,dvipsnames]{xcolor}
> #+LaTeX_HEADER: \renewenvironment{verbatim}{\begin{alltt} \scriptsize 
> \color{Bittersweet} \vspace{0.2cm} }{\vspace{0.2cm} \end{alltt} 
> \normalsize \color{black}}
> #+LaTeX_HEADER: \definecolor{lightcolor}{gray}{.55}
> #+LaTeX_HEADER: \definecolor{shadecolor}{gray}{.85}
> #+LaTeX_HEADER: \usepackage{minted}
> #+LaTeX_HEADER: \hypersetup{colorlinks=true}
>
> #+NAME: org-latex-set-up
> #+BEGIN_SRC emacs-lisp :results silent :exports none
> (setq org-latex-listings 'minted)
> (setq org-latex-minted-options
>        '(("bgcolor" "shadecolor")
>      ("fontsize" "\\scriptsize")))
> (setq org-latex-pdf-process
>        '("pdflatex -shell-escape -interaction nonstopmode 
> -output-directory %o %f"
>      "biber %b"
>      "pdflatex -shell-escape -interaction nonstopmode -output-directory 
> %o %f"
>      "pdflatex -shell-escape -interaction nonstopmode -output-directory 
> %o %f"))
> #+END_SRC
>
> * Introduction
> This is a re-analysis of the data presented in 
> [[http://dx.doi.org/10.1371/journal.pone.0115069][An Efficiency 
> Comparison of Document Preparation Systems Used in Academic Research and 
> Development]]. My "interest" in this paper was triggered by a discussion 
> on the [[http://article.gmane.org/gmane.emacs.orgmode/93655][emacs org 
> mode mailing list]]. Ignoring the "message" of the paper, what stroke me 
> was the systematic use of bar graphs: a way of displaying data that 
> *should never be used* since when many observations are considered, a 
> box plot is going to do a much better job and when, like in the present 
> paper, few (10 in each of the 4 categories) observations are available, 
> a direct display or even a simple table is going to do a *much better* 
> job. Since it turns out that the data are available both on the Plos web 
> site and on 
> [[http://figshare.com/articles/_An_Efficiency_Comparison_of_Document_Preparation_Systems_Used_in_Academic_Research_and_Development_/1275631][figshare]], 
> I decided to re-analyze them.
>
> * Getting the data, etc.
>
> We get the data with:
>
> #+BEGIN_SRC sh
> wget http://files.figshare.com/1849394/S1_Materials.xlsx
> #+END_SRC
>
> #+RESULTS:
> Using for instance [[http://dag.wiee.rs/home-made/unoconv/][unoconv]], 
> we can convert the =Excel= file into a friendlier =csv= file:
>
> #+BEGIN_SRC sh
> unoconv -f csv S1_Materials.xlsx
> #+END_SRC
>
> #+RESULTS:
> We then get the data with =R= =read.csv= function:
>
> #+NAME: data-table
> #+BEGIN_SRC R :session *R* :results silent
> efficiency <- read.csv("S1_Materials.csv",header=TRUE,dec=",")
> #+END_SRC
> The description of this table is obtained with:
>
> #+BEGIN_SRC sh :exports both :results output
> wget http://files.figshare.com/1849395/S2_Materials.txt
> cat "S2_Materials.txt"
> #+END_SRC
>
> * Making some figures
> We can now make a figure out of the same data as figures 4, 5 and 6 of 
> the paper but showing the actual data. We start with the "continuous 
> text" exercise. We represent, in each of the four categories, each of 
> the 10 individuals by a number between 0 and 9. Some horizontal jitter 
> has been added to avoid overlaps. Category 1 corresponds to expert 
> =Word= users; 2 to novice =Word= users; 3 to expert \LaTeX{} users; 4 to 
> novice \LaTeX{} users:
>
> #+HEADER: :file continuous.png :width 1000 :height 1000
> #+BEGIN_SRC R :session *R* :exports both :results output graphics
> layout(matrix(1:4,nc=2,byrow=TRUE))
> par(cex=2)
> plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100),
>       xlab="User category",ylab="",main="Fraction of text")
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 PROZENT1[Kenntnisse==k],
>                                 pch = paste(0:9))))
>
> with(efficiency,
>       plot(c(1,4),c(0,100),type="n",
>            xlim=c(0.5,4.5),ylim=range(FEHLERSFT),xlab="User category",
>            ylab="",main="Formatting errors and typos"))
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 FEHLERSFT[Kenntnisse==k],
>                                 pch = paste(0:9))))
>
> with(efficiency,
>       plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
>            ylim=range(FEHLEROFT),xlab="User category",ylab="",
>            main="Orthographic and grammatical mistakes"))
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 FEHLEROFT[Kenntnisse==k],
>                                 pch = paste(0:9))))
>
> with(efficiency,
> plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLENDFT),
>            xlab="User category",ylab="",main="Missing words and signs"))
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 FEHLENDFT[Kenntnisse==k],
>                                 pch = paste(0:9))))
> #+END_SRC
>
>
> Notice that the number of "missing words and signs" exactly mirrors the 
> fraction of written text. We will see that this observation holds for 
> the two following exercises. This "missing words and signs" is always 
> roughly ten times as large as the two other sources of mistakes. This 
> explains the inverse relationship between the blue and pink bars on each 
> of the 3 figures.
>
> Let's keep going with the "table exercise":
>
> #+HEADER: :file table.png :width 1000 :height 1000
> #+BEGIN_SRC R :session *R* :exports both :results output graphics
> layout(matrix(1:4,nc=2,byrow=TRUE))
> par(cex=2)
> plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100),
>       xlab="User category",ylab="",main="Fraction of text")
> with(efficiency,sapply(1:4,
>                         function(k) points(runif(10,k-0.2,k+0.2),
>                                            PROZENT2[Kenntnisse==k],
>                                            pch = paste(0:9))))
>
> with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
>                       ylim=range(FEHLERST),xlab="User category",
>                       ylab="",main="Formatting errors and typos"))
> with(efficiency,sapply(1:4,
>                         function(k) points(runif(10,k-0.2,k+0.2),
>                                            FEHLERST[Kenntnisse==k],
>                                            pch = paste(0:9))))
>
> with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
>                       ylim=range(FEHLEROT),xlab="User category",
>                       ylab="",main="Orthographic and grammatical mistakes"))
> with(efficiency,sapply(1:4,
>                         function(k) points(runif(10,k-0.2,k+0.2),
>                                            FEHLEROT[Kenntnisse==k],
>                                            pch = paste(0:9))))
>
> with(efficiency,plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
>                       ylim=range(FEHLENDT),xlab="User category",ylab="",
>                       main="Missing words and signs"))
> with(efficiency,sapply(1:4,
>                         function(k) points(runif(10,k-0.2,k+0.2),
>                                            FEHLENDT[Kenntnisse==k],
>                                            pch = paste(0:9))))
> #+END_SRC
>
> We also see a strange thing here: in each of the expert \LaTeX{} and the 
> novice \LaTeX{} users we have one individual who did not right anything 
> but still manage to produce 22 "formatting errors and typos" (!) but 
> luckily no orthographic or grammatical error...
>
> #+BEGIN_SRC R :session *R* :exports both
> with(efficiency,cbind(c(PROZENT2[Kenntnisse==3][10],
>                          FEHLERST[Kenntnisse==3][10],
>                          FEHLEROT[Kenntnisse==3][10],
>                          FEHLENDT[Kenntnisse==3][10]),
>                        c(PROZENT2[Kenntnisse==4][7],
>                          FEHLERST[Kenntnisse==4][7],
>                          FEHLEROT[Kenntnisse==4][7],
>                          FEHLENDT[Kenntnisse==4][7])))
> #+END_SRC
>
>
> Now for the "equations" exercise:
>
> #+HEADER: :file equation.png :width 1000 :height 1000
> #+BEGIN_SRC R :session *R* :exports both :results output graphics
> layout(matrix(1:4,nc=2,byrow=TRUE))
> par(cex=2)
> plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=c(0,100),
>       xlab="User category",ylab="",main="Fraction of text")
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 PROZENT3[Kenntnisse==k],
>                                 pch = paste(0:9))))
>
> with(efficiency,
>       plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
>            ylim=range(FEHLERSFOR),xlab="User category",ylab="",
>            main="Formatting errors and typos"))
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 FEHLERSFOR[Kenntnisse==k],
>                                 pch = paste(0:9))))
>
> with(efficiency,
> plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),ylim=range(FEHLEROFOR),
>            xlab="User category",ylab="",
>            main="Orthographic and grammatical mistakes"))
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 FEHLEROFOR[Kenntnisse==k],
>                                 pch = paste(0:9))))
>
> with(efficiency,
>       plot(c(1,4),c(0,100),type="n",xlim=c(0.5,4.5),
>            ylim=range(FEHLENDFOR),xlab="User category",ylab="",
>            main="Missing words and signs"))
> with(efficiency,
>       sapply(1:4,
>              function(k) points(runif(10,k-0.2,k+0.2),
>                                 FEHLENDFOR[Kenntnisse==k],
>                                 pch = paste(0:9))))
> #+END_SRC

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Efficiency of Org v. LaTeX v. Word ---LOOK AT THE DATA!

[Colin Baxter]

Dear Christophe,

Great work. You should submit it to http://www.plosone.org/ as a
response. It would be interesting to see what the Referees make of it.


Best wishes,

Colin.

> Hi all,
>
> After seeing Ken's mail:
>
> Le 26/12/2014 23:47, Ken Mankoff a écrit :
>> People here might be interested in a publication from [2014-12-19 Fri]
>> available at http://dx.doi.org/10.1371/journal.pone.0115069
>>
>> Title: An Efficiency Comparison of Document Preparation Systems Used
>> in Academic Research and Development
>>
>> Summary: Word users are more efficient and have less errors than even
>> experienced LaTeX users.
>>
>> Someone here should repeat experiment and add Org into the mix, perhaps
>> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
>> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
>> pandoc) beat straight Word?
>>
>>    -k.
>>
>>
> and some of replies it triggered on the list, I went to check the paper. 
> As many of you guys I found some "results" puzzling in particular:
> 1. the use of bar graphs when the data would better be displayed 
> directly (that qualifies immediately the paper as "low quality" for me).
> 2. the larger error bars observed for LaTeX when compared to Word.
> 3. the systematic inverse relationship between the blue and pink bars 
> heights.
>
> So I went to figshare to download the data and looked at them. A quick 
> and dirty "analysis" is attached to this mail in PDF format (generated 
> with org, of course, and this awful software called LaTeX!) and the 
> source org file can be found at the bottom of this mail. I used R to do 
> the figures (and I'm sure the authors of the paper will then criticize 
> me for not using Excel with which everyone knows errors are generated 
> much more efficiently).
>
> I managed to understand the inverse relationship in point 3 above: the 
> authors considered 3 types of mistakes / errors:
> 1. Formatting and typos error.
> 2. Orthographic and grammatical errors.
> 3. Missing words and signs.
> Clearly, following the mail of Tom (Dye) on the list and on the Plos web 
> site, I would argue that formatting errors in LaTeX are bona fide bugs. 
> But the point I want to make is that the third source accounts for 80% 
> of the total errors (what's shown in pink bars in the paper) and clearly 
> the authors counted what the subjects did not have time to type as an 
> error of this type. Said differently, the blue and pink bars are showing 
> systematically the same thing by construction! The second type of error 
> in not a LaTeX issue (and in fact does not differ significantly from the 
> Word case) but an "environment" issue (what spelling corrector had the 
> LaTeX users access to?).
>
> There is another strange thing in the table copy case. For both the 
> expert and novice group in LaTeX, there is one among 10 subjects that 
> did produce 0% of the table but still manage to produce 22 typographic 
> errors!
>
> The overall worst performance of LaTeX users remains to be explained and 
> as mentioned in on the mails in the list, that does not make sense at 
> least for the continuous text exercise. The method section of the paper 
> is too vague but my guess is that some LaTeX users did attempt to 
> reproduce the exact layout of the text they had to copy, something LaTeX 
> is definitely not design to provide quickly.
>
> One more point: how many of you guys could specify their total number of 
> hours of experience with LaTeX (or any other software you are currently 
> using)? That what the subjects of this study had to specify...
>
> Let me know what you think,
>
> Christophe
>
>
----- Snip -----

Re: Efficiency of Org v. LaTeX v. Word

[Paul Rudin]

M <Elwood151-S0/GAf8tV78@public.gmane.org> writes:

>> Von: Paul Rudin <paul-sqPYmOVXOov10XsdtD+oqA@public.gmane.org>
>> 
>> No mention of emacs... who uses anything else to prepare their LaTeX?
>> 
> Did you forget the " ;-)" or are you serious?

I wasn't being entirely serious; but I was alluding to a serious
point. You can't really compare a command line typesetting system alone
with a word processor. To make a proper comparison you'd have to look at
the complete toolchain.

For example, some of the errors are typos. Word, of course, has a speil
chucker. Did the LaTeX users use an editor that highlights such errors?

But as others have pointed out the more fundamental problem with the
study is that it tries to assess secretarial or copy-editing skills
rather than authoring skills.

(I haven't actually read the paper, just what has been said in this
thread.)

Re: Efficiency of Org v. LaTeX v. Word

[John Kitchin]

Ken Mankoff <mankoff@gmail.com> writes:

Did anyone see the parody of this here:
http://mjambon.github.io/vim-vs-emacs/

It is pretty funny!

> People here might be interested in a publication from [2014-12-19 Fri]
> available at http://dx.doi.org/10.1371/journal.pone.0115069
>
> Title: An Efficiency Comparison of Document Preparation Systems Used
> in Academic Research and Development
>
> Summary: Word users are more efficient and have less errors than even
> experienced LaTeX users.
>
> Someone here should repeat experiment and add Org into the mix, perhaps
> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
> pandoc) beat straight Word?
>
>   -k.
>
>
>

-- 
-----------------------------------
John Kitchin
Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

Re: Efficiency of Org v. LaTeX v. Word

[Andreas Leha]

Hi John,

John Kitchin <johnrkitchin@gmail.com> writes:
> Ken Mankoff <mankoff@gmail.com> writes:
>
> Did anyone see the parody of this here:
> http://mjambon.github.io/vim-vs-emacs/
>
> It is pretty funny!

Indeed.  Thanks for sharing.

Regards,
Andreas


>
>> People here might be interested in a publication from [2014-12-19 Fri]
>> available at http://dx.doi.org/10.1371/journal.pone.0115069
>>
>> Title: An Efficiency Comparison of Document Preparation Systems Used
>> in Academic Research and Development
>>
>> Summary: Word users are more efficient and have less errors than even
>> experienced LaTeX users.
>>
>> Someone here should repeat experiment and add Org into the mix, perhaps
>> Org -> ODT and/or Org -> LaTeX and see if it helps or hurts. I assume
>> Org would trump LaTeX, but would Org -> ODT or Org -> X -> DOCX (via
>> pandoc) beat straight Word?
>>
>>   -k.
>>
>>
>>