Capturing Reference Material for Scientific Articles Written in Orgmode

Capturing Reference Material for Scientific Articles Written in Orgmode

[Sven Bretfeld]

Hi to all

For those who are interested.

[x] Do you write scientific articles in Orgmode?
[x] Do you collect data (possible citations, ideas etc.) while reading
    books?
[x] Do you own an Android phone?

I just want to share a convenient method to collect scientific reference
material with those who fulfill the points above.

When I'm doing research for a paper or lecture, I use to collect
reference material by photocopying parts of books or papers and labeling
them according to my purposes. I follow a method similar to the famous
"Zettelkasten" of Niklas Luhmann (I think there is no English equivalent
for "Zettelkasten", it's a paper-based filing and reference system for
citations, titles etc.). When I'm actually writing, I usually spread
these "Zettel" (slips of paper) on the floor and start arranging them
according to the argumentation line I want to develop and formulate.

Since I write my stuff with Orgmode, I thought it would be handy to do
this arrangement directly within the org-file of the paper I'm about to
write. But, how to do this without transcribing the needed passages, and
how to capture them where ever you are reading your books (train,
library, toilet)?

I own an Android phone, and I have installed a free app called
"Scan2PDF". With Scan2PDF you can take a photocopy of a page in a book
and send as a pdf-attachment to your mail address. The quality is
sufficient. I wrote a shellscript that directly moves the
mail-attachment to a predefined folder associated with the project and
automatically creates an item and a link within the org-file of the
paper I'm working on.

This can be further automatized when running the script as a cronjob or
as a procmail recipe. So, I can sit in a train, read a book, and create
plenty of pdfs of the interesting passages. When I'm home again, these
files are already linked to my paper and I can start re-reading and
arranging them.

Here is the script:

--8<---------------cut here---------------start------------->8---
#!/bin/bash

cd /home/sven/Mail/capture
for i in $( ls ~/Mail/capture/ ); do
    PDFNAME="$(cat $i | grep ^Subject: | sed -e 's/Subject: //g' | sed -e 's/.*_//g')"
    PROJECT="$(cat $i | grep ^Subject: | sed -e 's/Subject: //g' | sed -e 's/_.*//g')"
    PDFDIR="/home/sven/documents/reference/$PROJECT"
    PAGE="$(cat $i | grep ^\<p\>[0-9]* | sed -e 's/<p>//g' | sed -e 's=</p>==g' | sed -e 's=<br>==g')"
    TITLE="$(cat $i | grep ^.*\<\/p\> | sed -e 's=</p>==g')"
    if [ ! -d $PDFDIR ]
    then
    	mkdir $PDFDIR
    fi
    if [ ! -f /home/sven/documents/reference/$PROJECT.org ]
    then
	echo "* References" > /home/sven/documents/reference/$PROJECT.org
    fi
    munpack $i
    mv ~/Mail/capture/file*.pdf $PDFDIR/$PDFNAME:$PAGE.pdf
    sed -i "s=* References=* References\n** $TITLE\n\t [[file:$PDFDIR/$PDFNAME:$PAGE.pdf][$PDFNAME:$PAGE]]=g" /home/sven/documents/reference/$PROJECT.org
    rm $i
done
--8<---------------cut here---------------end--------------->8---

For example, you are sitting in a train reading a book by Niklas Luhmann
published in 1978. You are reading this book for a paper you are working
on. Now, you find an interesting passage about the self-referentiality
of systems. You copy this page with your phone and mail the pdf to
yourself.

At home, the Email with the pdf-attachment is fetched by fetchmail and
placed in a maildir folder "~/Mail/capture/" by procmail. The script is
started once in an hour via cron. It reads the Subject which has to
contain the project title and a key that represents the title of the
book/article you were reading, separated by an understroke:

,----
| mypaper_luhmann1978
`----

This can be configured within Scan2PDF as a standard header as long as
you are reading the respective book/article. If you change to another
reading, just change the header configuration accordingly.

The text-body of the Email contains the pagenumber of the reference and
a short description (which will be the headline of the org-entry):

,----
| 43
| Systems are self-referencial
`----

The effect is:

1. If not extant, the script creates a folder "mypaper" under
   ~/documents/references. 
2. The attached photocopy of the page is detached from the mail[¹] and moved to
   this folder as luhmann:43.pdf
3. A link will be created in ~/documents/reference/mypaper.org, looking
   like this[²]:

,----
| * References
| ** Systems are self-referential
|    [[file:~/documents/references/mypaper/luhmann1978:43.pdf][luhman1978:43]]
`----

4. The Email is deleted.

You can capture plenty of those references within a short time. All you
have to do is to take the photo and to type in the pagenumber and a
short description. The rest will be done by Scan2PDF and the script.

Thereafter you can move the org-entry with the link where ever you need
it inside your paper (the "arrangement-on-the-floor phase"). Just click
on the link and the file opens, so you can read it again and use its
contents as a citation, inspiration, reminder or whatever.

I'm not a programmer, so the script is awkward for sure, but it works. I
hope that someone finds it useful.

Greetings,

Sven

[¹]  The program 'munpack' has to be installed on the system.

[²] At the moment I use a separate org-file for the captured references,
not the main file of the project itself, until I trust the procedure
fully.

Re: Capturing Reference Material for Scientific Articles Written in Orgmode

[Eric S Fraga]

On 17 Apr 2010 18:47:31 +0200, "Sven Bretfeld" <sven.bretfeld@gmx.ch> wrote:
> 
> Hi to all
> 
> For those who are interested.
> 
> [x] Do you write scientific articles in Orgmode?
> [x] Do you collect data (possible citations, ideas etc.) while reading
>     books?
> [x] Do you own an Android phone?

Yes, no and no! ;-)  (mind you, I'm looking at getting an Android
phone when my current contract expires in 3 months...)

Most of my research reading is online with electronic journals so I
use a combination of the firefox org-mode links and zotero.  What
would be useful to me, if anybody has looked at this, is a link
between zotero and org-mode!  It would be great to have an automated
procedure which creates an org file which maps to the zotero
database.