Org-mode to feed a database

Org-mode to feed a database

[Vikas Rawal]

This is, in all likelihood, a crazy idea. Please excuse me for that.

I often need to get a number of people enter complex survey data into a database using some kind of forms. The forms I need are fairly complex (with several nested grids, for example). I need radio-buttons and drop-down menus-equivalents.

I was thinking if it would be worth it to use Org-mode for data entry.

The idea would be something like this. We have a template org file that is copied for each survey respondent. The template contains “blank” org tables and some source blocks. Responses are filled into the org tables, and then the source blocks are evaluated to feed the data into the database. You can version-control the whole thing so that different people can create these records and push it to a centralised git repository.

Has anyone done anything like this? What has been the experience?

There are two aspects that immediately come to my mind.

1. Is org adequately feature rich for such a scenario? There are two concerns I can think of right now. 
   1.1 Would feeding information from one Org table to another become too complicated?
   1.2 Is there a way to build in tab-completion for columns in Org-table using pre-specified categories? 

2. In my case, people who would enter the data into org files would not have used emacs before. So, there is a learning curve. Is it worth it? Or should one just build a web-based graphic application for such people to use?

Vikas

Re: Org-mode to feed a database

[Marcin Borkowski]

On 2014-12-29, at 03:01, Vikas Rawal <vikaslists@agrarianresearch.org> wrote:

> This is, in all likelihood, a crazy idea. Please excuse me for that.

No.

Have you considered the Emacs Widget library?

HTH,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Org-mode to feed a database

[Vikas Rawal]

> 
> Have you considered the Emacs Widget library?
> 


Thanks. Did not know about this. Very interesting.

Vikas

Re: Org-mode to feed a database

[Nick Dokos]

Vikas Rawal <vikaslists@agrarianresearch.org> writes:

> This is, in all likelihood, a crazy idea. Please excuse me for that.
>
> I often need to get a number of people enter complex survey data into
> a database using some kind of forms. The forms I need are fairly
> complex (with several nested grids, for example). I need radio-buttons
> and drop-down menus-equivalents.
>
> I was thinking if it would be worth it to use Org-mode for data entry.
>
> The idea would be something like this. We have a template org file
> that is copied for each survey respondent. The template contains
> “blank” org tables and some source blocks. Responses are filled into
> the org tables, and then the source blocks are evaluated to feed the
> data into the database. You can version-control the whole thing so
> that different people can create these records and push it to a
> centralised git repository.
>
> Has anyone done anything like this? What has been the experience?
>
> There are two aspects that immediately come to my mind.
>
> 1. Is org adequately feature rich for such a scenario? There are two concerns I can think of right now. 
>    1.1 Would feeding information from one Org table to another become too complicated?
>    1.2 Is there a way to build in tab-completion for columns in Org-table using pre-specified categories? 
>
> 2. In my case, people who would enter the data into org files would
> not have used emacs before. So, there is a learning curve. Is it worth
> it? Or should one just build a web-based graphic application for such
> people to use?
>

My knee-jerk reaction was: Use Django, not org - but that may be a
failure of imagination on my part.

-- 
Nick

Re: Org-mode to feed a database

[Vikas Rawal]

>> 
>> 2. In my case, people who would enter the data into org files would
>> not have used emacs before. So, there is a learning curve. Is it worth
>> it? Or should one just build a web-based graphic application for such
>> people to use?
>> 
> 
> My knee-jerk reaction was: Use Django, not org - but that may be a
> failure of imagination on my part.
> 


You are perhaps right. I will perhaps end up using Django.

But there would be many interesting things one could do in an emacs/org-based solution. My  biggest worry is that it would require continuous training of people coming in to enter data.

Vikas

Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Karl Voit]

* Vikas Rawal <vikaslists@agrarianresearch.org> wrote:
>
> But there would be many interesting things one could do in an
> emacs/org-based solution. My  biggest worry is that it would
> require continuous training of people coming in to enter data.

I think you can differ between the data entry platform and the
platform where you analyze the data.

You do have to train people using Emacs/Org-mode as well.

When I was a researcher, I used SurveyMonkey for my surveys. It's
open source and web-based. You get the data in CSV files AFAIR.

I also tested Google Drive (spreadsheet) forms which were very easy
to set up. However, the person filling out the survey needs a Google
Drive account. The results were exported in various formats AFAIR.

So, choosing a well known interface type (web-based forms) for
end-users and choosing a separate power-tool for analysis (R,
Org-mode, ...) is not a bad idea IMHO.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Ista Zahn]

[-- Attachment #1: Type: text/plain, Size: 1625 bytes --]

On Dec 31, 2014 5:38 AM, "Karl Voit" <devnull@karl-voit.at> wrote:
>
> * Vikas Rawal <vikaslists@agrarianresearch.org> wrote:
> >
> > But there would be many interesting things one could do in an
> > emacs/org-based solution. My  biggest worry is that it would
> > require continuous training of people coming in to enter data.
>
> I think you can differ between the data entry platform and the
> platform where you analyze the data.
>
> You do have to train people using Emacs/Org-mode as well.
>
> When I was a researcher, I used SurveyMonkey for my surveys. It's
> open source and web-based. You get the data in CSV files AFAIR.

No, I don't believe surveymonkey is open source. Perhaps you are thinking
of limesurvey?

>
> I also tested Google Drive (spreadsheet) forms which were very easy
> to set up. However, the person filling out the survey needs a Google
> Drive account.

No, that is not true. You don't need a google account to fill in a Google
drive form.

The results were exported in various formats AFAIR.
>
> So, choosing a well known interface type (web-based forms) for
> end-users and choosing a separate power-tool for analysis (R,
> Org-mode, ...) is not a bad idea IMHO.

I agree that doing it in emacs doesn't make sense from a productivity
standpoint. The only reason to do something like this in emacs is because
it would be cool and increase your geek cred.

Best,
Ista

>
> --
> mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
>        > get Memacs from https://github.com/novoid/Memacs <
>
> https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on
github
>
>

[-- Attachment #2: Type: text/html, Size: 2289 bytes --]

Re: Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Karl Voit]

* Ista Zahn <istazahn@gmail.com> wrote:
>
> On Dec 31, 2014 5:38 AM, "Karl Voit" <devnull@karl-voit.at> wrote:
>>
>> When I was a researcher, I used SurveyMonkey for my surveys. It's
>> open source and web-based. You get the data in CSV files AFAIR.
>
> No, I don't believe surveymonkey is open source. Perhaps you are thinking
> of limesurvey?

Yes, absolutely true. I mixed them up - sorry for that. It was
https://www.limesurvey.org with a local installation.

>> I also tested Google Drive (spreadsheet) forms which were very easy
>> to set up. However, the person filling out the survey needs a Google
>> Drive account.
>
> No, that is not true. You don't need a google account to fill in a Google
> drive form.

Really? I was not able to do so. Probably I have to re-try it again.

In case this is true, then I'd got for Google Drive forms because of
the very easy handling!

> The results were exported in various formats AFAIR.

... and I guess everything we need here is CSV :-)


Thanks for your correction!

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Ista Zahn]

On Wed, Dec 31, 2014 at 8:40 AM, Karl Voit <devnull@karl-voit.at> wrote:
> * Ista Zahn <istazahn@gmail.com> wrote:
>>
>> On Dec 31, 2014 5:38 AM, "Karl Voit" <devnull@karl-voit.at> wrote:
>>>
>>> When I was a researcher, I used SurveyMonkey for my surveys. It's
>>> open source and web-based. You get the data in CSV files AFAIR.
>>
>> No, I don't believe surveymonkey is open source. Perhaps you are thinking
>> of limesurvey?
>
> Yes, absolutely true. I mixed them up - sorry for that. It was
> https://www.limesurvey.org with a local installation.
>
>>> I also tested Google Drive (spreadsheet) forms which were very easy
>>> to set up. However, the person filling out the survey needs a Google
>>> Drive account.
>>
>> No, that is not true. You don't need a google account to fill in a Google
>> drive form.
>
> Really? I was not able to do so. Probably I have to re-try it again.

I've never had a problem with this--for example http://j.mp/1zvWP1G
should work without signing into google.

>
> In case this is true, then I'd got for Google Drive forms because of
> the very easy handling!

Yes, google is very convenient.

>
>> The results were exported in various formats AFAIR.
>
> ... and I guess everything we need here is CSV :-)
>
>
> Thanks for your correction!
>
> --
> mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
>        > get Memacs from https://github.com/novoid/Memacs <
>
> https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github
>
>

Re: Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Marcin Borkowski]

On 2014-12-31, at 14:49, Ista Zahn <istazahn@gmail.com> wrote:

> Yes, google is very convenient.

That's why they're so dangerous.

Regards,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Karl Voit]

* Marcin Borkowski <mbork@wmi.amu.edu.pl> wrote:
>
> On 2014-12-31, at 14:49, Ista Zahn <istazahn@gmail.com> wrote:
>
>> Yes, google is very convenient.
>
> That's why they're so dangerous.

In case you're survey data contains privacy-related sensible data,
you can not use Google anyhow - I do think that there's no need to
mention this explicitly in this community.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Conducting end user surveys and analyze data (was: Org-mode to feed a database)

[Marcin Borkowski]

On 2015-01-01, at 17:22, Karl Voit <devnull@Karl-Voit.at> wrote:

> * Marcin Borkowski <mbork@wmi.amu.edu.pl> wrote:
>>
>> On 2014-12-31, at 14:49, Ista Zahn <istazahn@gmail.com> wrote:
>>
>>> Yes, google is very convenient.
>>
>> That's why they're so dangerous.
>
> In case you're survey data contains privacy-related sensible data,
> you can not use Google anyhow - I do think that there's no need to
> mention this explicitly in this community.

Yes, but not only this.  You get accustomed to a nice service, like
Google Reader, and then boom! and it disappears.  How could one rely on
Gmail now?

(Disclaimer: I use neither, i.e., I /do/ use Gmail, but not for critical
stuff.  Anecdote: I used to use my private mail account for one
particular purpose, but when people I exchanged email with started to
send me scanned documents in pretty high resolution /and in bmp format/,
I decided it's time to switch to Gmail...)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

OT: Cloud-dependencies, privacy, decentralization (was: Conducting end user surveys and analyze data)

[Karl Voit]

* Marcin Borkowski <mbork@wmi.amu.edu.pl> wrote:
>
> On 2015-01-01, at 17:22, Karl Voit <devnull@Karl-Voit.at> wrote:
>
>> * Marcin Borkowski <mbork@wmi.amu.edu.pl> wrote:
>>>
>>> On 2014-12-31, at 14:49, Ista Zahn <istazahn@gmail.com> wrote:
>>>
>>>> Yes, google is very convenient.
>>> That's why they're so dangerous.
>>
>> In case you're survey data contains privacy-related sensible data,
>> you can not use Google anyhow - I do think that there's no need to
>> mention this explicitly in this community.
>
> Yes, but not only this.  You get accustomed to a nice service, like
> Google Reader, and then boom! and it disappears.  

I am 100% on your side.

In my opinion, you should avoid cloud/vendor lock-in as much as
possible wherever you can even with additional effort. You'll be
rewarded on the long run. 

Org-mode is able to replace *many* things including some fancy cloud
services like Evernote.

> How could one rely on Gmail now?

Even I do think that Gmail will never be discontinued (in general)
as long as Google will exist. It will be changed in a way that you
are not going to like - this is for sure. But Gmail is going down
not before the day that Google will go down. And I do also think
that we are going to see both days in the next twenty years.

However, I also grieve for Google Reader, Google Code Search, Google
Wave and lots of other discontinued services I was investing my
time. I learned from these mistakes. Most people don't.

> (Disclaimer: I use neither, i.e., I /do/ use Gmail, but not for critical
> stuff.  Anecdote: I used to use my private mail account for one
> particular purpose, but when people I exchanged email with started to
> send me scanned documents in pretty high resolution /and in bmp format/,
> I decided it's time to switch to Gmail...)

I do not use any Google service at all besides Google search
which is giving me better search results than DuchDuckGo -
unfortunately.

And no cloud service is ever going to get my complete set of emails.
This is *way* too sensible in my opinion.

Although I have to confess that GMail offers the best mail
experience out there, I stick with my ssh+mutt+gvim setup. It works
quite well for me and I can do tons of stuff that Gmail (or any
other cloud service or MUA) will never be able to deliver. 

For example, my mail address book is generated from my org-contacts
file. And the same goes for my mail filters (whitelisting) and
from-addresses (I am using different addresses for different
purposes). This flexibility is not possible when you've got a
monolithic MUA with no open interfaces.

This is no strategy for an average computer user - I agree. But hey,
I want to get advantages out of 25 years of learning IT :-)

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: OT: Cloud-dependencies, privacy, decentralization

[Rasmus]

Karl Voit <devnull@Karl-Voit.at> writes:

> I do not use any Google service at all besides Google search
> which is giving me better search results than DuchDuckGo -
> unfortunately.

Did you try startpage.com?  It can also be searched directly from DDG, I think.

—Rasmus

-- 
With monopolies the cake is a lie!

OT: privacy-aware web search via startpage.com (was: OT: Cloud-dependencies, privacy, decentralization)

[Karl Voit]

* Rasmus <rasmus@gmx.us> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> I do not use any Google service at all besides Google search
>> which is giving me better search results than DuchDuckGo -
>> unfortunately.
>
> Did you try startpage.com?  It can also be searched directly from DDG, I think.

Oh, this looks great - thanks!

I tested Ixquick a couple of years ago but it wasn't that good back
then.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: OT: Cloud-dependencies, privacy, decentralization (was: Conducting end user surveys and analyze data)

[Tom Prince]

Karl Voit <devnull@Karl-Voit.at> writes:
> And no cloud service is ever going to get my complete set of emails.
> This is *way* too sensible in my opinion.

What percentange of your contact list is using gmail?

Re: OT: Cloud-dependencies, privacy, decentralization

[Karl Voit]

* Tom Prince <tom.prince@ualberta.net> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>> And no cloud service is ever going to get my complete set of emails.
>> This is *way* too sensible in my opinion.
>
> What percentange of your contact list is using gmail?

Out of 672 active contacts, there are 39 contacts that use gmail. So
this is less than six percent.

However, I do not tend to think that this is relevant. Even if this
percentage is much higher, I am not going to give away sensible data
to cloud companies located in the US or even in the EU.

With "complete set of emails" I was referring to the contents of
every email. If agencies want to harvest them, they have to invest
work and effort to do so. I am not going to do their work.

To keep that awareness, I do have configured my MUA to highlight the
mail addresses that are hosted at Gmail and such. I configured the
MUA of my parents to send only GnuPG encrypted emails to my email
address. I switched text messages on my phone to TextSecure even if
only a fraction of my contacts are using TextSecure by themselves. I
do not send super-sensitive information via unencrypted email at all
any more.

The situation is pretty fucked up but there is still something
everybody is able to do against it. The fact that agencies are
able to get anything they want if they really want it should not
lead to mind-sets that say that nothing should/can be done at all.

Just because there are professional thieves does not imply that you
stop locking your door and shutting your windows before leaving your
house. This also is true when you're living in a country where
almost nobody is locking their doors when they're at home - just as
it is here, where I live.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Conducting end user surveys and analyze data

[Paul Rudin]

Marcin Borkowski <mbork-12VZH/wba7B4rM3dGMyr8Q@public.gmane.org> writes:

> On 2015-01-01, at 17:22, Karl Voit <devnull-lKKztaIpiO84+ND2ZU0ZlA@public.gmane.org> wrote:
>
>> * Marcin Borkowski <mbork-12VZH/wba7B4rM3dGMyr8Q@public.gmane.org> wrote:
>>>
>>> On 2014-12-31, at 14:49, Ista Zahn <istazahn-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>
>>>> Yes, google is very convenient.
>>>
>>> That's why they're so dangerous.
>>
>> In case you're survey data contains privacy-related sensible data,
>> you can not use Google anyhow - I do think that there's no need to
>> mention this explicitly in this community.
>
> Yes, but not only this.  You get accustomed to a nice service, like
> Google Reader, and then boom! and it disappears.  How could one rely on
> Gmail now?
>

Well - you can't rely on anyone else beyond that which they're
contractually (or statutorily) obliged to provide (and even then they
might go out of business). But you can pull all your email out of gmail
via pop or imap, so it's not like you'll lose your emails. At worst
you'd lose your email address (although if you register a domain name
then you can keep your email address and still use gmail for as long as
it's there and some other provider later).

On the privacy point, you can send encrypted data via non-secure email
providers.

OT: Gmail and cloud-dependencies (was: Conducting end user surveys and analyze data)

[Karl Voit]

* Paul Rudin <paul@rudin.co.uk> wrote:
>
> But you can pull all your email out of gmail via pop or imap, so
> it's not like you'll lose your emails. 

There are cases where people lost access to Gmail instantly because
of false accusations or idendity theft. You have to prepare *now*
for not losing yous emails if you don't synchronize often.

> At worst you'd lose your email address (although if you register a
> domain name then you can keep your email address and still use
> gmail for as long as it's there and some other provider later).

You'll lose some more. At least you are going to lose your labels
AFAIR.

> On the privacy point, you can send encrypted data via non-secure
> email providers.

But that's even more pain in the a**.

Besides the point that you'll lose search functionality on Gmail,
you have to find a reliable system that supports you with de- and
encryption. Then you have to trust those tools which are often
browser-based. I would not trust web-browser-modules and
web-browsers at all. Those tools will never see my private GnuPG
key.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: OT: Gmail and cloud-dependencies

[Paul Rudin]

Karl Voit <devnull-lKKztaIpiO84+ND2ZU0ZlA@public.gmane.org> writes:

> * Paul Rudin <paul-sqPYmOVXOov10XsdtD+oqA@public.gmane.org> wrote:
>>
>> But you can pull all your email out of gmail via pop or imap, so
>> it's not like you'll lose your emails. 
>
> There are cases where people lost access to Gmail instantly because
> of false accusations or idendity theft. You have to prepare *now*
> for not losing yous emails if you don't synchronize often.
>
Sure, but that kind of thing could happen with pretty much any third
party service.

>> At worst you'd lose your email address (although if you register a
>> domain name then you can keep your email address and still use
>> gmail for as long as it's there and some other provider later).
>
> You'll lose some more. At least you are going to lose your labels
> AFAIR.
>

Labels map to mailboxes if you access your gmail via imap IIRC, so you
don't really lose that information.

Re: OT: Gmail and cloud-dependencies

[Karl Voit]

* Paul Rudin <paul@rudin.co.uk> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> * Paul Rudin <paul@rudin.co.uk> wrote:
>>>
>>> But you can pull all your email out of gmail via pop or imap, so
>>> it's not like you'll lose your emails. 
>>
>> There are cases where people lost access to Gmail instantly because
>> of false accusations or idendity theft. You have to prepare *now*
>> for not losing yous emails if you don't synchronize often.
>>
> Sure, but that kind of thing could happen with pretty much any third
> party service.

Yes. That's the point, when you do not consider open source services
as third party services.

>>> At worst you'd lose your email address (although if you register a
>>> domain name then you can keep your email address and still use
>>> gmail for as long as it's there and some other provider later).
>>
>> You'll lose some more. At least you are going to lose your labels
>> AFAIR.
>
> Labels map to mailboxes if you access your gmail via imap IIRC, so you
> don't really lose that information.

AFAIR do you lose information for sure: you can't have combinations
of labels (e.g., +important+business) as mailboxes and you can't
export this information in a meaningful way for migration.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: OT: Gmail and cloud-dependencies

[Paul Rudin]

Karl Voit <devnull-lKKztaIpiO84+ND2ZU0ZlA@public.gmane.org> writes:

> * Paul Rudin <paul-sqPYmOVXOov10XsdtD+oqA@public.gmane.org> wrote:
>> Karl Voit <devnull-lKKztaIpiO84+ND2ZU0ZlA@public.gmane.org> writes:
>>
>>> * Paul Rudin <paul-sqPYmOVXOov10XsdtD+oqA@public.gmane.org> wrote:
>>>>
>>>> But you can pull all your email out of gmail via pop or imap, so
>>>> it's not like you'll lose your emails. 
>>>
>>> There are cases where people lost access to Gmail instantly because
>>> of false accusations or idendity theft. You have to prepare *now*
>>> for not losing yous emails if you don't synchronize often.
>>>
>> Sure, but that kind of thing could happen with pretty much any third
>> party service.
>
> Yes. That's the point, when you do not consider open source services
> as third party services.

Aren't you mixing up two different things here? Suppose google were to
open source all the code behind gmail. That wouldn't really change the
situation for users if the service suddenly disappeared.

Re: OT: Gmail and cloud-dependencies

[Karl Voit]

* Paul Rudin <paul@rudin.co.uk> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> * Paul Rudin <paul@rudin.co.uk> wrote:
>>>
>>> Sure, but that kind of thing could happen with pretty much any third
>>> party service.
>>
>> Yes. That's the point, when you do not consider open source services
>> as third party services.
>
> Aren't you mixing up two different things here? Suppose google were to
> open source all the code behind gmail. That wouldn't really change the
> situation for users if the service suddenly disappeared.

Oh, I disagree. I think that this would change a *lot*. Consider the
possibility that I can select who is hosting the service and my
data. As a side-effect: my data is not locked in at all. I can
backup/migrate whenever/wherever I want.

Believe it or not: the mail system is meant to be like that with
good reason :-)

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: OT: Gmail and cloud-dependencies

[Daniele Pizzolli]

Karl Voit writes:

> * Paul Rudin wrote:
>> Karl Voit writes:
>>
>
>>> * Paul Rudin wrote:

[]

>>> You'll lose some more. At least you are going to lose your labels
>>> AFAIR.
>>
>> Labels map to mailboxes if you access your gmail via imap IIRC, so you
>> don't really lose that information.
>
> AFAIR do you lose information for sure: you can't have combinations
> of labels (e.g., +important+business) as mailboxes and you can't
> export this information in a meaningful way for migration.

Hello,

I never used that but, offlineimap seems to be able to sync the labels
using a custom(?) message header. See:

https://github.com/OfflineIMAP/offlineimap/blob/fc4c7549d6d03415edc2537b29c83fd6baea1bc6/offlineimap/folder/Gmail.py#L67

I am not sure if that is really meaningful.  Maybe mail clients like
mu4e can be used to search this header in a way easy to use.

Best,
Daniele

Re: Org-mode to feed a database

[Karl Voit]

* Vikas Rawal <vikaslists@agrarianresearch.org> wrote:
> This is, in all likelihood, a crazy idea. Please excuse me for that.

Hehe.

> I often need to get a number of people enter complex survey data
> into a database using some kind of forms. The forms I need are
> fairly complex (with several nested grids, for example). I need
> radio-buttons and drop-down menus-equivalents.
>
> I was thinking if it would be worth it to use Org-mode for data
> entry.
>
> The idea would be something like this. We have a template org file
> that is copied for each survey respondent. The template contains
> “blank” org tables and some source blocks. Responses are filled
> into the org tables, and then the source blocks are evaluated to
> feed the data into the database. You can version-control the whole
> thing so that different people can create these records and push
> it to a centralised git repository.

Never done this.

But if I'd do this, I'd definitely take a look at yasnippet and
using it for default values, dropdowns, ...

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Org-mode to feed a database

[John Kitchin]

Vikas Rawal <vikaslists@agrarianresearch.org> writes:

It is possible, but not worth the effort.

Even with moderately experienced users, you will be amazed at how
org-files can get mangled by users. In a recent course I taught, I used
links for some data entry; the links stored data as properties in a
headline. Even then a handful of users found some ways to delete
headlines, modify what got stored, etc...

As someone else mentioned, if Emacs is the input interface, widgets is
the way to go. Otherwise, a webform is a much better idea. Alternatives
include a PDF form, an MS Word or Excel form, or one of the online
survey tools.

> This is, in all likelihood, a crazy idea. Please excuse me for that.
>
> I often need to get a number of people enter complex survey data into
> a database using some kind of forms. The forms I need are fairly
> complex (with several nested grids, for example). I need radio-buttons
> and drop-down menus-equivalents.
>
> I was thinking if it would be worth it to use Org-mode for data entry.
>
> The idea would be something like this. We have a template org file
> that is copied for each survey respondent. The template contains
> “blank” org tables and some source blocks. Responses are filled into
> the org tables, and then the source blocks are evaluated to feed the
> data into the database. You can version-control the whole thing so
> that different people can create these records and push it to a
> centralised git repository.
>
> Has anyone done anything like this? What has been the experience?
>
> There are two aspects that immediately come to my mind.
>
> 1. Is org adequately feature rich for such a scenario? There are two concerns I can think of right now. 
>    1.1 Would feeding information from one Org table to another become too complicated?
>    1.2 Is there a way to build in tab-completion for columns in Org-table using pre-specified categories? 
>
> 2. In my case, people who would enter the data into org files would
> not have used emacs before. So, there is a learning curve. Is it worth
> it? Or should one just build a web-based graphic application for such
> people to use?
>
> Vikas
>

-- 
-----------------------------------
John Kitchin
Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

Re: Org-mode to feed a database

[Vikas Rawal]

> 
> It is possible, but not worth the effort.
> 
> Even with moderately experienced users, you will be amazed at how
> org-files can get mangled by users. In a recent course I taught, I used
> links for some data entry; the links stored data as properties in a
> headline. Even then a handful of users found some ways to delete
> headlines, modify what got stored, etc...

Thanks, John.

Vikas