Ignore pattern when syncing Org-mode files between hosts

Ignore pattern when syncing Org-mode files between hosts

[Karl Voit]

Hi!

I wonder how others are handling synchronization ignore patterns
between different hosts.

In particular, I do have "issues" with temporary files of Emacs +
Org-mode like «.#myfile.org» or «#myfile.org#». 

The first one is a symlink which does not bother me as long as I
keep quitting my Emacs instances when leaving a host and as long as
Windows sync does not result in an error because of not properly
handling symlinks. So lets forget about them until someone points me
to a potential issue.

The latter one is an autosave file which takes as much space as the
original file except for the changes in between. This is causing
many MBs of unnecessary synchronization data.


So far, I was using unison file synchronizer which I invoked
manually before shutting down the host I was working with. Therefore
there were no temp files left since I had to save and quit Emacs. A
clean status of my Org-mode files was transferred, no unwanted temp
files synced.

Currently, I am evaluating Syncthing[0] which looks very good so
far. It is a game-changer to be because similar to Dropbox[1], it
synchronizes in background on any change within 60s (by default).
Thus, resulting in symlinks and tempfiles transferred as well.

On the one hand side, I clearly don't want those temp files
synchronized to other hosts because my Org-mode files tend to get
very large[2].  A short test showed that before and after invoking
save-some-buffers() autosave files of about 10MB of data got
deleted. This data was transferred beforehand resulting in lots of
sync overhead I want to avoid.

On the other hand side, I do want those temp files synced because
they contain unsaved changes I'd lose when starting to work in a
different host before saving/quitting Emacs.

I had mixed results when I tried real-auto-save[3] a year ago. This
would be a method to make sure that there are no unsaved changes
older than xx seconds. Somehow, this was not working to my
satisfaction so I had to disable it again.


So how do you handle ignore patterns with sync tools like Dropbox or
Syncthing?


[0] https://syncthing.net/

[1] I'll never trust my (unencrypted) Org-mode files be placed in
    the public cloud like Dropbox or similar.

[2] Several of my Org-mode files holds a couple of MBs of data.

[3] from https://www.emacswiki.org/emacs/AutoSave - seems to be down

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Ignore pattern when syncing Org-mode files between hosts

[Skip Collins]

On Wed, Dec 28, 2016 at 10:42 AM, Karl Voit <devnull@karl-voit.at> wrote:
> I wonder how others are handling synchronization ignore patterns
> between different hosts.

Perhaps you should consider git annex. It is extremely flexible about
choosing which file to sync. It works cross-platform and is highly
configurable. It allows manual syncing or automatic syncing (via 'git
annex assistant'). It uses all the power of git to ensure the safety
and availability of your files, including the ability to use strong
encryption if desired.

Re: Ignore pattern when syncing Org-mode files between hosts

[Karl Voit]

Hi Skip,

* Skip Collins <skip.collins@gmail.com> wrote:
> On Wed, Dec 28, 2016 at 10:42 AM, Karl Voit <devnull@karl-voit.at> wrote:
>> I wonder how others are handling synchronization ignore patterns
>> between different hosts.
>
> Perhaps you should consider git annex. It is extremely flexible about
> choosing which file to sync. 

I know git-annex. It is very powerful, yes. BTDT. I don't want that:
I don't want to have a central server. I don't want versioning on
all files. I doubt the performance and efficiency with binaries in
git. I consider myself rather advanced in IT but the complexity of
git-annex is a bit scary to me.

YMMV and this is OK.

Syncthing seems to be exactly the tool of choice for me for now:
very simple to set up, nice and simple clients for every important
platform[1] and yet powerful enough in case I need anything advanced.

BUT:

My email was not about «what is a sync tool that allows for complex
ignore patterns» but rather «how are you using your ignore patterns
with Org-mode files to avoid issues».

So: do you sync everything of your Org-mode directory? 

If not: what are you ignoring? 

What is your experience especially when changing hosts? 


And the question which sums up probably all the related issues I can
think of by now: How do you sync Org-mode files, avoid temp data
being synced while preventing unsaved data lost.


[1] Except iOS but I wrote «important» on purpose as you might have
    noticed ;-)

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

Re: Ignore pattern when syncing Org-mode files between hosts

[Thierry Banel]

Le 28/12/2016 16:42, Karl Voit a écrit :
> Hi!
>
> I wonder how others are handling synchronization ignore patterns
> between different hosts.
>
> ...
>
> Currently, I am evaluating Syncthing[0] which looks very good so
> far. It is a game-changer to be because similar to Dropbox[1], it
> synchronizes in background on any change within 60s (by default).
> Thus, resulting in symlinks and tempfiles transferred as well.
>
>

Syncthing supports ignore patterns.
https://docs.syncthing.net/users/ignoring.html

On the web interface http://localhost:8080/
there is a button "Ignore Patterns" under "Advanced settings"

Re: Ignore pattern when syncing Org-mode files between hosts

[Karl Voit]

* Thierry Banel <tbanelwebmin@free.fr> wrote:
> Le 28/12/2016 16:42, Karl Voit a écrit :
>> Hi!
>>
>> I wonder how others are handling synchronization ignore patterns
>> between different hosts.
>>
>> ...
>>
>> Currently, I am evaluating Syncthing[0] which looks very good so
>> far. It is a game-changer to be because similar to Dropbox[1], it
>> synchronizes in background on any change within 60s (by default).
>> Thus, resulting in symlinks and tempfiles transferred as well.
>
> Syncthing supports ignore patterns.

I know.

This was not the reason for my posting ;-)

I want to know what files YOU are ignoring considering the thoughts
I mentioned in my original posting.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github