git and orgmode: teaching git a bit of orgmode syntax

git and orgmode: teaching git a bit of orgmode syntax

[Karl Voit]

Hi!

I am using gitwatch[1] to auto-commit any changes to my org-mode
files in a git repository.

Unfortunately, git does not handle diffs in a meaningful way. For
example, when there are sub-hierarchies added or removed, it ends up
with many diff-lines like the following:

-***** NEXT test with DAVdroid
+* Lesestoff [1/26]                                        :2read:
 :PROPERTIES:
-:CREATED:  [2016-05-08 Sun 12:51]
+:CREATED: [2012-04-17 Tue 10:39]
+:ARCHIVE:  %s_archive::* Lesestoff
+:CATEGORY: reading
 :END:

The two headings being compared by git are totally unrelated. Git
did find a match for the :PROPERTIES: and :END: lines and therefore
assumed that I modified the level five heading "NEXT test with
DAVdroid" into the first level heading "Lesestoff", not recognizing
that in fact, there were many org-mode lines moved to the archive
file, resulting in many deleted lines instead of modified ones.

The underlying issue is, to my understanding, that git does not know
about Org-mode syntax elements. Git does not know that :PROPERTIES:
and :END: lines should not be used to match similar lines. It does
not know about headings, identifying CREATED time-stamps, and other
things.

I am not familiar with git other than from an end user point of
view. But I do think that git can be taught here via some kind of
extension so that meaningful changes are recognized and displayed.

Is there somebody with the same issue and/or somebody with a better
understanding how this issue can be addressed?

Thanks!

[1] https://github.com/nevik/gitwatch


-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: git and orgmode: teaching git a bit of orgmode syntax

[Nick Dokos]

Karl Voit <devnull@Karl-Voit.at> writes:

> Hi!
>
> I am using gitwatch[1] to auto-commit any changes to my org-mode
> files in a git repository.
>
> Unfortunately, git does not handle diffs in a meaningful way. For
> example, when there are sub-hierarchies added or removed, it ends up
> with many diff-lines like the following:
>
> -***** NEXT test with DAVdroid
> +* Lesestoff [1/26]                                        :2read:
>  :PROPERTIES:
> -:CREATED:  [2016-05-08 Sun 12:51]
> +:CREATED: [2012-04-17 Tue 10:39]
> +:ARCHIVE:  %s_archive::* Lesestoff
> +:CATEGORY: reading
>  :END:
>
> The two headings being compared by git are totally unrelated. Git
> did find a match for the :PROPERTIES: and :END: lines and therefore
> assumed that I modified the level five heading "NEXT test with
> DAVdroid" into the first level heading "Lesestoff", not recognizing
> that in fact, there were many org-mode lines moved to the archive
> file, resulting in many deleted lines instead of modified ones.
>
> The underlying issue is, to my understanding, that git does not know
> about Org-mode syntax elements. Git does not know that :PROPERTIES:
> and :END: lines should not be used to match similar lines. It does
> not know about headings, identifying CREATED time-stamps, and other
> things.
>
> I am not familiar with git other than from an end user point of
> view. But I do think that git can be taught here via some kind of
> extension so that meaningful changes are recognized and displayed.
>
> Is there somebody with the same issue and/or somebody with a better
> understanding how this issue can be addressed?
>
> Thanks!
>
> [1] https://github.com/nevik/gitwatch

You can plug in a different diff tool into git. The next problem is to
write/find/conjure a diff tool that *does* understand org syntax (at least
to some extent). IIRC, there was a GSOC(?) project to do that some years ago,
but I don't think it got finished (and maybe I'm misremembering in any case).
If that really exists[fn:1], maybe it can form a starting point for another push.

* Footnotes

[fn:1] I'd search, but Gmane still seems non-compos-mentis and
       searching in the archives is just awful (or my fu is too
       weak). Anybody know what is happening with Gmane?  How do you
       search the list nowadays?

-- 
Nick

Re: git and orgmode: teaching git a bit of orgmode syntax

[Michael Welle]

Hello,

Nick Dokos <ndokos@gmail.com> writes:
[...]
> [fn:1] I'd search, but Gmane still seems non-compos-mentis and
>        searching in the archives is just awful (or my fu is too
>        weak). Anybody know what is happening with Gmane?  How do you
>        search the list nowadays?
I have a mail folder with about 60k mails. The oldest seem to be 10
years old. I doubt that it is complete, but who knows. What should I
look for? I searched the subjects for gsoc|summer of code and found one
from 2013, IIRC. But that doesn't lead to anything. Searching for
difference in the subject wasn't fruitful either.

Regards
hmw

Re: git and orgmode: teaching git a bit of orgmode syntax

[Nick Dokos]

Michael Welle <mwe012008@gmx.net> writes:

> Hello,
>
> Nick Dokos <ndokos@gmail.com> writes:
> [...]
>> [fn:1] I'd search, but Gmane still seems non-compos-mentis and
>>        searching in the archives is just awful (or my fu is too
>>        weak). Anybody know what is happening with Gmane?  How do you
>>        search the list nowadays?
> I have a mail folder with about 60k mails. The oldest seem to be 10
> years old. I doubt that it is complete, but who knows. What should I
> look for? I searched the subjects for gsoc|summer of code and found one
> from 2013, IIRC. But that doesn't lead to anything. Searching for
> difference in the subject wasn't fruitful either.
>

Thanks. It is quite likely that I misremembered: I found some things on Worg
about an org-aware merge tool for git:

http://orgmode.org/worg/org-contrib/gsoc2012/student-projects/git-merge-tool/proposal.html
http://orgmode.org/worg/org-contrib/gsoc2012/student-projects/git-merge-tool/

but not for a diff tool. And I'm not sure it ever saw the light of day.

-- 
Nick

Re: git and orgmode: teaching git a bit of orgmode syntax

[Michael Welle]

Hello,

Nick Dokos <ndokos@gmail.com> writes:
[...]
> Thanks. It is quite likely that I misremembered: I found some things on Worg
> about an org-aware merge tool for git:
>
> http://orgmode.org/worg/org-contrib/gsoc2012/student-projects/git-merge-tool/proposal.html
> http://orgmode.org/worg/org-contrib/gsoc2012/student-projects/git-merge-tool/
>
> but not for a diff tool. And I'm not sure it ever saw the light of day.
I will have a look at it later this week.

If you have enough information to perform a semantic merge, there should
be also enough information to spit out a diff. Defining what a
difference is in Org would be a starting point. An implementation will
follow then. Maybe a task for the next winter ;).

Regards
hmw

Re: git and orgmode: teaching git a bit of orgmode syntax

[Brett Viren]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

Karl Voit <devnull@Karl-Voit.at> writes:

> -***** NEXT test with DAVdroid
> +* Lesestoff [1/26]                                        :2read:
>  :PROPERTIES:
> -:CREATED:  [2016-05-08 Sun 12:51]
> +:CREATED: [2012-04-17 Tue 10:39]
> +:ARCHIVE:  %s_archive::* Lesestoff
> +:CATEGORY: reading
>  :END:

Without seeing before/after versions it's not clear that this diff is
somehow wrong.

Depending on the editing one does to a given file, a diff may seem "non
local" with some chunks appearing to undo something and others redoing
it.  This happens when parts of the file get rearranged.  For example,
do either of your headlines appear elsewhere in your diff with their
"+/-" signs reversed?

-Brett.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: git and orgmode: teaching git a bit of orgmode syntax

[Stig Brautaset]

Karl Voit <devnull@Karl-Voit.at> writes:

> Unfortunately, git does not handle diffs in a meaningful way. For
> example, when there are sub-hierarchies added or removed, it ends up
> with many diff-lines like the following:
>
> -***** NEXT test with DAVdroid
> +* Lesestoff [1/26]                                        :2read:
>  :PROPERTIES:
> -:CREATED:  [2016-05-08 Sun 12:51]
> +:CREATED: [2012-04-17 Tue 10:39]
> +:ARCHIVE:  %s_archive::* Lesestoff
> +:CATEGORY: reading
>  :END:
>
> The two headings being compared by git are totally unrelated. Git
> did find a match for the :PROPERTIES: and :END: lines and therefore
> assumed that I modified the level five heading "NEXT test with
> DAVdroid" into the first level heading "Lesestoff", not recognizing
> that in fact, there were many org-mode lines moved to the archive
> file, resulting in many deleted lines instead of modified ones.

You may want to try setting =diff.algorithm= (see =man git-config=) to
either =patience= or its variant =histogram=. This Stack Overflow answer
contains an example of how the effects of the diff created with the
patience algorithm, and it looks similar to the case you're describing:
http://stackoverflow.com/questions/4045017/what-is-git-diff-patience-for#4045087

You may also want to check out the option =diff.compactionHeuristic=,
which is described thus:

,----
| Set this option to true to enable an experimental heuristic that
| shifts the hunk boundary in an attempt to make the resulting patch
| easier to read.
`----

Stig