Re: WIP: promote nmbug to user sync tool

[Sean Whitton <spwhitton@spwhitton.name>]

[please keep me CCed]

Hello David,

On Sat 23 Apr 2022 at 10:38am -03, David Bremner wrote:

> One of the things that new (and old) users of notmuch often miss is a
> way to sync tags between hosts. muchsync exists, but it's a third
> party tool, and (if I understand correctly) it directly modifies the
> notmuch database. Meanwhile we ship a moderately complex python script
> called 'nmbug' which can be used to sync arbitrary tags between hosts.
> This series is my (first pass at an) attempt to promote nmbug to a
> notmuch subcommand "notmuch git". My plan is to replace my own
> homebrew "commit notmuch dump output to git" script with this new
> command; I think it may appeal to other users as well. As a bonus, it
> provides a simple (for the user) way to do incremental backups of the
> notmuch database.
>
> I had to add a couple of caching tricks to make the script usable for
> my database, but it is now reasonably fast (say 20s) to commit a days
> worth of changes on my machine.

I think I'm the person who's already been using nmbug for personal tags
for an appreciable length of time.  It is great to see these caching
improvements and the addition of a test suite.

> There are still some rough edges, mainly due to the heritage of the
> script. Some I am aware of include:
>
> - the (new) man page is inadequate
> - notmuch git doesn't understand the common arguments to notmuch (--config and friends)
> - the defaults for prefix and git repo are wrong for most users. I was thinking about either a
>   --nmbug argument to set the old defaults, or having the script recognize if it is invoked as nmbug.
> - There are a few too many global variables
> - I just remembered, MAX_LASTMOD is unused, leftover from a previous design for querying tags.

Just looking at my current usage, there are two cases where I've wrapped
nmbug in some additional myrepos scripting.  The first is a status
command:

    status =
        nmbug-spw status | grep -v "^U\s" || true
        # `nmbug status` does not catch committed but unpushed changes
        git --no-pager log --branches \
            --not --remotes \
            --simplify-by-decoration --decorate --oneline

Possibly notmuch-git could learn how to do this?

Secondly, I've got this auto-commit command:

    autoci =
        nmbug-spw status | perl -ne'/^[AD][ad]?\s/ and $i++ > 500 and exit 1' \
                  && nmbug-spw commit

The guard has two purposes.  Firstly, it avoids wiping out git as part
of a cronjob auto-commit because the nmbug repo has just been cloned but
'nmbug checkout' hasn't been run.  Secondly, it avoids doing any
committing if there are any known remote changes that haven't been
integrated.  The latter thing is probably just a personal preference.

As for the former thing, I wonder if instead there could be some
mechanism, connected with the new caching, to associate nmbug repos with
the notmuch database, and refuse to operate unless that association
already exists?  So, 'nmbug checkout' would mark it as safe to sync back
and forth between the database and that repo no matter the number of
changes.

On Sat 23 Apr 2022 at 03:49pm -03, David Bremner wrote:

> Related, the current script does not understand NOTMUCH_PROFILE. That
> would be a natural way to locate the default git repo.

It would, but it wouldn't help with configuring a default prefix.
Perhaps an entry in .notmuch-config for that?  Currently I use a tiny
wrapper script:

    #!/bin/sh

    NMBGIT="$HOME/lib/nmbug-spw" NMBPREFIX="spw::" nmbug "$@"

but it would be great to just be able to type 'notmuch git ...'.

-- 
Sean Whitton