[PATCH] scripts: add script to export v1 inboxes to maildir

[PATCH] scripts: add script to export v1 inboxes to maildir

[Evan Gates]

---

I'm pretty sure this is a gross misuse of the public-inbox format, and
as such will not be offended if you decide not to include this script.

I read the public-inbox README and liked the idea of the pull model.
I briefly scanned clients.txt but didn't easily figure out how I was
supposed to pull and read mail locally.  Since the v1 format already
has a bunch of uniquely named files, that are themselves emails,
it was pretty simple to symlink them into a maildir format so I can
read them in my mail client.

As a followup to this abomination, what is the correct way to pull
and read mail locally?

 scripts/v1-to-maildir | 58 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)
 create mode 100755 scripts/v1-to-maildir

diff --git a/scripts/v1-to-maildir b/scripts/v1-to-maildir
new file mode 100755
index 00000000..1cfcd2de
--- /dev/null
+++ b/scripts/v1-to-maildir
@@ -0,0 +1,58 @@
+#!/bin/bash
+
+# Script to export v1 public-inbox repos to Maildir format.  Supports
+# updates.  Creates a .Maildirs directory in the PWD and a subdirectory
+# for each public-inbox git repo passed as an argument.  The repos ARE
+# NOT bare repos.
+#
+# Usage:
+#
+#     v1-to-maildir [public-inbox-v1-repo]...
+#
+# Example:
+#
+#     mkdir ~/lists
+#     cd ~/lists
+#     git clone https://public-inbox.org/meta public-inbox-meta
+#     git clone https://public-inbox.org/test public-inbox-test
+#     v1-to-maildir */
+# 
+#     Then point mail client to ~/lists/.Maildirs
+#     Run v1-to-maildir */ again to git pull and add new messages.
+
+dir=.Maildirs
+fmt=%-10s
+
+for d do
+	printf %s\\n "-- $d"
+	printf "$fmt" git\ pull:
+	if ! git -C "$d" pull -q; then
+		continue
+	fi
+	echo OK
+
+	printf "$fmt" mkdir:
+	for md in new cur tmp; do
+		if ! mkdir -p "$dir/$d/$md"; then
+			continue 2
+		fi
+	done
+	echo OK
+
+	printf "$fmt" new:
+	declare -A cur
+	new=()
+	for f in "$dir/$d/cur/"*; do
+		f=${f##*/}
+		f=${f%%:*}
+		cur[$f]=1
+	done
+	for f in "$d/"*/*; do
+		((cur[${f##*/}])) || new+=("$f")
+	done
+	((${#new[@]})) &&
+		printf %s\\0 "${new[@]/#/..\/..\/..\/}" |
+			xargs -0 ln -sft "$dir/$d/new"
+	printf %d\\n "${#new[@]}"
+	echo
+done
-- 
2.35.1

Re: [PATCH] scripts: add script to export v1 inboxes to maildir

[Eric Wong]

Evan Gates <evan.gates@gmail.com> wrote:
> ---
> 
> I'm pretty sure this is a gross misuse of the public-inbox format, and
> as such will not be offended if you decide not to include this script.

Heh, OK :)

> I read the public-inbox README and liked the idea of the pull model.
> I briefly scanned clients.txt but didn't easily figure out how I was
> supposed to pull and read mail locally.  Since the v1 format already
> has a bunch of uniquely named files, that are themselves emails,
> it was pretty simple to symlink them into a maildir format so I can
> read them in my mail client.

*gasp*

cloning v1 w/o bare gets really ugly for larger inboxes.

> As a followup to this abomination, what is the correct way to pull
> and read mail locally?

With v1.7.0+, the initial clone:

	public-inbox-clone https://try.public-inbox.org/test public-inbox-test
	public-inbox-index public-inbox-test

# search for all messages >0 bytes from the last year until now:

	lei q -o /path/to/maildir -O public-inbox-test z:0.. rt:last.year..

# You can change "rt:last.year.." to any time range or leave it out,
# but giant Maildirs get too slow for me.

# periodic updates:

	make -C public-inbox-test update
	lei up /path/to/maildir

> +#     cd ~/lists
> +#     git clone https://public-inbox.org/meta public-inbox-meta
> +#     git clone https://public-inbox.org/test public-inbox-test

"git clone" is fine for older versions, but --mirror (or --bare)
is strongly encouraged since v1 inboxes can grow trees w/o bounds.