does "lei q" --format/-f need to exist?

does "lei q" --format/-f need to exist?

[Eric Wong]

"maildir:/path/to/dir" has been supported by public-inbox-watch
for years, now.

The following all work today:

	lei q -o mboxrd:/tmp/foo.mboxrd ...
	lei q -o mboxcl2:/tmp/foo.mboxcl2 ...
	lei q -o maildir:/tmp/foo/ ...

So -f/--format seems redundant.  I'm working on on
"lei import" for multiple sources, so being able to specify the
type/URL-scheme on a per-source basis seems like the way to go:

	lei import mboxrd:/tmp/foo.mboxrd maildir:/tmp/md/ ...
	lei import imaps://$host/INBOX.foo nntps://$host/news.group ...

And there's also "lei convert" in the wings (mainly for
testing/development purposes, but could be useful stand-alone):

	lei convert mboxrd:/tmp/foo.mboxrd maildir:/tmp/md/

Re: does "lei q" --format/-f need to exist?

[Kyle Meyer]

Eric Wong writes:

> "maildir:/path/to/dir" has been supported by public-inbox-watch
> for years, now.
>
> The following all work today:
>
> 	lei q -o mboxrd:/tmp/foo.mboxrd ...
> 	lei q -o mboxcl2:/tmp/foo.mboxcl2 ...
> 	lei q -o maildir:/tmp/foo/ ...
>
> So -f/--format seems redundant.

I find "<format>:<destination>" pretty natural/intuitive, even if
perhaps the stdout case (e.g., "mboxrd:-" or "concatjson:-") looks a bit
odd.  Dropping --format makes sense to me.

Re: does "lei q" --format/-f need to exist?

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> Eric Wong writes:
> 
> > "maildir:/path/to/dir" has been supported by public-inbox-watch
> > for years, now.
> >
> > The following all work today:
> >
> > 	lei q -o mboxrd:/tmp/foo.mboxrd ...
> > 	lei q -o mboxcl2:/tmp/foo.mboxcl2 ...
> > 	lei q -o maildir:/tmp/foo/ ...
> >
> > So -f/--format seems redundant.
> 
> I find "<format>:<destination>" pretty natural/intuitive, even if
> perhaps the stdout case (e.g., "mboxrd:-" or "concatjson:-") looks a bit
> odd.  Dropping --format makes sense to me.

How about we just drop --format from the documentation, for now?
(or at least stop recommending it when using with -o)

The stdout case might be a reason to keep it for "lei q",
especially since stdout is the default output:

# this defaults to stdout, looks reasonable:
lei q -f concatjson SEARCH_TERMS...

# this does the same thing, but is more difficult to type and
# looks strange:
lei q -o concatjson:- SEARCH_TERMS

# more readable, but more typing:
lei q -o concatjson:/dev/stdout SEARCH_TERMS

Re: does "lei q" --format/-f need to exist?

[Kyle Meyer]

Eric Wong writes:

> How about we just drop --format from the documentation, for now?
> (or at least stop recommending it when using with -o)
>
> The stdout case might be a reason to keep it for "lei q",
> especially since stdout is the default output:
[...]

I don't feel strongly one way or the other about keeping --format, but
if it is kept around for stdout, I think it'd be good to document it
(i.e. your "stop recommending it when using with -o option").

Re: does "lei q" --format/-f need to exist?

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> Eric Wong writes:
> 
> > How about we just drop --format from the documentation, for now?
> > (or at least stop recommending it when using with -o)
> >
> > The stdout case might be a reason to keep it for "lei q",
> > especially since stdout is the default output:
> [...]
> 
> I don't feel strongly one way or the other about keeping --format, but
> if it is kept around for stdout, I think it'd be good to document it
> (i.e. your "stop recommending it when using with -o option").

Alright, I think keeping it and only recommending it for stdout
(or --stdin with import) is the way to go.

"-o format:/path/name" should be encouraged for regular file and
directory args.

Do you have time to update the current manpages?  Thanks in
advance if so, otherwise I'll try to do it at some point....


On a side note, it also occurs to me some users may expect paths
like /dev/fd/[0-2] or /proc/self/fd/[0-2] to work like the
/dev/stdout handling in lei-daemon.  We'll have to account for
that in daemon mode...

Re: does "lei q" --format/-f need to exist?

[Kyle Meyer]

Eric Wong writes:

> Do you have time to update the current manpages?  Thanks in
> advance if so, otherwise I'll try to do it at some point....

Yes, as long as getting to it in the next few days is speedy enough :)

Re: does "lei q" --format/-f need to exist?

[Eric Wong]

Eric Wong <e@80x24.org> wrote:
> Kyle Meyer <kyle@kyleam.com> wrote:
> > Eric Wong writes:
> > 
> > > How about we just drop --format from the documentation, for now?
> > > (or at least stop recommending it when using with -o)
> > >
> > > The stdout case might be a reason to keep it for "lei q",
> > > especially since stdout is the default output:
> > [...]
> > 
> > I don't feel strongly one way or the other about keeping --format, but
> > if it is kept around for stdout, I think it'd be good to document it
> > (i.e. your "stop recommending it when using with -o option").
> 
> Alright, I think keeping it and only recommending it for stdout
> (or --stdin with import) is the way to go.
> 
> "-o format:/path/name" should be encouraged for regular file and
> directory args.

Actually, maybe "-o $format" be implicitly stdout?
(and "-i $format" be implicitly stdin for convert|import)

Re: does "lei q" --format/-f need to exist?

[Kyle Meyer]

Eric Wong writes:

> Eric Wong <e@80x24.org> wrote:
[...]
>> Alright, I think keeping it and only recommending it for stdout
>> (or --stdin with import) is the way to go.
>> 
>> "-o format:/path/name" should be encouraged for regular file and
>> directory args.
>
> Actually, maybe "-o $format" be implicitly stdout?
> (and "-i $format" be implicitly stdin for convert|import)

Hmm, true.  If we went that route, I guess the format auto-detection in
LeiOverview::detect_fmt() (currently just for maildir) would be dropped
because an -o argument without a colon would be taken as the format
rather than a destination that the format can be detected from?

Re: does "lei q" --format/-f need to exist?

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> Eric Wong writes:
> 
> > Eric Wong <e@80x24.org> wrote:
> [...]
> >> Alright, I think keeping it and only recommending it for stdout
> >> (or --stdin with import) is the way to go.
> >> 
> >> "-o format:/path/name" should be encouraged for regular file and
> >> directory args.
> >
> > Actually, maybe "-o $format" be implicitly stdout?
> > (and "-i $format" be implicitly stdin for convert|import)
> 
> Hmm, true.  If we went that route, I guess the format auto-detection in
> LeiOverview::detect_fmt() (currently just for maildir) would be dropped
> because an -o argument without a colon would be taken as the format
> rather than a destination that the format can be detected from?

Maybe not dropped, but probably tweaked for DWIM-ness.

Maybe:

  If somebody wants a Maildir to dump JSON search results in they
  could use "-o ./json" or "-o json/" or "-o /path/to/json".

  "-o json" (no slashes or colons) would mean JSON output to stdout.


But then, "json" could be the name of an existing directory,
so if it exists...

Part of me thinks its too magical...

On the other hand, maybe only requiring the colon: "-o json:"
is enough to disambiguate and isn't too much typing.

We'll just assume nobody would want to end a directory
with ":".  They can still use "-o maildir:/i/like/colons:"
if they really want to end a dirname with ":" for whatever
reason...

[PATCH] doc: lei: favor "-o format:$PATHNAME" over "-f"

[Kyle Meyer]

Eric Wong writes:

> Maybe not dropped, but probably tweaked for DWIM-ness.
>
> Maybe:
>
>   If somebody wants a Maildir to dump JSON search results in they
>   could use "-o ./json" or "-o json/" or "-o /path/to/json".
>
>   "-o json" (no slashes or colons) would mean JSON output to stdout.
>
> But then, "json" could be the name of an existing directory,
> so if it exists...
>
> Part of me thinks its too magical...

That's kind of my feeling, though I suspect that would at least
consistently do what I mean and be unsurprising.

> On the other hand, maybe only requiring the colon: "-o json:"
> is enough to disambiguate and isn't too much typing.

Yeah, I don't mind that, but I guess that almost gets us back to "-o
json:-".  Then again, I didn't mind that either or really any of the
options proposed in this thread :)

Anyway, no matter where this lands, the manpages should switch to
using/recommending the <format>: prefix, so here's a patch for that.

-- >8 --
Subject: [PATCH] doc: lei: favor "-o format:$PATHNAME" over "-f"

The --format argument is redundant and may be dropped entirely.
Update the lei manpages to prefer the format prefix.

cf. https://public-inbox.org/meta/20210217044032.GA17934@dcvr/
---
 Documentation/lei-import.pod   | 10 ++++++----
 Documentation/lei-overview.pod |  4 ++--
 Documentation/lei-q.pod        | 20 ++++++++++++++------
 3 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/Documentation/lei-import.pod b/Documentation/lei-import.pod
index 14ca2d45d6d8bfa1..2051e6bc86c5fd36 100644
--- a/Documentation/lei-import.pod
+++ b/Documentation/lei-import.pod
@@ -11,8 +11,10 @@ lei import [OPTIONS] --stdin
 =head1 DESCRIPTION
 
 Import messages into the local storage of L<lei(1)>.  C<LOCATION> is a
-source of messages: a directory (Maildir) or a file (whose format is
-specified via C<--format>).
+source of messages: a directory (Maildir) or a file.  For a regular
+file, the location must have a C<E<lt>formatE<gt>:> prefix specifying
+one of the following formats: C<eml>, C<mboxrd>, C<mboxcl2>,
+C<mboxcl>, or C<mboxo>.
 
 TODO: Update when URL support is added.
 
@@ -22,8 +24,8 @@ TODO: Update when URL support is added.
 
 =item -f MAIL_FORMAT, --format=MAIL_FORMAT
 
-Message input format: C<eml>, C<mboxrd>, C<mboxcl2>, C<mboxcl>,
-C<mboxo>.
+Message input format.  Unless messages are given on C<stdin>, using a
+format prefix with C<LOCATION> is preferred.
 
 =item --stdin
 
diff --git a/Documentation/lei-overview.pod b/Documentation/lei-overview.pod
index 840d011b27adb088..62b62280ad2ddd69 100644
--- a/Documentation/lei-overview.pod
+++ b/Documentation/lei-overview.pod
@@ -16,7 +16,7 @@ L<public-inbox-v2-format(5)>.
 
 =over
 
-=item $ lei import --format=mboxrd t.mbox
+=item $ lei import mboxrd:t.mbox
 
 Import the messages from an mbox into the local storage.
 
@@ -64,7 +64,7 @@ Search for messages whose subject includes "lei" and "skeleton".
 Do the same, but also report unmatched messages that are in the same
 thread as a matched message.
 
-=item $ lei q -t -o t.mbox -f mboxcl2 --mua=mutt s:lei s:skeleton
+=item $ lei q -t -o mboxcl2:t.mbox --mua=mutt s:lei s:skeleton
 
 Write mboxcl2-formatted results to t.mbox and enter mutt to view the
 file by invoking C<mutt -f %f>.
diff --git a/Documentation/lei-q.pod b/Documentation/lei-q.pod
index c8df6fc7244bfae6..75fdc613579cdc18 100644
--- a/Documentation/lei-q.pod
+++ b/Documentation/lei-q.pod
@@ -26,17 +26,25 @@ Read search terms from stdin.
 
 =item -o MFOLDER, --output=MFOLDER, --mfolder=MFOLDER
 
-Destination for results (e.g., C<path/to/Maildir> or - for stdout).
+Destination for results (e.g., C<path/to/Maildir> or
+C<mboxcl2:path/to/mbox>).  The format can be specified by adding a
+C<E<lt>formatE<gt>:> prefix with any of these values: C<maildir>,
+C<mboxrd>, C<mboxcl2>, C<mboxcl>, C<mboxo>, C<json>, C<jsonl>, or
+C<concatjson>.
+
+TODO: Provide description of formats?
+
+When a format isn't specified, it's chosen based on the destination.
+C<json> is used for the default destination (stdout), and C<maildir>
+is used for an existing directory or non-existing path.
 
 Default: -
 
 =item -f FORMAT, --format=FORMAT
 
-Format of results: C<maildir>, C<mboxrd>, C<mboxcl2>, C<mboxcl>,
-C<mboxo>, C<json>, C<jsonl>, or C<concatjson>.  The default format
-used depends on C<--output>.
-
-TODO: Provide description of formats?
+Format of results.  This option exists as a convenient way to specify
+the format for the default stdout destination.  Using a C<format:>
+prefix with the C<--output> destination is preferred otherwise.
 
 =item --pretty
 

base-commit: c1ad789a90c274f9912d53bb1c7f1a3cc07cb233
-- 
2.30.1

Re: [PATCH] doc: lei: favor "-o format:$PATHNAME" over "-f"

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> Eric Wong writes:
> 
> > Maybe not dropped, but probably tweaked for DWIM-ness.
> >
> > Maybe:
> >
> >   If somebody wants a Maildir to dump JSON search results in they
> >   could use "-o ./json" or "-o json/" or "-o /path/to/json".
> >
> >   "-o json" (no slashes or colons) would mean JSON output to stdout.
> >
> > But then, "json" could be the name of an existing directory,
> > so if it exists...
> >
> > Part of me thinks its too magical...
> 
> That's kind of my feeling, though I suspect that would at least
> consistently do what I mean and be unsurprising.
> 
> > On the other hand, maybe only requiring the colon: "-o json:"
> > is enough to disambiguate and isn't too much typing.
> 
> Yeah, I don't mind that, but I guess that almost gets us back to "-o
> json:-".  Then again, I didn't mind that either or really any of the
> options proposed in this thread :)

I'll ponder it more while I work on some other features...
And bash completion still needs to be better in that area.

> Anyway, no matter where this lands, the manpages should switch to
> using/recommending the <format>: prefix, so here's a patch for that.

Yup, thanks, pushed as commit 56b3493c79087979f10f5a3cae7deedaf4ec9fa3