Elisp problem 9 of 10: NOT `interactive', but INTERACTIVE!

Elisp problem 9 of 10: NOT `interactive', but INTERACTIVE!

[Emanuel Berg]

----------------------------------------------------------------------
Part 10:
  https://lists.gnu.org/archive/html/emacs-devel/2024-08/msg00154.html
----------------------------------------------------------------------  

~~~~~  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTE:  Long post, but 2 people asked me to provide
~~~~~  concrete examples. This post is 100% on topic with no
       reference to any person. If you feel like it, answer in the
       same way.                                              --i.
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Okay, so problem 9 _isn't_ `interactive' per se but the whole
interactive/non-interactive business - including setting up
both interfaces - and everything else that has to do with it -
so this is problem number 9 with Elisp.

I have seen the C code for `interactive', it is clean so no
disrespect, but as for me, that miniature, space-less format
string does push me away, also, when I set out to code, I feel
good, enthusiastic, it feels like this isn't gonna be just
another `defun'. If I, at that point, feel like I can probably
do something - 90% - with `interactive', I'm still not gonna
do it but instead write Elisp, since, that I _know_ I can do.

Particular problems with interactive: we saw the other day how
this hack was used

  (interactive (if (numberp current-prefix-arg)
                   (list current-prefix-arg)
                 current-prefix-arg))

M-x is n = 1 I know, but that function was about drawing
a line, and drawing a line of one char, that wasn't useful
for that application.

And, I have not found a way to have the same defaults in the
interactive and non-interactive parts (in the interactive
part, they would be used as the default suggestions) - one can
do that with a lexical `let'-lambda, or global variables, but
that's crazy for such a task.

But ... I said that not `interactive' but ALL of it was
the problem?

Indeed. You start writing a `defun'. You write the formal
parameters, maybe put in some special notation. This is all
pleasant since it is part of writing the defun.

Then comes the `interactive' part, I think all the efforts
there are pretty much unneccessary, we could have a tool for
that, so I don't enjoy it, it is just something that needs to
be done for it to work.

And when it is done - no, it isn't! Now comes the
non-interactive part. DWIM functions involving the region and
BEG and END defaults, default values for other optional
arguments, validation of indata, conversion between data types
as I want to to work for both strings and chars, or something
like that - it just goes on, and examples do follow.
It is crazy!

Her is a part from my ELPA package `wrap-search'.

1. Everything is in a let-lambda to begin with, and there we
   have 5 binding so 5 lines.

2. Then `interactive', not really using it but instead Elisp,
   is a monstrous 6 lines with - how many - 10 functions
   calls? Not counting all the backticks so again special
   syntax and notation. Reading the indata, setting the
   options, and using the region. Pretty pretty basic stuff.
   That is here, and always, unnecessarily done manually
   writing Elisp!

3. But after that, is it done? NO! It goes on with _19 lines_
   of post-processing, converting, validating, storing the
   defaults and I don't - well, I do know, because that code
   is written in such a way. Still!

4. After all that, we see that a mere 12 lines are used to
   actually do stuff!

So:

   6 lines that has to do with the let-lambda;
   1 line  for the defun head;
  12 lines for the docstring, almost exclusively about args
     and keys, i.e., the interface;
   6 lines for `interactive', here in Elisp so shame;
+ 19 lines of post-processing and setting up.
----------------------------------------------------
     to do 12 lines of Elisp!

Or (+ 6 1 12 6 19) 44 lines for the interface.

Or ( / 12 (+ 44. 12.0)) just 21% has to do with solving the
problem - the rest is setting everything up. And so in so
many ways! For what? Saving the default, suggesting
the default? What more? Have a look!

I know everyone don't rely on DWIM functions as much as I do,
they don't ever validate anything, and I also don't do that
always, either, but I do it sometimes - like here - and the
result is just crazy code in that sense.

Okay, but then, then it is finally done, right?

*NO*

5. You go to the Emacs source, you see the functions branch
   halfway based on `called-interactively-p',

6. we see `call-interactively' and `funcall-interactively',

7. we see the trailing interactive' argument certain functions
   use - also a hack of course - bastardization of the
   function to make the interactive interface work - why
   needed BTW?

8. You finally run your function. Nope, it works for
   interactive use only. You have to return something, not
   just messages. It goes on and on. Debug time.

9. And if you are an even worse Elisper than I, why don't you
   consult the help? What do you see?

   "This is for interactive use."

   "Don't use this from Emacs Lisp."

   "If you are considering using this from Emacs Lisp, you are
    almost always wrong."

This is _completely crazy_, these are mere _interfaces_ and
should ONLY be used for that purpose, i.e. only for providing
the functions formal parameters (arguments) with data and this
by some interaction with the user or tech system that suits
the situation. After that, functions should execute
completely transparently, shouldn't matter one bit who called
them, from where. Only their param values determine what
happens. Yes, based on the _values_ and no metadata how they
were called.

I get it there are exceptions, and optimizations, and a huge
program like Emacs isn't ever gonna be consistent, but this is
just crazy crazy crazy. It is everywhere.

Help for some function? First thing. Is it interactive or not?

Bind a function to a key? Not a command!

Yet do TAB complete in the minibuffer and there are so many
functions anyway it is impossible to find anything.
Do `apropos-function' for "end-of" you get 212 lines of
output, everything from w3m, to slime, to rmail. And, very
clearly indicated, M-x ... RET to denote keybinding,
"Command" and not "Function" to indicate, I don't know, some
distinction that has been promoted beyond belief.

But: I'm fine with how it is in Emacs, no problem. I just
don't want to write all that Elisp. I function, one tool to do
the interactive, the non-interactive, the defaults, the
validation, the conversions and the saving of history (or
a single new default). That's okay. More than that -
a problem. And now we have I don't know how many times more
than that.

In the Emacs source, "interactive" is mentioned 17 744 times.
In Gnus it is mentioned 1222 times, in org 1223 times and in
relatively tiny ERC 213 times.

So to sum it up, we have a weird situation where interactive
is everywhere. Yet, how can it be, that the tool just do
a tiny part - that you need so much more? Whole programs are
wasted on it, yet it isn't contained there, instead it
is everywhere.

What one should do:

1. Policy change, interactive vs non-interactive should only
   be a matter of communicating with the user or tech
   back-/frontend while setting up the interface.
   Nothing more.

2. One tool to setup _everything_. The defaults, the
   validation, the typechecks, whatever a little bit different
   that needs to happen for interactive use, docstrings should
   be automated (and based on the variables, not the function)
   and prompt strings can be automated poor man-style as well,
   unless explicitely provided for those who enjoy that.

Closure, docstring which is all about the arguments, almost,
function head including special syntax, `interactive' involving
Elisp including special syntax (backticks and more), then
start all over with post-processing for the non-interactive
ditto. All this is crazy crazy crazy.

Yeah, that is why I have it on position number 9 for big
problems with Emacs Lisp - not elegant, in this case, and not
powerful at all, takes too much time from programming. Sorry.

My own idea? It will be great time but I think not just me
should think in those terms, that is why I wrote all this.

Okay, take care.


PS. Instead of me yanking the code here, you can check it out
    here:
    
      https://dataswamp.org/~incal/elpa/wrap-search.el @ v 4.16.17
      - or -
      git clone https://dataswamp.org/~incal/wrap-search.git
      
    No it sounds like I'm very unhappy about it. On the
    contrary, I think the code - with the help of a friend -
    is awesome. But that doesn't change the aspect I bring up
    which I just think is a huge problem.

PPS. Here is how i it is done in another language. [Note: No
     language wars allowed here on gmane.emacs.help, I just
     want to show you how easy it can be to do everything
     (these are the CLI arguments from the shell).] But it can
     be done the same way here as well, for CLI and
     for functions.

     Why not? In Emacs, we have, err, `command-line-args'?

     Please stop boasting about Lisp when we can't do this
     simple thing. But maybe it exists in some package?
     [M]ELPA, maybe? If so, I'll be happy to try it!

     *Too long post*, now, piece. No flame wars following
     this, stick to the subject and don't talk about _me_,
     talk about what the post is about. Aight :)

  cli = argparse.ArgumentParser(description=f'swamp thing ({sth_nick}) IRC bot')
  cli.add_argument('-c',  '--channel',        type=str, help='IRC channel',                 default='#swamp-thing')
  cli.add_argument('-D',  '--debug',                    help='enable debug messages',                                   action='store_true')
  cli.add_argument('-d',  '--dry',                      help="dry run, don't connect",                                  action='store_true')
  cli.add_argument('-e',  '--encode',                   help='use SSL',                                                 action='store_true')
  cli.add_argument('-lf', '--log-file',       type=str, help='log file')
  cli.add_argument('-ll', '--log-level',      type=int, help='log level, high is more',     default=3)
  cli.add_argument('-m',  '--track-messages',           help='track messages',                                          action='store_true')
  cli.add_argument('-n',  '--nick',           type=str, help='bot nick',                    default=sth_nick)
  cli.add_argument('-p',  '--port',           type=int, help='IRC port',                    default=6697)
  cli.add_argument('-s',  '--server',         type=str, help='IRC server (host)',           default='irc.freenode.net')
  cli.add_argument('-v',  '--version',                  help='show version and exit',                                   action='store_true')
  cli.add_argument('-vl', '--verbose-level',  type=int, help='verbose level, high is more', default=2)
  opts = cli.parse_args()

PPPS. For GNU, it is argp (-p for parse, I think) and getopt,
      maybe some guys on this list worked on those? There are
      such tools everywhere. But not here? Of course there
      are? Maybe there truly isn't.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Elisp problem 9 of 10: NOT `interactive', but INTERACTIVE!

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Sat, 10 Aug 2024 18:19:41 +0200
> 
> Help for some function? First thing. Is it interactive or not?
> 
> Bind a function to a key? Not a command!
> 
> Yet do TAB complete in the minibuffer and there are so many
> functions anyway it is impossible to find anything.
> Do `apropos-function' for "end-of" you get 212 lines of
> output, everything from w3m, to slime, to rmail.

You forgot about "C-h a" (apropos-command).

Re: Elisp problem 9 of 10: NOT `interactive', but INTERACTIVE!

[Emanuel Berg]

Eli Zaretskii wrote:

>> Yet do TAB complete in the minibuffer and there are so many
>> functions anyway it is impossible to find anything.
>> Do `apropos-function' for "end-of" you get 212 lines of
>> output, everything from w3m, to slime, to rmail.
>
> You forgot about "C-h a" (apropos-command).

Indeed, this 'meaningful' distinction is everywhere, it is all
over Emacs, massive.

But actually there are 399 lines for C-h a `end-of'.

I have forgotten what it was we were doing with 'end-of', but
my own software is useful here. Well, here is what it says,
anyway (I should make that scrollable, but it is nowhere near
399 lines, forget about it.

---------------------------------------  
POS          SYMBOL          WRDS  H  P
---------------------------------------  
 1. end-of-line              0.80  2  3
 2. end-of-buffer            0.80  2  3
 3. end-of-defun             0.80  2  3
 4. end-of-sexp              0.80  2  3
 5. end-of-thing             0.80  2  3
 6. end-of-paragraph-text    0.67  2  3
 7. end-of-visual-line       0.67  2  3
 8. end-of-visible-line      0.67  2  3
 9. markdown-end-of-subtree  0.67
10. outline-end-of-subtree   0.67
11. org-end-of-item          0.67
12. mark-end-of-buffer       0.67
13. c-end-of-macro           0.67
14. bibtex-end-of-field      0.67
15. sh-end-of-command        0.67
16. sldb-end-of-backtrace    0.67
17. diff-end-of-file         0.67
18. c-end-of-statement       0.67
19. bibtex-end-of-string     0.67
20. bibtex-end-of-entry      0.67
21. calendar-end-of-month    0.67
22. markdown-end-of-defun    0.67
23. eshell-end-of-output     0.67
---------------------------------------

As for forgetting about `apropos-command', oh, no, I'm aware
of it, but there are tons of stuff. Yeah, let's use
`fun-names' for just "command", then!

Indeed, a lot of stuff no one ever heard of. But you can guess
what most does. `command-apropos' is on position 15.

--------------------------------------
POS         SYMBOL          WRDS  H  P
--------------------------------------
 1. command-query           0.67  1  1
 2. command-line            0.67  1  1
 3. command-execute         0.67  1  1
 4. command-modes           0.67  1  1
 5. command-remapping       0.67  1  1
 6. command-history         0.67  1  1
 7. command-apropos         0.67  1  1
 8. read-command            0.67
 9. describe-command        0.67
10. enable-command          0.67
11. eshell-command          0.67
12. process-command         0.67
13. nnimap-command          0.67
14. disable-command         0.67
15. apropos-command         0.67
16. shell-command           0.67
17. command-execute--query  0.50  1  1
18. command-line-1          0.50  1  1
19. vc-git-command          0.50
20. eshell-command-started  0.50
21. tex-summarize-command   0.50
22. set-mark-command        0.50
23. shell-command-sentinel  0.50
--------------------------------------

Just for sport, let's do it with `apropos-command' as well.
Ha! 197 lines, with `apropos-command' on line 14.

In this case, I'm not gonna bother with false modesty, my tool
is much more pleasant, better. Anyone wants to colorize, make
those links interactive buttons, speak to me (by mail).
Not because you like me, but if you like the tool, I mean.
I checks out for every linter but our man at ELPA, I don't
think he as much time with everyone sending him packages.
So it hasn't appeared, but I hope it will, soon! But no
stress, everything - always - in time.

Version @ 6.13.4:
  https://dataswamp.org/~incal/elpa/fun-names.el
  git clone https://dataswamp.org/~incal/fun-names.git

Meanwhile, `apropos-command' reports on line 68:

  erc-command-indicator-toggle-hidden

Yeah, probably that is the one when we search for 'command' :)

Or maybe

  Electric-command-history-redo-expression

the third line suggested :)

But to bring this thread back on topic, here is what
`fun-names' report for 'interactive'. Yeah, just crazy.

`whitespace-interactive-char' anyone?

--------------------------------------------
POS            SYMBOL             WRDS  H  P
--------------------------------------------
 1. interactive-p                 0.67  1  1
 2. interactive-form              0.67  1  1
 3. checkdoc-interactive          0.67
 4. gnus-interactive              0.67
 5. slime-interactive-eval        0.50
 6. sql-product-interactive       0.50
 7. checkdoc-message-interactive  0.50
 8. checkdoc-ispell-interactive   0.50
 9. cconv--interactive-helper     0.50
10. eshell-interactive-filter     0.50
11. url-interactive-p             0.50
12. find-tag-interactive          0.50
13. eshell-interactive-print      0.50
14. whitespace-interactive-char   0.50
15. oclosure-interactive-form     0.50
16. ad-interactive-form           0.50
17. advice--interactive-form      0.50
18. eshell-interactive-process    0.50
19. mh-interactive-range          0.50
20. checkdoc-interactive-loop     0.50
--------------------------------------------

But enough trash talking `apropos' which will, I can say even
now, not appear on the top 10 list.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Elisp problem 9 of 10: NOT `interactive', but INTERACTIVE!

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Sat, 10 Aug 2024 19:22:37 +0200
> 
> Eli Zaretskii wrote:
> 
> >> Yet do TAB complete in the minibuffer and there are so many
> >> functions anyway it is impossible to find anything.
> >> Do `apropos-function' for "end-of" you get 212 lines of
> >> output, everything from w3m, to slime, to rmail.
> >
> > You forgot about "C-h a" (apropos-command).
> 
> Indeed, this 'meaningful' distinction is everywhere, it is all
> over Emacs, massive.

And for a very good reason.

> But actually there are 399 lines for C-h a `end-of'.

Not here.  In "emacs -Q", I get just 21, for 9 commands.  Maybe you
have apropos-do-all customized to t, in which case "C-h a" will report
functions as well, not just commands.