How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

(defun db/table-id-plist (table id pg)
  "Returns plist for table by its table_id"
  (let* ((columns (db/table-columns table pg))
	 (sql (db/sql-by-columns table columns id))
	 (values (rcd-sql-first sql pg))
	 (values (append values nil))
	 (length (length columns))
	 (plist '()))
    (dotimes (i length plist)
      (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))


What is proper way to avoid this warning in `dolist' or `dotimes':

rcd-db.el:841:1: Warning: Unused lexical variable ‘i’

Declaring variable `i' in `let' before `dolist' or `dotimes' did not
help.

Jean

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> (defun db/table-id-plist (table id pg)
>   "Returns plist for table by its table_id"
>   (let* ((columns (db/table-columns table pg))
> 	 (sql (db/sql-by-columns table columns id))
> 	 (values (rcd-sql-first sql pg))
> 	 (values (append values nil))
> 	 (length (length columns))
> 	 (plist '()))
>     (dotimes (i length plist)
>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
>
> What is proper way to avoid this warning in `dolist' or
> `dotimes':
>
> rcd-db.el:841:1: Warning: Unused lexical variable ‘i’
>
> Declaring variable `i' in `let' before `dolist' or `dotimes'
> did not help.

Right. Let's just say it is a `dotimes' thing, see line
309-310, /usr/local/share/emacs/28.0.50/lisp/subr.el

  ;; FIXME: This let often leads to "unused var" warnings.
  `((let ((,(car spec) ,counter)) ,@(cddr spec))))))

(require 'cl-lib)

(defun db/table-id-plist (table id pg)
  "Do something with TABLE, ID and PG."
  (let*((cols   (db/table-columns table pg))
        (sql    (db/sql-by-columns table cols id))
        (vals   (rcd-sql-first sql pg))
        (plist) )
    (cl-loop for c in cols
             for v in vals
             do (setf plist (plist-put plist (intern c) v)) )))

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-01-07 13:07]:
> Jean Louis wrote:
> 
> > (defun db/table-id-plist (table id pg)
> >   "Returns plist for table by its table_id"
> >   (let* ((columns (db/table-columns table pg))
> > 	 (sql (db/sql-by-columns table columns id))
> > 	 (values (rcd-sql-first sql pg))
> > 	 (values (append values nil))
> > 	 (length (length columns))
> > 	 (plist '()))
> >     (dotimes (i length plist)
> >       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
> >
> > What is proper way to avoid this warning in `dolist' or
> > `dotimes':
> >
> > rcd-db.el:841:1: Warning: Unused lexical variable ‘i’
> >
> > Declaring variable `i' in `let' before `dolist' or `dotimes'
> > did not help.
> 
> Right. Let's just say it is a `dotimes' thing, see line
> 309-310, /usr/local/share/emacs/28.0.50/lisp/subr.el
> 
>   ;; FIXME: This let often leads to "unused var" warnings.
>   `((let ((,(car spec) ,counter)) ,@(cddr spec))))))
> 
> (require 'cl-lib)
> 
> (defun db/table-id-plist (table id pg)
>   "Do something with TABLE, ID and PG."
>   (let*((cols   (db/table-columns table pg))
>         (sql    (db/sql-by-columns table cols id))
>         (vals   (rcd-sql-first sql pg))
>         (plist) )
>     (cl-loop for c in cols
>              for v in vals
>              do (setf plist (plist-put plist (intern c) v)) )))

Thank you.

cl-loop I have used all times in Common Lisp, fine, but not so
nice. So instead of `dolist' I can use mapcar or similar function with
lambda and there is no complaint.

I would rather like to solve `dolist' compiler complain, and not
change to more complex `cl-loop' and avoid changing to `mapcar' usage.

Jean

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Philipp Stephani]

Am Do., 7. Jan. 2021 um 08:59 Uhr schrieb Jean Louis <bugs@gnu.support>:
>
> (defun db/table-id-plist (table id pg)
>   "Returns plist for table by its table_id"
>   (let* ((columns (db/table-columns table pg))
>          (sql (db/sql-by-columns table columns id))
>          (values (rcd-sql-first sql pg))
>          (values (append values nil))
>          (length (length columns))
>          (plist '()))
>     (dotimes (i length plist)
>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
>
>
> What is proper way to avoid this warning in `dolist' or `dotimes':
>
> rcd-db.el:841:1: Warning: Unused lexical variable ‘i’

This is https://debbugs.gnu.org/cgi/bugreport.cgi?bug=16206. tl;dr:
don't use the three-argument form of `dotimes':

(dotimes (i length)  ; no `plist' here
   ...)
plist

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Philipp Stephani wrote:

> This is https://debbugs.gnu.org/cgi/bugreport.cgi?bug=16206. tl;dr:
> don't use the three-argument form of `dotimes':
>
> (dotimes (i length)  ; no `plist' here

No length there - there are loops to iterate data structures,
so no need to measure them first :)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-01-07 13:11]:
> Philipp Stephani wrote:
> 
> > This is https://debbugs.gnu.org/cgi/bugreport.cgi?bug=16206. tl;dr:
> > don't use the three-argument form of `dotimes':
> >
> > (dotimes (i length)  ; no `plist' here
> 
> No length there - there are loops to iterate data structures,
> so no need to measure them first :)

I am sure it can be replaced with better functions. I use `dotimes'
sometimes when I do not know the length of the list to work on. It may
be replaced by other functions or nicer ones.

In regards to third argument, to tell not to use it is not logical as:

dotimes is a Lisp macro in ‘subr.el’.

(dotimes (VAR COUNT [RESULT]) BODY...)

(dotimes (n 10) (setq a n)) => returns nil

(dotimes (n 10 a) (setq a n)) => returns a as 9

So when I need the return value from dotimes, I need it.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> No length there - there are loops to iterate data
>> structures, so no need to measure them first :)
>
> I am sure it can be replaced with better functions. I use
> `dotimes' sometimes when I do not know the length of the
> list to work on.

It is enough that the computer knows it.

"Don't ask, don't tell"

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Philipp Stephani <p.stephani2@gmail.com> [2021-01-07 13:07]:
> Am Do., 7. Jan. 2021 um 08:59 Uhr schrieb Jean Louis <bugs@gnu.support>:
> >
> > (defun db/table-id-plist (table id pg)
> >   "Returns plist for table by its table_id"
> >   (let* ((columns (db/table-columns table pg))
> >          (sql (db/sql-by-columns table columns id))
> >          (values (rcd-sql-first sql pg))
> >          (values (append values nil))
> >          (length (length columns))
> >          (plist '()))
> >     (dotimes (i length plist)
> >       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
> >
> >
> > What is proper way to avoid this warning in `dolist' or `dotimes':
> >
> > rcd-db.el:841:1: Warning: Unused lexical variable ‘i’
> 
> This is https://debbugs.gnu.org/cgi/bugreport.cgi?bug=16206. tl;dr:
> don't use the three-argument form of `dotimes':
> 
> (dotimes (i length)  ; no `plist' here
>    ...)
> plist

Yes, thank you, that solves the problem.

But it implies that bug should be rather solved as function
description for `dotimes' allows return value as third argument.

This way is upside down, we tell programmers to not use function that
was defined as usable.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Philipp Stephani]

Am Do., 7. Jan. 2021 um 12:15 Uhr schrieb Jean Louis <bugs@gnu.support>:
>
> * Philipp Stephani <p.stephani2@gmail.com> [2021-01-07 13:07]:
> > Am Do., 7. Jan. 2021 um 08:59 Uhr schrieb Jean Louis <bugs@gnu.support>:
> > >
> > > (defun db/table-id-plist (table id pg)
> > >   "Returns plist for table by its table_id"
> > >   (let* ((columns (db/table-columns table pg))
> > >          (sql (db/sql-by-columns table columns id))
> > >          (values (rcd-sql-first sql pg))
> > >          (values (append values nil))
> > >          (length (length columns))
> > >          (plist '()))
> > >     (dotimes (i length plist)
> > >       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
> > >
> > >
> > > What is proper way to avoid this warning in `dolist' or `dotimes':
> > >
> > > rcd-db.el:841:1: Warning: Unused lexical variable ‘i’
> >
> > This is https://debbugs.gnu.org/cgi/bugreport.cgi?bug=16206. tl;dr:
> > don't use the three-argument form of `dotimes':
> >
> > (dotimes (i length)  ; no `plist' here
> >    ...)
> > plist
>
> Yes, thank you, that solves the problem.
>
> But it implies that bug should be rather solved as function
> description for `dotimes' allows return value as third argument.
>
> This way is upside down, we tell programmers to not use function that
> was defined as usable.

The discussion in that bug implies that it won't get fixed. Probably
we should discourage the three-argument form more strongly, e.g. by
removing it from the documentation and changing the advertised calling
convention.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Philipp Stephani wrote:

> The discussion in that bug implies that it won't get fixed.
> Probably we should discourage the three-argument form more
> strongly, e.g. by removing it from the documentation and
> changing the advertised calling convention.

What about fixing the byte-compiler so it won't warn a used
variable is unused? :)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Drew Adams]

[Forwarding to the list - forgot to explicitly add it again...]

> What about fixing the byte-compiler so it won't warn a
> used variable is unused? :)

Yes.  To which I expect the reply "Patch welcome". ;-)

The same inappropriate compiler warning is issued for
dotimes and cl-dotimes (the Common Lisp emulation).

Emacs Lisp has decided that use of the RETURN arg is
"deprecated" - even for the CL emulation, cl-dotimes.

The compiler shouldn't issue this warning, IMO.

[The bug thread claims Common Lisp's allowing the
RETURN arg is a "misfeature".]

But is this faulty warning a big deal?  No, not really.

Elisp byte-compiler warnings can be helpful.  But they
can also often be noisy and sometimes wrong/misleading.
Warning about warnings: Caveat lector.

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2021-01-07 19:06]:
> [Forwarding to the list - forgot to explicitly add it again...]
> 
> > What about fixing the byte-compiler so it won't warn a
> > used variable is unused? :)
> 
> Yes.  To which I expect the reply "Patch welcome". ;-)
> 
> The same inappropriate compiler warning is issued for
> dotimes and cl-dotimes (the Common Lisp emulation).
> 
> Emacs Lisp has decided that use of the RETURN arg is
> "deprecated" - even for the CL emulation, cl-dotimes.

What we talk is return value in third place of `(dolist (first second
third))' and it is strange that if return value is asked from `dolist'
that then the `first' one is warned about.

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> What we talk is return value in third place of `(dolist (first second
> third))' and it is strange that if return value is asked from `dolist'
> that then the `first' one is warned about.

The third arg is different from "the expression after `dotimes`" in one
important respect: the third arg can refer to the iteration variable
which will contain the "last" value:

    (dotimes (i 10 i) nil)

will return 10.
So the warning comes when you use that 3rd arg but without referring to
the iteration variable: in that case, you could have put the result
*after* `dotimes` rather than putting it in the 3rd arg.


        Stefan

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 05:31]:
> > What we talk is return value in third place of `(dolist (first second
> > third))' and it is strange that if return value is asked from `dolist'
> > that then the `first' one is warned about.
> 
> The third arg is different from "the expression after `dotimes`" in one
> important respect: the third arg can refer to the iteration variable
> which will contain the "last" value:
> 
>     (dotimes (i 10 i) nil)
> 
> will return 10.
> So the warning comes when you use that 3rd arg but without referring to
> the iteration variable: in that case, you could have put the result
> *after* `dotimes` rather than putting it in the 3rd arg.

I did not understand it, show me example.

Would that example remove compiler warning?

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> What we talk is return value in third place of `(dolist
>>> (first second third))' and it is strange that if return
>>> value is asked from `dolist' that then the `first' one is
>>> warned about.
>> 
>> The third arg is different from "the expression after
>> `dotimes`" in one important respect: the third arg can
>> refer to the iteration variable which will contain the
>> "last" value:
>> 
>>     (dotimes (i 10 i) nil)
>> 
>> will return 10.
>> So the warning comes when you use that 3rd arg but without
>> referring to the iteration variable: in that case, you
>> could have put the result *after* `dotimes` rather than
>> putting it in the 3rd arg.
>
> I did not understand it, show me example.

... :D

> Would that example remove compiler warning?

Yes.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

When replacing `dotimes' I also discover some of my errors:

(defun list-append-first-to-first-cons (list)
    "Cons with FIRST-ELT . FIRST-ELT"
  (let ((new-list '())
 (times (length list)))
    (dotimes (nr times new-list)
      (let* ((item (elt list nr)))
 (push (cons item item) new-list)))))

(list-append-first-to-first-cons '(1 2 3))
((3 . 3) (2 . 2) (1 . 1))

Then I replace it like:

(defun list-append-first-to-first-cons (list)
    "Cons with FIRST-ELT . FIRST-ELT"
    (mapcar (lambda (item) (cons item item)) list))

(list-append-first-to-first-cons '(1 2 3))
((1 . 1) (2 . 2) (3 . 3))

The reverse order was not detectable as results were used always in
completion lists and I have not ever put attention on the order.

In general I can replace `dotimes' without return value with `while'
and those that shall return values as sequences with mapping functions. 

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

>  (times (length list)))
>     (dotimes (nr times new-list)
>       (let* ((item (elt list nr)))

Please don't do that: such a "dotimes + elt" is fine when iterating over
a *vector*, but when iterating over a list it means it'll take time O(n²)
since `elt` on a list takes time O(n).

So it's *much* better to use `dolist` (or `mapc` or ...) in that case.


        Stefan

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

Now replacing:

(defun list-append-elts-to-first-cons (list)
    "Cons with FIRST-ELT . FIRST-ELT OTHER-ELT"
  (let ((new-list '())
 (times (length list)))
    (dotimes (nr times new-list)
      (let* ((item (elt list nr))
      (first (car item)))
 (push (cons first (list item)) new-list)))))

(list-append-elts-to-first-cons '((1 2 3)))
((1 (1 2 3)))

(defun list-append-elts-to-first-cons (list)
  "Cons with FIRST-ELT . FIRST-ELT OTHER-ELT"
  (mapcar (lambda (item) (cons (car item) (list item))) list))

(list-append-elts-to-first-cons '((1 2 3)))
((1 (1 2 3)))

RE: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Drew Adams]

> The third arg is different from "the expression after `dotimes`" in one
> important respect: the third arg can refer to the iteration variable
> which will contain the "last" value:
>     (dotimes (i 10 i) nil)
> will return 10.

In Common Lisp, at least, both dolist and dotimes let the return expression you specify refer to the iteration variable.  The difference is that for dolist the value at the end is nil; for dotimes it is the number of iterations.

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

>> The third arg is different from "the expression after `dotimes`" in one
>> important respect: the third arg can refer to the iteration variable
>> which will contain the "last" value:
>>     (dotimes (i 10 i) nil)
>> will return 10.
>
> In Common Lisp, at least, both dolist and dotimes let the return expression
> you specify refer to the iteration variable.  The difference is that for
> dolist the value at the end is nil; for dotimes it is the number
> of iterations.

IIRC we consciously decided to break this compatibility in `dolist`
because a variable that's always bound to nil is not useful enough to
justify the extra hassle.


        Stefan

Re: FW: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Drew Adams wrote:

> Elisp byte-compiler warnings can be helpful. But they can
> also often be noisy and sometimes wrong/misleading.
> Warning about warnings: Caveat lector.

Individual warnings can be incorrect (if so, they are bugs)
but the warning concept is good.

In the future compilers will be able to refactor and polish
code to the degree it won't be possible to determine who wrote
it, a seasoned pro, or a guy they just found on the street,
who seems to have an unusual talent: what is really simple, he
seems to understand almost instantly...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-01-07 14:39]:
> Philipp Stephani wrote:
> 
> > The discussion in that bug implies that it won't get fixed.
> > Probably we should discourage the three-argument form more
> > strongly, e.g. by removing it from the documentation and
> > changing the advertised calling convention.
> 
> What about fixing the byte-compiler so it won't warn a used
> variable is unused? :)

That is what I was thinking, or some other way of programming to
declare the variable.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>>> The discussion in that bug implies that it won't get
>>> fixed. Probably we should discourage the three-argument
>>> form more strongly, e.g. by removing it from the
>>> documentation and changing the advertised
>>> calling convention.
>> 
>> What about fixing the byte-compiler so it won't warn a used
>> variable is unused? :)
>
> That is what I was thinking, or

Not "or", and :)

> some other way of programming to declare the variable.

Use `cl-loop', the `for' variant, for this.

No manual/explicit "length" business, no byte-compiler issues,
looks and works great.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> The discussion in that bug implies that it won't get fixed. Probably
> we should discourage the three-argument form more strongly, e.g. by
> removing it from the documentation and changing the advertised calling
> convention.

Indeed, I don't like this 3-arg form of `dotimes` and `dolist` because
it tends to "hide" the return value.  But we keep it because there's
something to be said for not being "gratuitously" different from
Common-Lisp.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-07 18:41]:
> > The discussion in that bug implies that it won't get fixed. Probably
> > we should discourage the three-argument form more strongly, e.g. by
> > removing it from the documentation and changing the advertised calling
> > convention.
> 
> Indeed, I don't like this 3-arg form of `dotimes` and `dolist` because
> it tends to "hide" the return value.  But we keep it because there's
> something to be said for not being "gratuitously" different from
> Common-Lisp.

I do not see how it hides when I know how it is used from Common
Lisp. So I look there in third value and I know which one will be
returned.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

>> Indeed, I don't like this 3-arg form of `dotimes` and `dolist` because
>> it tends to "hide" the return value.  But we keep it because there's
>> something to be said for not being "gratuitously" different from
>> Common-Lisp.
>
> I do not see how it hides when I know how it is used from Common
> Lisp.  So I look there in third value and I know which one will be
> returned.

It's hidden because it's not at the usual place where the return value
is usually placed (i.e. in the last expression).

There are several other macros/specialforms that also place the return
value elsewhere than the last expression, such as `unwind-protect`, (and
to a lesser extent `if`, `cond`, `condition-case`, `prog1`), but these
still place the return value in one of the immediate subexpressions,
many of them use indentation to try and make the return value "stand
out" from the rest, etc...

In contrast in `dotimes` and `dolist` it's placed in
a sub-sub-expression and moreover it's one that's rarely used (arguably
*because* it's too deeply nested), so you're likely to miss it if you
don't go looking for it very actively.

Not that it matters anyway,


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier wrote:

> It's hidden because it's not at the usual place where the
> return value is usually placed (i.e. in the last
> expression).
>
> There are several other macros/specialforms that also place
> the return value elsewhere than the last expression, such as
> `unwind-protect`, (and to a lesser extent `if`, `cond`,
> `condition-case`, `prog1`), but these still place the return
> value in one of the immediate subexpressions, many of them
> use indentation to try and make the return value "stand out"
> from the rest, etc...
>
> In contrast in `dotimes` and `dolist` it's placed in
> a sub-sub-expression and moreover it's one that's rarely
> used (arguably *because* it's too deeply nested), so you're
> likely to miss it if you don't go looking for it
> very actively.
>
> Not that it matters anyway,

It matters in the sense I just enjoyed reading that.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 05:22]:
> >> Indeed, I don't like this 3-arg form of `dotimes` and `dolist` because
> >> it tends to "hide" the return value.  But we keep it because there's
> >> something to be said for not being "gratuitously" different from
> >> Common-Lisp.
> >
> > I do not see how it hides when I know how it is used from Common
> > Lisp.  So I look there in third value and I know which one will be
> > returned.
> 
> It's hidden because it's not at the usual place where the return value
> is usually placed (i.e. in the last expression).

I understand your viewpoint. But I learned it in Common Lisp and for
me it is not hidden. I know that nothing will be returned unless
declared what to return. And I just guess, that `dotimes' was included
in Emacs Lisp as to duplicate Common Lisp.

However, I try to have Emacs Lisp mostly clean from Common Lisp
constructs. So I may change my `dotimes' forms into `while' forms or
where it applies to some of the mapping functions.

> In contrast in `dotimes` and `dolist` it's placed in
> a sub-sub-expression and moreover it's one that's rarely used (arguably
> *because* it's too deeply nested), so you're likely to miss it if you
> don't go looking for it very actively.
> 
> Not that it matters anyway

Maybe somebody would miss, but I don't. This may be because forms I
have are rather short, example:

(defun pct-plus (amount percent)
  "Return AMOUNT increased for PERCENT of it."
  (let* ((percent (* amount percent))
         (result (+ amount percent)))
    result))

(defun pct-plus-times (amount percent times)
  "Return AMOUNT increased for PERCENT for number of TIMES."
  (dotimes (var times amount)
    (setq amount (pct-plus amount percent))))

And I see that `var' is unused there. I have never used those
functions in a package so there were no warnings.

Then I can replace the above function to following:

(defun pct-plus-times (amount percent times)
  "Return AMOUNT increased for PERCENT for number of TIMES."
  (let ((n 0))
    (while (/= n times)
      (setq amount (pct-plus amount percent))
      (setq n (1+ n)))
    amount))

And it does the same hopefully, it appears to me more Emacs Lisp-ish,
but looks uglier, though it is more understandable to me and it is the
kind of what I learned about loops from old times.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> (defun pct-plus-times (amount percent times)
>   "Return AMOUNT increased for PERCENT for number of TIMES."
>   (dotimes (var times amount)
>     (setq amount (pct-plus amount percent))))
>
> And I see that `var' is unused there. I have never used those
> functions in a package so there were no warnings.

You can eliminate the warning as follows:

    (defun pct-plus-times (amount percent times)
      "Return AMOUNT increased for PERCENT for number of TIMES."
      (dotimes (var times)
        (setq amount (pct-plus amount percent)))
      amount)

No need to resort to `while`.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 06:41]:
> > (defun pct-plus-times (amount percent times)
> >   "Return AMOUNT increased for PERCENT for number of TIMES."
> >   (dotimes (var times amount)
> >     (setq amount (pct-plus amount percent))))
> >
> > And I see that `var' is unused there. I have never used those
> > functions in a package so there were no warnings.
> 
> You can eliminate the warning as follows:
> 
>     (defun pct-plus-times (amount percent times)
>       "Return AMOUNT increased for PERCENT for number of TIMES."
>       (dotimes (var times)
>         (setq amount (pct-plus amount percent)))
>       amount)
> 
> No need to resort to `while`.

To me that means that I need to use workaround because `dotimes' does
not work well as described in docstring. I need to remember something
that is nowhere documented.

Either compiler warning is wrong, or docstring is wrong. One of those
shall be improved.

Users or programmers shall not be directed to avoid using function
which is described how to be used in their documentation.

Isn't it? No logic there for me.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> To me that means that I need to use workaround because `dotimes' does
> not work well as described in docstring. I need to remember something
> that is nowhere documented.

In which way doesn't it work as documented?
The warning is just that: a warning.  It doesn't affect the actual behavior.

> Either compiler warning is wrong, or docstring is wrong.  One of those
> shall be improved.

AFAIK both are right: you put into the 3rd arg an expression that is
evaluated in a context where a new variable `var` has been added for you
but you don't make use of that variable.  This fact doesn't prevent the
code from being valid and executed correctly, but it is suspicious so it
deserves a warning.

> Users or programmers shall not be directed to avoid using function
> which is described how to be used in their documentation.

"Warning" is not the same as "error".

A warning is emitted when the code is valid but where we think it's
useful to bring the attention to something of which the programmer may
not be aware.  E.g. in this case, the fact that your code has a useless
extra `let` as you can see in the macro-expanded version of the code:

    (macroexpand '(dotimes (var times amount)
                    (setq amount (pct-plus amount percent))))
=>
    (let ((--dotimes-limit-- times)
          (--dotimes-counter-- 0))
      (while (< --dotimes-counter-- --dotimes-limit--)
        (let ((var --dotimes-counter--))
          (setq amount (pct-plus amount percent)))
          (setq --dotimes-counter-- (1+ --dotimes-counter--)))
      (let ((var --dotimes-counter--)) amount))

Many/most of the warnings depend on value judgments which depend on what
is considered "good style", so of course there is bound to be people
who disagree in some cases.

We occasionally also use those warnings to nudge programmers toward
a particular programming style, i.e. with a conscious decision to make
people change their style, which makes it yet more likely that some
people will disagree (at least at first, until they get used to the new
style).  E.g. in early Emacs it was a lot more common to find packages
which relied on non-trivial ways uses of dynamic scoping and undeclared
global vars.  After adding warnings for uses of "free variables" the
style changed over the years, making it possible in Emacs-24 to
introduce lexical scoping such that the vast majority of the code works
unchanged when `lexical-binding` is activated.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Robert Thorpe]

I'm still struggling to see how this isn't a bug.

Here is the original code that Jean Louis posted:

> (defun db/table-id-plist (table id pg)
>   "Returns plist for table by its table_id"
>   (let* ((columns (db/table-columns table pg))
> 	 (sql (db/sql-by-columns table columns id))
> 	 (values (rcd-sql-first sql pg))
> 	 (values (append values nil))
> 	 (length (length columns))
> 	 (plist '()))
>     (dotimes (i length plist)
>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
 
And here is the original warning:

> rcd-db.el:841:1: Warning: Unused lexical variable ‘i’

Jean Louis actually is using the variable i, it's not unused.

You could argue that the warning should be "The three element version of
dotimes is discouraged".  But I can't see how you can argue that the
warning is right, since i is used.

BR,
Robert Thorpe

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Robert Thorpe wrote:

> I'm still struggling to see how this isn't a bug.

It is a bug alright, somewhere around

  (byte-compile-warn "Unused lexical %s `%S'%s"
                              varkind var
                              (if suggestions (concat "\n  " suggestions) "")))))

lines 580-582A

emacs-28.0.50/lisp/emacs-lisp/cconv.el

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

>>     (dotimes (i length plist)
>>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))))
> And here is the original warning:
>> rcd-db.el:841:1: Warning: Unused lexical variable ‘i’
> Jean Louis actually is using the variable i, it's not unused.

There are 2 places where `i` is introduced:
- one around the body of the loop
- another around the 3rd arg used as return value.
To see it you need to macroexpand the `dotimes`:

    (dotimes (i length plist)
      (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
=>
    (let ((--dotimes-limit-- length)
          (--dotimes-counter-- 0))
      (while (< --dotimes-counter-- --dotimes-limit--)
        (let ((i --dotimes-counter--))
          (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
        (setq --dotimes-counter-- (1+ --dotimes-counter--)))
      (let ((i --dotimes-counter--)) plist))

Clearly, this is not the only way to define the macro, but it's a valid
way (according to the Common Lisp definition, for example).
Now you hopefully see on the last time that `i` is indeed defined but
not used.  This is the code which the compiler sees.

You won't get this warning if you do:

    (dotimes (i length (progn (ignore i) plist))
      (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
or
    (dotimes (i length)
      (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
    plist

I'm not trying to argue that the byte-compiler warning here is
a feature, but in general the "unused var" byte compiler warnings are
very useful (especially when converting code to `lexical-binding`) and
while in this case it's kind of spurious it's very easy to change the
code so as to avoid the warning (and the resulting code is also more
efficient and arguably more readable for people not used to this 3rd
arg of dotimes/dolist), so this is not a very compelling motivation to
improve the warning machinery.

There can be situations when using `pcase` where the byte-compiler will
tend to emit similar warnings and these are more problematic because it
can be hard to rewrite the code to avoid the warning.

I haven't yet figured how to make it possible to avoid those warnings,
so in the `pcase` macro I added extra hacks to try and avoid introducing
bindings that aren't used, which basically silence those warnings, with
the downside that you won't get the warning even in the case where it's
really warranted.

> You could argue that the warning should be "The three element version of
> dotimes is discouraged".  But I can't see how you can argue that the
> warning is right, since i is used.

The var-use analysis takes place after macro expansion.

This is not specific to this warning, tho: all warnings suffer from
false positives, almost by definition.  If we knew for sure that the
warning points to a real "bug" we could turn it into an error.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Tomas Hlavaty]

On Fri 08 Jan 2021 at 02:10, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>       (let ((i --dotimes-counter--)) plist))

In Common Lisp, that part would be solved by (declare (ignorable i)) in
the macro:

(let ((i --dotimes-counter--))
  (declare (ignorable i))
  plist)

I do not know if Emacs Lisp has ignorable.
Maybe this is the missing bit?

> You won't get this warning if you do:
>
>     (dotimes (i length (progn (ignore i) plist))
>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
> or
>     (dotimes (i length)
>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
>     plist

This is just moving the problem from macro author to macro users.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

>>       (let ((i --dotimes-counter--)) plist))
>
> In Common Lisp, that part would be solved by (declare (ignorable i)) in
> the macro:
>
> (let ((i --dotimes-counter--))
>   (declare (ignorable i))
>   plist)
>
> I do not know if Emacs Lisp has ignorable.

We have something equivalent:

    (let ((i --dotimes-counter--))
      (ignore i)
      plist)

[ Which "comes for free" in the sense that it is not the result of
  deliberate design but rather the simple fact that the `ignore`
  function is usually optimized away (but it's optimized late enough
  that the `i` passed to it still counts as a use of that variable).  ]

>> You won't get this warning if you do:
>>
>>     (dotimes (i length (progn (ignore i) plist))
>>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
>> or
>>     (dotimes (i length)
>>       (setf plist (plist-put plist (intern (elt columns i)) (elt values i))))
>>     plist
>
> This is just moving the problem from macro author to macro users.

Indeed, and it reflects the fact that some of those who get to decide
(e.g. yours truly) don't like this 3rd arg.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Tomas Hlavaty]

On Sat 09 Jan 2021 at 14:01, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>       (let ((i --dotimes-counter--)) plist))
>>
>> In Common Lisp, that part would be solved by (declare (ignorable i)) in
>> the macro:
>>
>> (let ((i --dotimes-counter--))
>>   (declare (ignorable i))
>>   plist)
>>
>> I do not know if Emacs Lisp has ignorable.
>
> We have something equivalent:
>
>     (let ((i --dotimes-counter--))
>       (ignore i)
>       plist)
>
> [ Which "comes for free" in the sense that it is not the result of
>   deliberate design but rather the simple fact that the `ignore`
>   function is usually optimized away (but it's optimized late enough
>   that the `i` passed to it still counts as a use of that variable).  ]

I see that Emacs Lisp does not have a way to distinguish the two cases.

In Common Lisp, (declare (ignore i)) means ignore unused variable i.  If
i is used, I get a warning.  (declare (ignorable i)) means ignore the
variable in case it is unused, which is useful in cases like the one
discussed here where the author of the macro does not know upfront, if
the variable i should or should not be ignored.

If the dotimes macro contained (ignore i) but the user returned i, I
would expect warning:

(let ((i --dotimes-counter--))
  (ignore i)
  i)

It is a shame that Emacs Lisp does not do that.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> I see that Emacs Lisp does not have a way to distinguish the two cases.
>
> In Common Lisp, (declare (ignore i)) means ignore unused variable i.  If
> i is used, I get a warning.

ELisp's equivalent is to use a variable name with a leading underscore
such as `_i` (or just `_`).


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Tomas Hlavaty]

On Sun 10 Jan 2021 at 09:52, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> I see that Emacs Lisp does not have a way to distinguish the two cases.
>>
>> In Common Lisp, (declare (ignore i)) means ignore unused variable i.  If
>> i is used, I get a warning.
>
> ELisp's equivalent is to use a variable name with a leading underscore
> such as `_i` (or just `_`).

That's good to know, thank you!

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 08:14]:
> > To me that means that I need to use workaround because `dotimes' does
> > not work well as described in docstring. I need to remember something
> > that is nowhere documented.
> 
> In which way doesn't it work as documented?
> The warning is just that: a warning.  It doesn't affect the actual behavior.
> 
> > Either compiler warning is wrong, or docstring is wrong.  One of those
> > shall be improved.
> 
> AFAIK both are right: you put into the 3rd arg an expression that is
> evaluated in a context where a new variable `var` has been added for you
> but you don't make use of that variable.  This fact doesn't prevent the
> code from being valid and executed correctly, but it is suspicious so it
> deserves a warning.

I think there is some mistake here.

I have placed third value where I use the third value, but compiler warning is complaining about first one.

For example in (dolist (n 10 return) ...) compiler would warn about `n' and not about `return'.

If it complains about `return' when not used, that would be just fine
for me, good warning and logical. But I do not speak of that case.

Complete example, in file new.el:

;; -*- lexical-binding: t; -*-

(defun my-fun ()
  (let ((list '()))
    (dotimes (n 10 list)
      (push n list))))

Compiling file /home/data1/protected/new.el at Fri Jan  8 08:34:04 2021
new.el:3:1: Warning: Unused lexical variable n

So that is what I speak about. The `n' variable is assigned right
there similar to `let' so I think that maybe macro shall be improved.

> A warning is emitted when the code is valid but where we think it's
> useful to bring the attention to something of which the programmer may
> not be aware.

I wish to make all programs warning free. If I need to get aware of
some function than such warning should be placed in the function
documentation. For example, what I was thinking first is that `n' from
above example should be somehow already declared. The docstring of
`dotimes' should tell me something about that. But we have here hidden
anti-features that are not documented, known only to developers and
available only in mailing list messages.

Sorry I cannot understand the macro expansion at this moment, but
thank you. Maybe somebody else understands it.

> We occasionally also use those warnings to nudge programmers toward
> a particular programming style, i.e. with a conscious decision to make
> people change their style, which makes it yet more likely that some
> people will disagree (at least at first, until they get used to the new
> style).  E.g. in early Emacs it was a lot more common to find packages
> which relied on non-trivial ways uses of dynamic scoping and undeclared
> global vars.  After adding warnings for uses of "free variables" the
> style changed over the years, making it possible in Emacs-24 to
> introduce lexical scoping such that the vast majority of the code works
> unchanged when `lexical-binding` is activated.

That is great. 

Only it is not relevant to macro or function that is documented to
work, but then again it gives warnings for `n' and not for `return' as
in above example. 

I am surprised.

For me personally I will stick to the built-in functions such as
`while' and mapping functions and will rather try to avoid `dotimes'
and `dolist', unless the error is corrected.

Jean

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> Only it is not relevant to macro or function that is documented to
> work, but then again it gives warnings for `n' and not for `return' as
> in above example. 

Not sure what you mean by "documented", but `C-h f dotimes` says:

    dotimes is a Lisp macro in ‘subr.el’.
    
    (dotimes (VAR COUNT [RESULT]) BODY...)
    
      Probably introduced at or before Emacs version 21.1.
    
    Loop a certain number of times.
    Evaluate BODY with VAR bound to successive integers running from 0,
    inclusive, to COUNT, exclusive.
    
    Finally RESULT is evaluated to get the return value (nil if
    RESULT is omitted).  Using RESULT is deprecated, and may result
    in compilation warnings about unused variables.

Notice the last sentence.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier wrote:

>> Only it is not relevant to macro or function that is
>> documented to work, but then again it gives warnings for
>> `n' and not for `return' as in above example.
>
> Not sure what you mean by "documented", but `C-h f dotimes` says:
>
>     dotimes is a Lisp macro in ‘subr.el’.
>
>     (dotimes (VAR COUNT [RESULT]) BODY...)
>
>       Probably introduced at or before Emacs version 21.1.
>
>     Loop a certain number of times. Evaluate BODY with VAR
>     bound to successive integers running from 0, inclusive,
>     to COUNT, exclusive.
>
>     Finally RESULT is evaluated to get the return value (nil
>     if RESULT is omitted). Using RESULT is deprecated, and
>     may result in compilation warnings about
>     unused variables.
>
> Notice the last sentence.

If the byte compiler says this guy has a problem with this
function, and this isn't so, then the actual problem is with
the byte compiler.

Question: When these features are removed for real and for
good from them functions, does that mean the byte compiler
problem will never appear again?

Question 2: Why is this problem so difficult to solve at its
source? What's with these functions that enables them to
cloak, if you will, so the byte compiler cannot se what the
human eye can see instantly, looking at the same material?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1304 bytes --]

On Fri, Jan 08, 2021 at 09:28:58AM +0100, Emanuel Berg via Users list for the GNU Emacs text editor wrote:

[...]

> Question 2: Why is this problem so difficult to solve at its
> source? What's with these functions that enables them to
> cloak, if you will, so the byte compiler cannot se what the
> human eye can see instantly, looking at the same material?

I think Stefan explained that pretty well in this thread:
dotimes is implemented as a macro (the alternative  would
be to special-case it) which introduces a let-binding to
the counting index... which is used in the result part,
unless it's not used, which then causes the byte compiler
to issue the warning.

Fixing that would take

 (a) special-casing dotimes somehow. For maintaniability
    reasons probably out of the question
 (b) making the macro expander for dotimes smarter, to
    skip introducing the let-binding. I'm not good enough
    at macrology to even fathom what that means. Possibly
    we end up with a macro three times as complex for very
    little gain
 (c) (that seems Stefan's favourite, and I tend to trust
    him blindly in those things) discourage the three
    argument use of dotimes, since it's not that idiomatic
    in Elisp anyway (those idioms are more Common Lisp-y).

Makes more sense now?

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

tomas wrote:

> I think Stefan explained that pretty well in this thread:
> dotimes is implemented as a macro (the alternative would be
> to special-case it) which introduces a let-binding

But surely this happens elsewhere as well?

> to the counting index... which is used in the result part,
> unless it's not used, which then causes the byte compiler to
> issue the warning.

But the warning isn't specifically introduced for this
situation, is it? Because then it could just say that, instead
of the confusing wasn't used thing.

But if it isn't, then why is it triggered by this at all and
how can it be so specific to even act upon a certain part of
the construct?

> Fixing that would take
>
>  (a) special-casing dotimes somehow. For maintaniability
>      reasons probably out of the question

No.

>  (b) making the macro expander for dotimes smarter, to
>     skip introducing the let-binding.

No.

>  (c) (that seems Stefan's favourite, and I tend to trust him
>      blindly in those things) discourage the three argument
>      use of dotimes, since it's not that idiomatic in Elisp
>      anyway (those idioms are more Common Lisp-y).

Ha :) Well, whatever.

> Makes more sense now?

No! Again, this is a problem with the byte compiler.
So rather, fixing it would take

   (d) fixing the byte compiler

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 3264 bytes --]

On Fri, Jan 08, 2021 at 01:38:24PM +0100, Emanuel Berg via Users list for the GNU Emacs text editor wrote:
> tomas wrote:
> 
> > I think Stefan explained that pretty well in this thread:
> > dotimes is implemented as a macro (the alternative would be
> > to special-case it) which introduces a let-binding
> 
> But surely this happens elsewhere as well?

Possibly, yes.

> > to the counting index... which is used in the result part,
> > unless it's not used, which then causes the byte compiler to
> > issue the warning.
> 
> But the warning isn't specifically introduced for this
> situation, is it?

No, it's not.

>                   Because then it could just say that, instead
> of the confusing wasn't used thing.


> 
> But if it isn't, then why is it triggered by this at all and
> how can it be so specific to even act upon a certain part of
> the construct?
> 
> > Fixing that would take
> >
> >  (a) special-casing dotimes somehow. For maintaniability
> >      reasons probably out of the question
> 
> No.
> 
> >  (b) making the macro expander for dotimes smarter, to
> >     skip introducing the let-binding.
> 
> No.
> 
> >  (c) (that seems Stefan's favourite, and I tend to trust him
> >      blindly in those things) discourage the three argument
> >      use of dotimes, since it's not that idiomatic in Elisp
> >      anyway (those idioms are more Common Lisp-y).
> 
> Ha :) Well, whatever.
> 
> > Makes more sense now?
> 
> No! Again, this is a problem with the byte compiler.
> So rather, fixing it would take

No. It's a problem somewhere between the macro expander
and the byte compiler. Or with the idiom's design itself,
depending on how you squint.

>    (d) fixing the byte compiler

No. The byte compiler can't know. Fixing the macro expander
perhaps. Or introducing a special communication channel
between both ("Hey, byte compiler: this is the macro
expander talking. This funny variable "i2975" you are seeing
has been introduced by me behind the user's back, because
she insists on using that quaint and curious three-args
dotimes macro. So don't warn if that variable is unused.
But then, perhaps it being unused isn't what's intended,
so please -- uh -- do whatever"

Have you had a look at a dotimes macro expansion?

Try:

  (macroexpand
    '(dotimes (i 10 0)
        (message "%d\n" i)))

There you'll find that last thingy:

  (let ((i --dotimes-counter--))
    0)

(i.e. the RESULT arg in dotimes is there, but it's NOT using
the count variable. There it is! An unused variable! And the
byte compiler wasn't involved yet at all. When it sees that,
it'll balk at the unused i. Fixing the byte compiler seems
like the least attractive variant to me.

Fixing the macro definition is a bit more attractive (e.g.
it could try to decide whether RESULT depends on i and
not generate that let-binding if not), but I can't decide
right now whether that involves the halting problem or not.

Here's a challenge for you :) (I guess Stefan would accept
a patch /if/ it makes sense and isn't too painful otherwise).

Fixing the usage (or the user expectation) seems to me like
the most attractive way out, at the moment :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Philipp Stephani]

Am Fr., 8. Jan. 2021 um 14:14 Uhr schrieb <tomas@tuxteam.de>:

> Fixing the macro definition is a bit more attractive (e.g.
> it could try to decide whether RESULT depends on i and
> not generate that let-binding if not), but I can't decide
> right now whether that involves the halting problem or not.

Certainly not, since that's what the byte compiler is doing. It's just
a lexical walk through the code after macroexpansion.

(require 'cconv)
(let ((env '((VAR nil nil nil nil))))
  (cconv-analyze-form (macroexpand-all RESULT-FORM t) env)
  (cadr (assq VAR env)))

Should return t if VAR is used in RESULT-FORM.

But then, I don't know whether that's worth it or not. Alternatively,
you could add an (ignore VAR) to the macro expansion of the result
form so that cconv treats the variable as used.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

tomas wrote:

>> No! Again, this is a problem with the byte compiler.
>> So rather, fixing it would take
>
> No. It's a problem somewhere between the macro expander
> and the byte compiler.

The byte compiler, after user invocation, tells the user the
user isn't using the variable. When the user is using it.

This bug tho is easy to fix. Add to the warning string that it
refers not to what the user has just written and what is
referenced by the byte compiler, BUT to what is macro-expanded
(one will have to do this manually, in scratch, I guess), and
maybe certain parts of that, even. The second let?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

So here, the byte compiler isn't concerned with what is
written textually as code, it is rather a tool to
debug macros?

OK, that's another thing than what I thought that message
referred to... I don't really care for macros so I can then
disengage from the discussion.

Most users I dare say tho who reads that will think it refers
to THEIR use of the variable...

ESPECIALLY since they are not developing any macros, they are
simply USING `dotimes'  or `dolist'!

ELLER HUR???

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Philipp Stephani]

Am Fr., 8. Jan. 2021 um 13:39 Uhr schrieb Emanuel Berg via Users list
for the GNU Emacs text editor <help-gnu-emacs@gnu.org>:

> No! Again, this is a problem with the byte compiler.

Not really. The byte compiler can only compile what's macro-expanded,
and the macro-expansion of e.g. (dotimes (i 5 t) i) is roughly:

(let ((counter 0))
  (while (< counter 5)
    (let ((i counter))
      i)
    (setq counter (1+ counter)))
  (let ((i counter))
    t))

As you can see, `i' is definitely unused in the second `let' form -
the byte compiler is right.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Philipp Stephani wrote:

>> No! Again, this is a problem with the byte compiler.
>
> Not really. The byte compiler can only compile what's
> macro-expanded

If the byte compiler because of its nature is incapable of
doing something, it shouldn't attempt or say anything...

> and the macro-expansion of e.g. (dotimes (i 5 t) i) is
> roughly:
>
> (let ((counter 0))
>   (while (< counter 5)
>     (let ((i counter))
>       i)
>     (setq counter (1+ counter)))
>   (let ((i counter))
>     t))
>
> As you can see, `i' is definitely unused in the second `let'
> form - the byte compiler is right.

Hm... what is the definition of being used in terms of
a variable? read/write?

Also, that is a stub, I don't remember what the OP did but he
did use it and I think this is what one thinks the byte
compiler is referring to with its warning rather than certain
parts of what is macro-expanded...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1739 bytes --]

On Fri, Jan 08, 2021 at 02:31:11PM +0100, Emanuel Berg via Users list for the GNU Emacs text editor wrote:
> Philipp Stephani wrote:
> 
> >> No! Again, this is a problem with the byte compiler.
> >
> > Not really. The byte compiler can only compile what's
> > macro-expanded
> 
> If the byte compiler because of its nature is incapable of
> doing something, it shouldn't attempt or say anything...

No. The byte compiler is doing exactly what it's been
designed to: dependency analysis. And it making that
"knowledge" it needs anyway available to the programmer
via a warning. That is a Good Thing. And it is even
right in its analysis: there /is/ an let-declared variable
there which is unused.

The macro expander put it there. I think Philipp and me
managed to convince you of that?

[...]

> >   (let ((i counter))
> >     t))
> >
> > As you can see, `i' is definitely unused in the second `let'
> > form - the byte compiler is right.
> 
> Hm... what is the definition of being used in terms of
> a variable? read/write?

I think it's just access (i.e. read). But you could try for yourself
(just cook up a RETURN expression where the count variable is
set and compile that code. Perhaps better don't run it :)

> Also, that is a stub, I don't remember what the OP did but he
> did use it

You mean he used the count variable? In the context of the
RETURN expression? And then (s)he gets the warning? I'd be
surprised. We'd be digging in the wrong place, then.

>            and I think this is what one thinks the byte
> compiler is referring to with its warning rather than certain
> parts of what is macro-expanded...

Hm. Could you provide the original code snippet?

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 10:16]:
> > Only it is not relevant to macro or function that is documented to
> > work, but then again it gives warnings for `n' and not for `return' as
> > in above example. 
> 
> Not sure what you mean by "documented", but `C-h f dotimes` says:
> 
>     dotimes is a Lisp macro in ‘subr.el’.
>     
>     (dotimes (VAR COUNT [RESULT]) BODY...)
>     
>       Probably introduced at or before Emacs version 21.1.
>     
>     Loop a certain number of times.
>     Evaluate BODY with VAR bound to successive integers running from 0,
>     inclusive, to COUNT, exclusive.
>     
>     Finally RESULT is evaluated to get the return value (nil if
>     RESULT is omitted).  Using RESULT is deprecated, and may result
>     in compilation warnings about unused variables.
> 
> Notice the last sentence.

Now I do notice it.

I do switch to `while' and mapping functions rather.

Jean

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Leo Butler]

Stefan,

I wonder why the lexical-binding version of dotimes is not implemented
like:

#+begin_src emacs-lisp
  (let* ((start 0) (end (nth 1 spec)) (counter (gensym))
	 (body (subst counter (car spec) body)))
          `(let ((,counter ,start))
             (while (< ,counter ,end)
	       ,@body
	       (setq ,counter (1+ ,counter)))
	     ,@ (cddr spec))))
#+end_src

I am familiar with the comment in subr.el about uninterned symbols, but
TBH, I don't have a sense of where the performance penalty might be
of any significant size.

Leo

Jean Louis <bugs@gnu.support> writes:

> ********************************************************
> Caution: This message was sent from outside the University of Manitoba.
> ********************************************************
>
> * Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 10:16]:
>> > Only it is not relevant to macro or function that is documented to
>> > work, but then again it gives warnings for `n' and not for `return' as
>> > in above example. 
>> 
>> Not sure what you mean by "documented", but `C-h f dotimes` says:
>> 
>>     dotimes is a Lisp macro in ‘subr.el’.
>>     
>>     (dotimes (VAR COUNT [RESULT]) BODY...)
>>     
>>       Probably introduced at or before Emacs version 21.1.
>>     
>>     Loop a certain number of times.
>>     Evaluate BODY with VAR bound to successive integers running from 0,
>>     inclusive, to COUNT, exclusive.
>>     
>>     Finally RESULT is evaluated to get the return value (nil if
>>     RESULT is omitted).  Using RESULT is deprecated, and may result
>>     in compilation warnings about unused variables.
>> 
>> Notice the last sentence.
>
> Now I do notice it.
>
> I do switch to `while' and mapping functions rather.
>
> Jean

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Philipp Stephani]

Am Sa., 9. Jan. 2021 um 14:30 Uhr schrieb Leo Butler <leo.butler@umanitoba.ca>:
>
> Stefan,
>
> I wonder why the lexical-binding version of dotimes is not implemented
> like:
>
> #+begin_src emacs-lisp
>   (let* ((start 0) (end (nth 1 spec)) (counter (gensym))
>          (body (subst counter (car spec) body)))
>           `(let ((,counter ,start))
>              (while (< ,counter ,end)
>                ,@body
>                (setq ,counter (1+ ,counter)))
>              ,@ (cddr spec))))
> #+end_src

That would be a semantically different operation. `dotimes' guarantees
that the counter variable is let-bound within each iteration. That
means that the macro expansion needs to contain something like
(let ((VAR ...)) BODY)
This distinction becomes relevant if BODY modifies VAR or captures it
in a closure.

> I am familiar with the comment in subr.el about uninterned symbols, but
> TBH, I don't have a sense of where the performance penalty might be
> of any significant size.

The uninterned vs. interned distinction is unrelated. `dotimes'
chooses to use an interned symbol as a micro-optimization, but it
doesn't affect its behavior.

>
> Leo
>
> Jean Louis <bugs@gnu.support> writes:
>
> > ********************************************************
> > Caution: This message was sent from outside the University of Manitoba.
> > ********************************************************
> >
> > * Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 10:16]:
> >> > Only it is not relevant to macro or function that is documented to
> >> > work, but then again it gives warnings for `n' and not for `return' as
> >> > in above example.
> >>
> >> Not sure what you mean by "documented", but `C-h f dotimes` says:
> >>
> >>     dotimes is a Lisp macro in ‘subr.el’.
> >>
> >>     (dotimes (VAR COUNT [RESULT]) BODY...)
> >>
> >>       Probably introduced at or before Emacs version 21.1.
> >>
> >>     Loop a certain number of times.
> >>     Evaluate BODY with VAR bound to successive integers running from 0,
> >>     inclusive, to COUNT, exclusive.
> >>
> >>     Finally RESULT is evaluated to get the return value (nil if
> >>     RESULT is omitted).  Using RESULT is deprecated, and may result
> >>     in compilation warnings about unused variables.
> >>
> >> Notice the last sentence.
> >
> > Now I do notice it.
> >
> > I do switch to `while' and mapping functions rather.
> >
> > Jean
>

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Leo Butler]

Philipp Stephani <p.stephani2@gmail.com> writes:

> Am Sa., 9. Jan. 2021 um 14:30 Uhr schrieb Leo Butler <leo.butler@umanitoba.ca>:
>>
>> Stefan,
>>
>> I wonder why the lexical-binding version of dotimes is not implemented
>> like:
>>
>> #+begin_src emacs-lisp
>>   (let* ((start 0) (end (nth 1 spec)) (counter (gensym))
>>          (body (subst counter (car spec) body)))
>>           `(let ((,counter ,start))
>>              (while (< ,counter ,end)
>>                ,@body
>>                (setq ,counter (1+ ,counter)))
>>              ,@ (cddr spec))))
>> #+end_src
>
> That would be a semantically different operation. `dotimes' guarantees
> that the counter variable is let-bound within each iteration. That
> means that the macro expansion needs to contain something like
> (let ((VAR ...)) BODY)
> This distinction becomes relevant if BODY modifies VAR or captures it
> in a closure.

Maybe I am being thick, but I cannot think of how there would be a
different behaviour.

Note that I am discussing only the lexical scope case. The macro
replaces each occurrence of VAR (=(car spec)) in BODY with the gensym.

>
>> I am familiar with the comment in subr.el about uninterned symbols, but
>> TBH, I don't have a sense of where the performance penalty might be
>> of any significant size.
>
> The uninterned vs. interned distinction is unrelated. `dotimes'
> chooses to use an interned symbol as a micro-optimization, but it
> doesn't affect its behavior.

The current dotimes macro is unhygenic, isn't it, whereas using an
uninterned symbol would make it hygenic. So, yes, I think there is a
difference in behavior.

Leo

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

>> This distinction becomes relevant if BODY modifies VAR or captures it
>> in a closure.
> Maybe I am being thick, but I cannot think of how there would be a
> different behaviour.

Try:

    (let (funs)
      (dotimes (i 5)
        (push (lambda () i) funs))
      (mapcar #'funcall funs))

IIUC the version you advocate would return

    (5 5 5 5 5)

whereas the current version returns

    (4 3 2 1 0)

Other differences occur if you do

    (let (vals)
      (dotimes (i 10)
        (push (setq i (1+ i)) vals))
      vals)


-- Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Leo Butler]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> ********************************************************
> Caution: This message was sent from outside the University of Manitoba.
> ********************************************************
>
>>> This distinction becomes relevant if BODY modifies VAR or captures it
>>> in a closure.
>> Maybe I am being thick, but I cannot think of how there would be a
>> different behaviour.
>
> Try:
>
>     (let (funs)
>       (dotimes (i 5)
>         (push (lambda () i) funs))
>       (mapcar #'funcall funs))
>
> IIUC the version you advocate would return
>
>     (5 5 5 5 5)
>
> whereas the current version returns
>
>     (4 3 2 1 0)

Thanks, I was being thick.

----

I realize that elisp does not claim to be a dialect of common lisp, but
it would be helpful to note that this behaviour deviates from the
behaviour of several CL implementations (gcl, clisp, sbcl, ecl).

The CL standard (http://clhs.lisp.se/Body/m_dotime.htm) says:

 ....
 It is implementation-dependent whether dotimes establishes a new
 binding of var on each iteration or whether it establishes a binding
 for var once at the beginning and then assigns it on any subsequent
 iterations. 



>
> Other differences occur if you do
>
>     (let (vals)
>       (dotimes (i 10)
>         (push (setq i (1+ i)) vals))
>       vals)

Yes.

Leo

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> I realize that elisp does not claim to be a dialect of common lisp, but
> it would be helpful to note that this behaviour deviates from the
> behaviour of several CL implementations (gcl, clisp, sbcl, ecl).

Yes, I know ;-)
I opted for the semantics which I consider cleaner.


        Stefan

RE: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Drew Adams]

> It's hidden because it's not at the usual place where the return value
> is usually placed (i.e. in the last expression).
> 
> There are several other macros/specialforms that also place the return
> value elsewhere than the last expression, such as `unwind-protect`, (and
> to a lesser extent `if`, `cond`, `condition-case`, `prog1`), but these
> still place the return value in one of the immediate subexpressions,
> many of them use indentation to try and make the return value "stand
> out" from the rest, etc...
> 
> In contrast in `dotimes` and `dolist` it's placed in
> a sub-sub-expression and moreover it's one that's rarely used (arguably
> *because* it's too deeply nested), so you're likely to miss it if you
> don't go looking for it very actively.
> 
> Not that it matters anyway,

FWIW, Common Lisp has several iteration constructs that let you specify an optional return value up front (as you would say, in a sub-sub-expression).  Not just `dolist' and `dotimes'.  `do', for instance.

There's nothing particularly odd, new, or unlispy about such design.  It's very old in Lisp iteration.

And most such iteration constructs wrap everything in an implicit nil block, so you can also use `return' anywhere to return a value.

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node88.html

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> FWIW, Common Lisp has several iteration constructs that let you specify an
> optional return value up front (as you would say, in a sub-sub-expression).
> Not just `dolist' and `dotimes'.  `do', for instance.

There are a few other such cases, indeed.  I think most of them are like
`do` in the sense that they're *very* rarely used in ELisp (and
correspondingly most ELisp programmers have no idea what it does).

FWIW, I remember that I used to like `do*` back when I was programming
in Common Lisp, but nowadays I find it rather inscrutable.

In that category I think Scheme's named let is infinitely superior: both
more general and easier to understand.  Too bad that it's kind of a pain
to implement efficiently in ELisp, but Vincent's `recur-let` gives
a pretty good approximation (still more general and easier to understand
than `do*`).
(c.f. https://github.com/VincentToups/recur).

> There's nothing particularly odd, new, or unlispy about such design.
> It's very old in Lisp iteration.

Old doesn't mean good.


        Stefan

RE: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Drew Adams]

> FWIW, I remember that I used to like `do*` back when I was programming
> in Common Lisp, but nowadays I find it rather inscrutable.

I agree with your last phrase, and I was never a fan of `do*'.

> In that category I think Scheme's named let is infinitely superior: both
> more general and easier to understand.  Too bad that it's kind of a pain
> to implement efficiently in ELisp, but Vincent's `recur-let` gives
> a pretty good approximation (still more general and easier to understand
> than `do*`).
> (c.f.
> https://urldefense.com/v3/__https://github.com/VincentToups/recur__;!!GqivPVa
> 7Brio!LHaFHC2fIL6O-KnVleahtiHtmmkLRwamJMjoe9mglmjt2ncWgr0mSNs9uMVnhdAd$ ).

Sure.  It can be useful to move recursion to iteration
under the covers, or even above the covers through the
idiom of using an accumulator.

Elisp has a long way to go, in terms of doing such
things under the covers.  Put differently, doing such
things is not something new in software engineering.

I guess Elisp hasn't had the necessary itch-scratchers
or the felt need.  For much (most?) user use of Elisp,
performance isn't particularly important.  The story
is different for other Lisps, like Common Lisp, which
have more general use.

> > There's nothing particularly odd, new, or unlispy
> > about such design. It's very old in Lisp iteration.
> 
> Old doesn't mean good.

Right. Nor does new. Or young.

There's still room for Elisp to learn from Common Lisp
(and, as you point out, from elsewhere).

How to get plist properties list?

[Jean Louis]

As I am replacing here `dotimes':

(setq plist '(age "47" name "Doe"))

(defun divisible-by-2-or-0-p (n)
  "Returns t if number is divisible by 2"
  (if (or (= n 0) (= (/ n 2.0) (truncate (/ n 2.0)))) t nil))

(defun plist-properties (plist)
  "Returns plist properties"
  (let* ((length (length plist))
  (properties '()))
    (dotimes (i length (reverse properties))
      (if (divisible-by-2-or-0-p i)
   (push (elt plist i) properties)))))

instead of the above complications to extreme length, I have replaced
it with this one below:

(defun plist-properties (plist)
  (let ((n 0)
 (properties '()))
    (while (/= n (length plist))
      (push (elt plist n) properties)
      (setq n (+ 2 n)))
    properties))

But maybe there is some more simpler way to get plist properties list?

Jean

Re: How to get plist properties list?

[Stefan Monnier]

>     (dotimes (i length (reverse properties))
>       (if (divisible-by-2-or-0-p i)
>           (push (elt plist i) properties)))))

This is another example of a dotimes+elt loop, i.e. a loop where you
end up with an unwarranted O(n²) complexity (i.e. a performance bug).

>     (while (/= n (length plist))
>       (push (elt plist n) properties)
>       (setq n (+ 2 n)))
>     properties))

Same problem here.

> But maybe there is some more simpler way to get plist properties list?

Probably not the simplest but this should work:

    (defun plist-keys (plist)
      (let (keys iskey)
        (dolist (x plist)
          (if (setq iskey (not iskey)) (push x keys)))
        (nreverse keys)))

The simplest might be:

    (map-keys plist)


-- Stefan

Re: How to get plist properties list?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 08:23]:
> >     (dotimes (i length (reverse properties))
> >       (if (divisible-by-2-or-0-p i)
> >           (push (elt plist i) properties)))))
> 
> This is another example of a dotimes+elt loop, i.e. a loop where you
> end up with an unwarranted O(n²) complexity (i.e. a performance bug).

Yes, I know there are complexities, but it did not bother me really.
Only compiler warnings bother me.

> >     (while (/= n (length plist))
> >       (push (elt plist n) properties)
> >       (setq n (+ 2 n)))
> >     properties))
> 
> Same problem here.

I would like to understand what is the problem. I don't. You tell me
that `elt' is problem, that is how I understand it.

Could I maybe rather use `nth' to replace `elt'?

(elt '(1 2 3) 0)
(nth 0 '(1 2 3))

Which one is faster?

> > But maybe there is some more simpler way to get plist properties list?
> 
> Probably not the simplest but this should work:
> 
>     (defun plist-keys (plist)
>       (let (keys iskey)
>         (dolist (x plist)
>           (if (setq iskey (not iskey)) (push x keys)))
>         (nreverse keys)))

As we already discussed it, `dolist' is not perfect, it would give
warnings, not logical at all. In the above example `x' would not give
warning, but if there would be return value then `x'  would be part of
warning, while return variable would not be. That is not logical to
me, so I consider `dolist' not well handled by compiler and is better
for me not to use it.

> The simplest might be:
> 
>     (map-keys plist)

That is it. Why not start with that one...


Jean

Re: How to get plist properties list?

[Stefan Monnier]

> I would like to understand what is the problem. I don't. You tell me
> that `elt' is problem, that is how I understand it.
>
> Could I maybe rather use `nth' to replace `elt'?

No, same problem.
Think of it this way: consider your list of N elements as a road that's
N kilometers long.  `elt` and `nth` are operations which involve walking
from the beginning of the road to the Nth kilometer.  So to collect all
the keys found at kilometers 0, 2, 4, ... your loop does:
- walk 0 kilometers and collect the key (then come back).
- walk 2 kilometers from the start and collect the key (then come back).
- walk 4 kilometers from the start and collect the key (then come back).
- walk 6 kilometers from the start and collect the key (then come back).
...
[ the "come back" happens to be feasible as a kind of teleportation:
  obviously my analogy breaks down there.  ]
So to collect all the keys on a road of 100km you'll end up walking
a total of about ... 5000km.  I hope you can see that it's far from
being the most efficient use of your feet.

>> > But maybe there is some more simpler way to get plist properties list?
>> 
>> Probably not the simplest but this should work:
>> 
>>     (defun plist-keys (plist)
>>       (let (keys iskey)
>>         (dolist (x plist)
>>           (if (setq iskey (not iskey)) (push x keys)))
>>         (nreverse keys)))
>
> As we already discussed it, `dolist' is not perfect, it would give
> warnings, not logical at all.

There's no warning for this code, AFAIK, no.

> In the above example `x' would not give warning, but if there would be
> return value then `x'  would be part of warning, while return variable
> would not be. That is not logical to me, so I consider `dolist' not
> well handled by compiler and is better for me not to use it.

Just never use that 3rd argument and you'll never get that warning.


        Stefan

Re: How to get plist properties list?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 09:51]:
> > I would like to understand what is the problem. I don't. You tell me
> > that `elt' is problem, that is how I understand it.
> >
> > Could I maybe rather use `nth' to replace `elt'?
> 
> No, same problem.
> Think of it this way: consider your list of N elements as a road that's
> N kilometers long.  `elt` and `nth` are operations which involve walking
> from the beginning of the road to the Nth kilometer.  So to collect all
> the keys found at kilometers 0, 2, 4, ... your loop does:
> - walk 0 kilometers and collect the key (then come back).
> - walk 2 kilometers from the start and collect the key (then come back).
> - walk 4 kilometers from the start and collect the key (then come back).
> - walk 6 kilometers from the start and collect the key (then come
> back).

So far I have seen from similar discussion on `length' on emacs-devel
mailing list, people were discussing of its efficiencies and maybe now
you also discuss of efficiency. For me as programmer of systems that I
need I use `elt' about 200+ times in all programs. `nth' I use less
times, maybe 30 times. I do not know what is a big difference between
those two.

If I wish to get the element like number 17th I do not know what I
should do. And I did not read all emails. I also do not feel why it
should be wrong, finally it is not really kilometers but computer
doing its iteration over elements. The above statement is unclear to
me as your distant student. I do understand you speak of efficiency,
but practically I yet do not see the problem. For example I cannot
practically at this moment find some delays that would bother me. 

> >> > But maybe there is some more simpler way to get plist properties list?
> >> 
> >> Probably not the simplest but this should work:
> >> 
> >>     (defun plist-keys (plist)
> >>       (let (keys iskey)
> >>         (dolist (x plist)
> >>           (if (setq iskey (not iskey)) (push x keys)))
> >>         (nreverse keys)))
> >
> > As we already discussed it, `dolist' is not perfect, it would give
> > warnings, not logical at all.
> 
> There's no warning for this code, AFAIK, no.
> 
> > In the above example `x' would not give warning, but if there would be
> > return value then `x'  would be part of warning, while return variable
> > would not be. That is not logical to me, so I consider `dolist' not
> > well handled by compiler and is better for me not to use it.
> 
> Just never use that 3rd argument and you'll never get that warning.

I do understand your point, but I also remember that warning was not
about the third argument and that when macro is defined as such then
programmers shall use it.

In my opinion compiler should not warn when programmer uses function
by its description. But when you make something obsolete, then I would
say it would be fine. `dolist' seem like Common Lisp so I was
expecting it to work as CL.

I will see more emails after this one.

Jean

Re: How to get plist properties list?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

On Sat, Jan 09, 2021 at 10:54:17AM +0300, Jean Louis wrote:
> * Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 09:51]:
> > > I would like to understand what is the problem. I don't. You tell me
> > > that `elt' is problem, that is how I understand it.
> > >
> > > Could I maybe rather use `nth' to replace `elt'?
> > 
> > No, same problem.
> > Think of it this way: consider your list of N elements as a road that's
> > N kilometers long [...]

(nice explanation, BTW :)

> So far I have seen from similar discussion on `length' on emacs-devel
> mailing list [...]

When you have a toolbox (a language & library), you usually develop
a set of "ways of doing things", corresponding to the properties
your tools have.

> If I wish to get the element like number 17th I do not know what I
> should do.

  (nth 17 my-list)

Unless... you are doing it very often. Then you do something
different (unless, again, you don't care that your program is
slow; if you are giving your program to other people, you should
care, somewhat, at least. And so on.

Lists are very flexible data structures. But they have one downside:
accessing a random element in them takes as long as walking through
half of it, in the average.

If you plan to access elements by index, Emacs Lisp has arrays.
They aren't as flexible, but faster for random access. If you
want more flexibility, there are hash tables. And so on.

It's like with tools. You use the screwdriver to drive screws.
In a pinch, you can use it to poke dirt out of a hole. I've
seen people using it to hit a hole in a wall (hm.) or even to
drive a nail into a piece of wood (ouch!).

Cheers :)
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: How to get plist properties list?

[Jean Louis]

* tomas@tuxteam.de <tomas@tuxteam.de> [2021-01-09 12:28]:
> On Sat, Jan 09, 2021 at 10:54:17AM +0300, Jean Louis wrote:
> > * Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-08 09:51]:
> > > > I would like to understand what is the problem. I don't. You tell me
> > > > that `elt' is problem, that is how I understand it.
> > > >
> > > > Could I maybe rather use `nth' to replace `elt'?
> > > 
> > > No, same problem.
> > > Think of it this way: consider your list of N elements as a road that's
> > > N kilometers long [...]
> 
> (nice explanation, BTW :)
> 
> > So far I have seen from similar discussion on `length' on emacs-devel
> > mailing list [...]
> 
> When you have a toolbox (a language & library), you usually develop
> a set of "ways of doing things", corresponding to the properties
> your tools have.
> 
> > If I wish to get the element like number 17th I do not know what I
> > should do.
> 
>   (nth 17 my-list)
> 
> Unless... you are doing it very often. Then you do something
> different (unless, again, you don't care that your program is
> slow; if you are giving your program to other people, you should
> care, somewhat, at least. And so on.
> 
> Lists are very flexible data structures. But they have one downside:
> accessing a random element in them takes as long as walking through
> half of it, in the average.
> 
> If you plan to access elements by index, Emacs Lisp has arrays.
> They aren't as flexible, but faster for random access. If you
> want more flexibility, there are hash tables. And so on.
> 
> It's like with tools. You use the screwdriver to drive screws.
> In a pinch, you can use it to poke dirt out of a hole. I've
> seen people using it to hit a hole in a wall (hm.) or even to
> drive a nail into a piece of wood (ouch!).

The comparison of different types of data structure does give me
better explanation.

I do use hashes. Recently I have switched from using list collections
in completing read to using hashes. But I have not measured
speed. Sometimes I have 20000 elements to choose from.

This function provides sql in form COLUMN_ID, COLUMN_TEXT (where by
COLUMN_TEXT can be concatenated from various multiple columns) and
then the SQL result is converted to hash and used with completing
read.

(defun rcd-completing-read-sql-hash (prompt sql &optional history)
  (let* ((hash (rcd-sql-hash sql *cf*))
	 (values (hash-table-values hash))
	 (length (length values)))
    (if (= length 1)
	  (car values)
      (let* ((completion-ignore-case t)
	     (choice (completing-read prompt hash nil t nil history))
	     (choice (string-trim choice))
	     (id (gethash choice hash)))
	id))))

I should maybe measure it against the other one that was using lists
to see the difference.

Jean

Re: How to get plist properties list?

[Stefan Monnier]

> If I wish to get the element like number 17th I do not know what I

If you need that, it's "too late" to do something else.

The question is "if I wish to do something over all (or most of) the
elements of a list".  And for that the answer is "don't use dotimes +
nth" but use `dolist` or `mapcar` or `while + cdr`.

Needing the N'th element of a list is actually rare (just like it's
rare to need to go to kilometer N of a road compared to the frequency
of having to walk along that road doing this along its way).


        Stefan

Re: How to get plist properties list?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-09 20:03]:
> > If I wish to get the element like number 17th I do not know what I
> 
> If you need that, it's "too late" to do something else.

I see. Example is here:

(defun mutt-identity-signature (identity)
  (let* ((sql (format "SELECT * FROM identities WHERE identities_id = %s" identity))
	 (list (first (rcd-sql-list sql *cf*)))
	 (identity-id (elt list 0))
	 (identity-name (elt list 1))
	 (identity-first-name (elt list 2))
	 (identity-last-name (elt list 3))
...snip...

As if I do not do like that above, then to construct hash from SQL, I
would need to make little more complex SQL query like:

SELECT 'identities_id', identities_id,
 'identities_name', identities_name,
 'identities_firstname', identities_firstname,
 'identities_lastname', identities_lastname
 AND SO ON.

I could automate it, and now I see I have a function to automate it.

(defun rcd-db-table-id-hash-values (table id pg)
  "Returns hash for TABLE by its table ID with its expanded foreign key values."
  (let* ((oid (rcd/table-oid table pg)) 
	 (attributes (rcd-table-full-attributes oid pg))
	 (foreign-keys (rcd/table-foreign-keys table pg))
	 (hash (make-hash-table)))
      (dolist (col attributes hash)
	(let* ((column (aref col 0))
	       (foreign-key (rcd/column-foreign-key column foreign-keys))
	       (bvalue (rcd-db-get-entry table column id pg))
	       (value (if (and foreign-key bvalue) (rcd/combo-query (car foreign-key) bvalue pg) bvalue)))
	  (puthash (intern column) value hash)))))

then instead of the initial SQL query that requires using `elt' to
fetch values, I get bunch of various functions that need to construct
a hash.

Then I can use instead of:

(format "SELECT * FROM identities WHERE identities_id = %s" identity)

with `elt' complexity, the new complexity:

(rcd-db-table-id-hash-values "identities" identity *cf*)

but underlying functions are many. I get the hash but I doubt I get
the speed up. In this example it is selection of the identity that
selects signature for the email being composed. I cannot visually see
any difference.

You were very right that it is probably too late, so I can put
attention in future. But not that I need to change much of
functionality for sake of speed when no delay is sensible.

> The question is "if I wish to do something over all (or most of) the
> elements of a list".  And for that the answer is "don't use dotimes +
> nth" but use `dolist` or `mapcar` or `while + cdr`.

That is my conclusion now as well.

> Needing the N'th element of a list is actually rare (just like it's
> rare to need to go to kilometer N of a road compared to the frequency
> of having to walk along that road doing this along its way).

I do not know if it is rare as I cannot count and see easily what
other people are doing

I know that I often use it like this, 

(defun mapping/map-locations-point-kml-placemark (p)
  (mapping/kml-placemark (elt p 3) (elt p 4) (list (elt p 0) (elt p 1) (elt p 2))))

(defun mapping/map-locations-point-marble (p)
  (mapping/kml-marble-bookmark (elt p 3) (elt p 4) (list (elt p 0) (elt p 1) (elt p 2))))

using it in different manner would complicate the code more I think.

If I do some grep in ~/.emacs.d/elpa/* I see many packages using it as
well.

In general I understood that one could think of speed in certain
situations, but in other situations where speed is not important using
lists is fine.

Jean

Re: How to get plist properties list?

[Stefan Monnier]

> (defun mutt-identity-signature (identity)
>   (let* ((sql (format "SELECT * FROM identities WHERE identities_id = %s" identity))
> 	 (list (first (rcd-sql-list sql *cf*)))
> 	 (identity-id (elt list 0))
> 	 (identity-name (elt list 1))
> 	 (identity-first-name (elt list 2))
> 	 (identity-last-name (elt list 3))

That's a completely different situation: the above code does not operate
on a list of elements of arbitrary length, but on what is fundamentally
a tuple of fixed length which just happens to be represented as a Lisp
linked list.

> (defun mapping/map-locations-point-kml-placemark (p)
>   (mapping/kml-placemark (elt p 3) (elt p 4) (list (elt p 0) (elt p 1) (elt p 2))))

Notice that the "n" passed to `elt` is a constant, whereas in the
original code where I pointed out the problem:

    (dotimes (i length (reverse properties))
      (if (divisible-by-2-or-0-p i)
          (push (elt plist i) properties)))))

the argument to `elt` (here `i`) is not a constant.
That changes the nature of the problem completely.

    (elt FOO 100)

may be arguably slow, but it's still constant time, so it won't get
slower when applied to a larger data-structure.  In contrast

    (elt FOO i)

will probably be fairly fast on small data structures, where `i` is
small, but it will get slower and slower as `i` grows with the size of
the data structure.


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Philipp Stephani <p.stephani2@gmail.com> [2021-01-07 14:34]:
> > This way is upside down, we tell programmers to not use function that
> > was defined as usable.
> 
> The discussion in that bug implies that it won't get fixed. Probably
> we should discourage the three-argument form more strongly, e.g. by
> removing it from the documentation and changing the advertised calling
> convention.

But that is contrary to what `dotimes' and `dolist' were always doing.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Stefan Monnier]

> What is proper way to avoid this warning in `dolist' or `dotimes':
> rcd-db.el:841:1: Warning: Unused lexical variable ‘i’

Same as for any other place where you might declare a variable that you
don't actually use: make sure the var's name starts with an underscore.
This tells the compiler that you're aware of the fact the var is not
used (but still want this variable there for some reason; e.g. in
dotimes and dolist you don't have the choice not to put a variable
there).

Traditionally for `dotimes`, I'd then use `_` as the variable name
unless I really want to give a hint to the reader about what kind of
things I'm counting like (dotimes (_column-nb ...) ...).


        Stefan

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2021-01-07 18:37]:
> > What is proper way to avoid this warning in `dolist' or `dotimes':
> > rcd-db.el:841:1: Warning: Unused lexical variable ‘i’
> 
> Same as for any other place where you might declare a variable that you
> don't actually use: make sure the var's name starts with an underscore.
> This tells the compiler that you're aware of the fact the var is not
> used (but still want this variable there for some reason; e.g. in
> dotimes and dolist you don't have the choice not to put a variable
> there).
> 
> Traditionally for `dotimes`, I'd then use `_` as the variable name
> unless I really want to give a hint to the reader about what kind of
> things I'm counting like (dotimes (_column-nb ...) ...).

That also sounds like workaround around the real problem. In `dotimes'
I do use the VAR counted. It is not same as (let ((_ (execute some
program without))))

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

>> Same as for any other place where you might declare
>> a variable that you don't actually use: make sure the var's
>> name starts with an underscore. This tells the compiler
>> that you're aware of the fact the var is not used (but
>> still want this variable there for some reason; e.g.
>> in dotimes and dolist you don't have the choice not to put
>> a variable there).
>> 
>> Traditionally for `dotimes`, I'd then use `_` as the
>> variable name unless I really want to give a hint to the
>> reader about what kind of things I'm counting like (dotimes
>> (_column-nb ...) ...).
>
> That also sounds like workaround around the real problem.
> In `dotimes' I do use the VAR counted. It is not same as
> (let ((_ (execute some program without))))

But is this really a workaround in your case? Because you _do_
use it!

  (dolist (_ignored-var '(a b c))
    (message "%s" _ignored-var) )

  geh.el:257:1: Warning: variable ‘_ignored-var’ not left unused

See? Sometimes the byte-compiler gets it right!

:)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Jean Louis]

* Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> [2021-01-08 04:21]:
> Jean Louis wrote:
> 
> >> Same as for any other place where you might declare
> >> a variable that you don't actually use: make sure the var's
> >> name starts with an underscore. This tells the compiler
> >> that you're aware of the fact the var is not used (but
> >> still want this variable there for some reason; e.g.
> >> in dotimes and dolist you don't have the choice not to put
> >> a variable there).
> >> 
> >> Traditionally for `dotimes`, I'd then use `_` as the
> >> variable name unless I really want to give a hint to the
> >> reader about what kind of things I'm counting like (dotimes
> >> (_column-nb ...) ...).
> >
> > That also sounds like workaround around the real problem.
> > In `dotimes' I do use the VAR counted. It is not same as
> > (let ((_ (execute some program without))))
> 
> But is this really a workaround in your case? Because you _do_
> use it!
> 
>   (dolist (_ignored-var '(a b c))
>     (message "%s" _ignored-var) )
> 
>   geh.el:257:1: Warning: variable ‘_ignored-var’ not left unused
> 
> See? Sometimes the byte-compiler gets it right!

Then it is not a real solution as proposed earlier. If used, byte compiler will warn. Is it?

I did not test it, it seem like you did.

Re: How to avoid compiler warning `unused lexical variable' for `dolist' or `dotimes'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean Louis wrote:

> Then it is not a real solution as proposed earlier. If used,
> byte compiler will warn. Is it?

In general the solution is to modify the byte-compiler.

But in this case one can argue that cl-loop is a better
choice anyway, unrelated to the byte compiler.

So a much easier solution is just to use that and forget all
about this warning...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal