Strange eval behaviour

Strange eval behaviour

[Stefan Huchler]

I had this strange bug in my code that I could not debug, everytime I
tried to print out some stuff to see whats happenig it startet to
working.

Now I have pinned it down to the point that whenever I restart emacs, it
does not work, till I evaluate manualy the function in that elisp file.

http://ix.io/1EEU

That should be the relevant points, the function I have to manualy
re-evaluate is the kodi-remote-get function.

in my init file I load that file/module with:

(require 'kodi-remote)
(setq kodi-host-name "myserver:9099")

other functions that use my post method all in this file, work, like
kodi-remote-musik that starts party mode music playback.

Its funny I can use the kodi-remote-play-pause function a hundret times
and it I think returns nil and does nothing, then I go with the help
function to that file eval the kodi-remote-get function and from then on
it works.

Do I miss maybe something very obvious?

Re: Strange eval behaviour

[Stefan Monnier]

> Now I have pinned it down to the point that whenever I restart emacs, it
> does not work, till I evaluate manualy the function in that elisp file.
> http://ix.io/1EEU

Please include such code directly in your messages.

Do the following:

    emacs -Q .../kodi-remote.el
    M-x byte-compile-file RET

then look at the errors/warnings.  Fix them (typically by adding the
missing `require`s) and try again until there's no more warnings/errors.


        Stefan

Re: Strange eval behaviour

[Stefan Huchler]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Now I have pinned it down to the point that whenever I restart emacs, it
>> does not work, till I evaluate manualy the function in that elisp file.
>> http://ix.io/1EEU
>
> Please include such code directly in your messages.
>
> Do the following:
>
>     emacs -Q .../kodi-remote.el
>     M-x byte-compile-file RET
>
> then look at the errors/warnings.  Fix them (typically by adding the
> missing `require`s) and try again until there's no more warnings/errors.
>
>
>         Stefan

Thanks,

that kind of pushed me into the right direction. So I test this module
by just requesting it. aperently that dont creates a .elc file.

So I use the function macro, that has somethnig to do with
bytecompiling, so doest it automaticly bytecompile that if its needed,
or did it ignore that I dont have let-alist loaded, because it is in the
function macro?

Normaly it should through a error if it cant find a macro/function that
is not availible right?

So I fixed it but would like to 100% understand what the problem was, to
avoid to make the same mistake again.

(defun kodi-remote-get (method params)
  "method to send get requests to the kodi instance"
  (let* ((request-data
	  `(("id" . 0)
	   ("jsonrpc" . "2.0")
	   ("method" . ,method))))
    (if (equal params nil) ()
    	(setq request-data
	      (append request-data params
		      )))
    ;; (print request-data)
    (request
     (kodi-json-url)
     :data (json-encode request-data)
     :headers '(("Content-Type" . "application/json"))
     :success (function* (lambda (&key data &allow-other-keys)
    		  (when data
    		    (setq kodi-properties (let-alist (json-read-from-string data)
					    .result))
    		    ;; (print (aref (let-alist kodi-properties .episodedetails) 0))
    		    ;; (print data)
    		    )))
     :error (function* (lambda (&key error-thrown &allow-other-keys&rest _)
     		  (message "Got error: %S" error-thrown)))
     :complete (lambda (&rest _) (message "Finished!"))
     :parser 'buffer-string)))

that was the code I did not have
(require 'let-alist)

in the file?

I importet it with:

  (add-to-list 'load-path "~/.emacs.d/config/")
  (require 'kodi-remote)


Again thanks so far so I can release that code soon, but again I would
like to understand what the problem was. Do I have to have a
byte-compiled .elc file if I use the "function" macro

oh wait its the function* which is a alias to the cl-function macro in
cl-macs

so I need the cl library imported too. But well thats more relevant for
packaging I think, I think the main problem was that there just was no
.elc file?

Strange :) cant it just interpret the sourcefile when I require it on
emacs-start?

Re: Strange eval behaviour

[Michael Heerdegen]

Stefan Huchler <stefan.huchler@mail.de> writes:

> Normaly it should through a error if it cant find a macro/function
> that is not availible right?

In case of a macro, it depends on whether the macro is defined when
compiling.  So you don't get a warning necessarily.  To avoid missing
dependencies, it's helpful to compile from a separate emacs instance
(emacs -Q).

> that was the code I did not have
> (require 'let-alist)

That was one problem, at least.

> in the file?

> Again thanks so far so I can release that code soon, but again I would
> like to understand what the problem was.

AFAIR, you had a strange handling of variables in your code: some
functions had used free variables that were declared nowhere.  Compiling
is a good way to reveal such problems.

> Do I have to have a byte-compiled .elc file if I use the "function"
> macro

No.

> oh wait its the function* which is a alias to the cl-function macro in
> cl-macs
>
> so I need the cl library imported too.

Yes.  It is now more or less consent now that the cl library should not
be loaded at run-time.  Please use cl-lib and the according cl- prefixed
functions instead.

> But well thats more relevant for packaging I think, I think the main
> problem was that there just was no .elc file?

No, Stefan suggested to compile so that you get warnings that help you
find the problems in your code.  Compiling is not required to run code.
Compiled code is just running faster, and the compiler produces
excellent warnings.


Michael.

Re: Strange eval behaviour

[Stefan Huchler]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Stefan Huchler <stefan.huchler@mail.de> writes:
>
> In case of a macro, it depends on whether the macro is defined when
> compiling.  So you don't get a warning necessarily.  To avoid missing
> dependencies, it's helpful to compile from a separate emacs instance
> (emacs -Q).

yes thats usefull thanks for that tipp again! But to really test it I
think I have to set the server address absolute else I need a
configuration to really test the thing.

Also I am a bit relactend to open to much emacs instances cause I work
in exwm which is basicly emacs as window manager, so I start emacs out
of emacs :) Which itself could theoreticly cause problems. :)


> AFAIR, you had a strange handling of variables in your code: some
> functions had used free variables that were declared nowhere.  Compiling
> is a good way to reveal such problems.

Yes I am a bit a elisp noob but it gets better, so I used to only use
setq and not let(*) at all, so its a bad habbit also in many other
languages you dont have or use so much let constructs.

But te point is, as long as I dont come in conflict with some system
variables it should still work, its no good praktise but it should not
cause bugs. Or do you see how that in that case can create this
behaviour.

I will clean that up if all other bugs are fixed, I even will break that
into 2 commits cause my kodi-remote-keyboard is more solid then the
other mode I have where I can get file lists of series/... .

Its a pretty big patch much bigger than the last version on github was :)




> No.

Well then I did not find the real causation of the problem apperently
cause that was the thing that seemed to make here the difference no .elc
restart does not work, with .elc restart and it works.


> Yes.  It is now more or less consent now that the cl library should not
> be loaded at run-time.  Please use cl-lib and the according cl- prefixed
> functions instead.

so like that:
(require 'cl-lib)
(require 'cl-macs)

?


> No, Stefan suggested to compile so that you get warnings that help you
> find the problems in your code.  Compiling is not required to run code.
> Compiled code is just running faster, and the compiler produces
> excellent warnings.

yes that was my idea about bytecompiling in general not elisp specific,
but if macros dont throw errors cause stuff only get evaluated if its
bytecompiled thats really hard to debug, or it seems at least to me :)

Re: Strange eval behaviour

[Michael Heerdegen]

Stefan Huchler <stefan.huchler@mail.de> writes:

> Also I am a bit relactend to open to much emacs instances cause I work
> in exwm which is basicly emacs as window manager, so I start emacs out
> of emacs :) Which itself could theoreticly cause problems. :)

You don't have to do it interactively.  You can also call Emacs from the
console with parameters.  Or use "make" if you want.

> But te point is, as long as I dont come in conflict with some system
> variables it should still work, its no good praktise but it should not
> cause bugs. Or do you see how that in that case can create this
> behaviour.

It makes the code and how it works very non-transparent (that's a reason
why I didn't try to understand the logic behind it).  I would definitely
fix this and all other compiler warnings as well first.

> so like that:
> (require 'cl-lib)
> (require 'cl-macs)
> ?

Yes.  But the second one is redundant: cl-libs requires 'cl-macs by
itself.

> yes that was my idea about bytecompiling in general not elisp
> specific, but if macros dont throw errors cause stuff only get
> evaluated if its bytecompiled thats really hard to debug, or it seems
> at least to me :)

I'm not sure if I understand what you mean here...


Regards,

Michael.

Re: Strange eval behaviour

[Stefan Huchler]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>> yes that was my idea about bytecompiling in general not elisp
>> specific, but if macros dont throw errors cause stuff only get
>> evaluated if its bytecompiled thats really hard to debug, or it seems
>> at least to me :)
>
> I'm not sure if I understand what you mean here...
>

If code only throughs errors when you bytecompile it but not when you
evaluate it, or run it, its hard to find the error. Aperently that
happens with such macros like function*?

Re: Strange eval behaviour

[Michael Heerdegen]

Stefan Huchler <stefan.huchler@mail.de> writes:

> If code only throughs errors when you bytecompile it but not when you
> evaluate it, or run it, its hard to find the error. Aperently that
> happens with such macros like function*?

What do you mean specifically?

Of cause can running code produce errors.  The compiler does some
analysis of the code and helps you to discover potential problems, like
typos, or function calls with an invalid number of arguments.  This
makes it much easier to detect such problems, contrary to running that
code and then debugging it.

With respect to the environment (which functions and macros are defined,
which libraries are loaded, etc), of cause the Lisp evaluator doesn't
check if evaluating the same code in a different Emacs session could
potentially error.  Apart from the fact that it would slow down running
code, it would not make any sense because it is impossible to do in
general.  The compiler OTOH has the means (and the time) to do some
checks.  Since compiling your code will be part of your working style,
that is hardly a problem.  On the contrary, you'll see that it's a big
help that the compiler catches most common errors without the need to
run your code.


Regards,

Michael.

Re: Strange eval behaviour

[Stefan Huchler]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Stefan Huchler <stefan.huchler@mail.de> writes:
>
>> If code only throughs errors when you bytecompile it but not when you
>> evaluate it, or run it, its hard to find the error. Aperently that
>> happens with such macros like function*?
>
> What do you mean specifically?
>
> Of cause can running code produce errors.

I have no problem if running code produces a error (a error message to
be more prezise) but if it just silently fails, and just the re-evaluate
fixes it, its kind of hard to find the problem.

Re: Strange eval behaviour

[Stefan Huchler]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> AFAIR, you had a strange handling of variables in your code: some
> functions had used free variables that were declared nowhere.  Compiling
> is a good way to reveal such problems.

Hmm I dont misunderstand you, you are talking about the params varible I
assigned a few times with setq?

I saw that only as warnings again that should not cause problems.

Or did you refer here to something different you saw? bytecompiling only
throw warnings about this params variables. And thats only warnings...

Re: Strange eval behaviour

[Michael Heerdegen]

Stefan Huchler <stefan.huchler@mail.de> writes:

> Or did you refer here to something different you saw? bytecompiling
> only throw warnings about this params variables. And thats only
> warnings...

But the warnings are there for a reason, and the reason is that the code
can potentially error when being run.  In general, the warnings mean
that you do something you definitely should not do.  Let's fix these
warnings, and then I can have a look at the code again if you want.

Also try to follow the essence of functional programming: reduce side
effects of functions to a minimum.  When a function has side effects,
make it clear (e.g. in the doc or the name of the function).  If you
really need to set any global variable, declare that variable with
`defvar' and prefix its name with the prefix of your library.  In
general, the way functions transfer information in Lisp is by the return
value, not by setting variables.  For avoiding some bad habits it's very
helpful to use lexical-binding - you can enable it with a specification
in the file's first line.

Maybe consider reading a good book about lisp (since learning by doing
is not a good approach in Lisp).  Maybe "Land of Lisp" (haven't read it,
but it has a good title song) or "On Lisp" by Graham (a classical, but
harder to read; reading some of the first chapters might be a good idea
anyway).


Regards,

Michael.

Re: Strange eval behaviour

[Stefan Huchler]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Stefan Huchler <stefan.huchler@mail.de> writes:
>
>> Or did you refer here to something different you saw? bytecompiling
>> only throw warnings about this params variables. And thats only
>> warnings...
>
> But the warnings are there for a reason, and the reason is that the code
> can potentially error when being run.  In general, the warnings mean
> that you do something you definitely should not do.  Let's fix these
> warnings, and then I can have a look at the code again if you want.

Yes but when I know (or belive strongly) that I have nothing that causes
problems I fix warnings as last thing, not as first (at least normaly)
and I am pretty shure that this setq does not create problems, only if
somethnig else also uses global variables with the name params :). So
basicly if I AND somebody else messed up, then it creates problems.

And yes I work best practises in my lisp workflow coding style in one by one.

> Also try to follow the essence of functional programming: reduce side
> effects of functions to a minimum.  When a function has side effects,
> make it clear (e.g. in the doc or the name of the function).

I dont write a doc for a code I will replace before I release it anyway :)

> If you
> really need to set any global variable, declare that variable with
> `defvar' and prefix its name with the prefix of your library.  In
> general, the way functions transfer information in Lisp is by the return
> value, not by setting variables.  For avoiding some bad habits it's very
> helpful to use lexical-binding - you can enable it with a specification
> in the file's first line.

Yeah I know that this is no good idea especialy if you upload it, cause
the main problem is not if you do it in one local package but if
everybody would do that, you have many times incompatible packages
basicly.

I know about that, I just used it because I concentrated primary on the
json code and to get something working, and at the time it was just
easier and faster to do it that way.

Also I dont want to refactor stuff while stuff is not working, I wanted
to get to a working state first. And I was not aware of the
bytecompiling and that macros dont get "live" evaluated basicly I am
used to code that when its wrong gives me some sort of error.

> Maybe consider reading a good book about lisp (since learning by doing
> is not a good approach in Lisp).  Maybe "Land of Lisp" (haven't read it,
> but it has a good title song) or "On Lisp" by Graham (a classical, but
> harder to read; reading some of the first chapters might be a good idea
> anyway).

I am not so good at reading books, I need a reason to do things other
than learn somethnig abstract :) I do that for fun, startet with a
send-youtube-to-kody thing and got then bigger and bigger.

Of course if I run more often into such things, I consider reading a few
pages of a book. I just learned no language by book, not Java, not c++,
not python, not php...

But I think you CAN learn everything online or with the offline
documentiation, books tend just to focus a bit narrower on the juice
stuff, but therefor you have to go through some stupid example code I
dont care and cant relate to.

Whatever still I might consider it in the future.

The main problem was that I did not even have the idea to try to
bytecompile it, so I did not even see that warnings ever.

Re: Strange eval behaviour

[Stefan Huchler]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> But the warnings are there for a reason, and the reason is that the code
> can potentially error when being run.  In general, the warnings mean
> that you do something you definitely should not do.  Let's fix these
> warnings, and then I can have a look at the code again if you want.

Hi,

I come back to your offer to look over the code again, I don't know if a
github link is the right form for you. If not, can I add here file
attachments like in a mail?

https://github.com/spiderbit/kodi-remote.el

Would be nice if you have a minute, if not it's ok too.

I mean I belive I solved all the related issues, no warning anymore. The
Software still of course use much love (replace sit-for statements with
locking mechanism or something like that).

But that are different issues, I also experimenting a bit with
ert. (especialy batch mode) But for some non-tagged imaginary version
0.1 thats not that bad I think.

I dont earn money with any elisp stuff, so primary its for myself, but I
thought could be useful for others, too.

Regards

Re: Strange eval behaviour

[Stefan Huchler]

Forgot to mention that its in melpa if you wanna install it:
http://melpa.org/#/kodi-remote

Well as I am writing that, its not the current github version, but
should be updated in the next hours.