try-this-for ?

try-this-for ?

[Emanuel Berg]

Is there something like this:

    (try-this-for SECONDS BODY)

If BODY hasn't completed after SECONDS,
everything will be aborted, i.e.
the equivalence of `keyboard-quit' (or whatever
appropriate).

But what has already happened until the point
of abortion don't have to be rolled back or any
state restored or dumped or anything like that.

If BODY do complete before SECONDS it should be
transparent in the sense that what should be
returned is the final evaluation of BODY.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[John Mastro]

Emanuel Berg <moasen@zoho.com> wrote:
> Is there something like this:
>
>     (try-this-for SECONDS BODY)
>
> If BODY hasn't completed after SECONDS,
> everything will be aborted, i.e.
> the equivalence of `keyboard-quit' (or whatever
> appropriate).
>
> But what has already happened until the point
> of abortion don't have to be rolled back or any
> state restored or dumped or anything like that.
>
> If BODY do complete before SECONDS it should be
> transparent in the sense that what should be
> returned is the final evaluation of BODY.

There's `with-timeout' in timer.el, however (from the docstring):

    The timeout is checked whenever Emacs waits for some kind of
    external event (such as keyboard input, input from subprocesses, or
    a certain time); if the program loops without waiting in any way,
    the timeout will not be detected.

So it may not be exactly what you want.

        John

Re: try-this-for ?

[Bob Newell]

>     (try-this-for SECONDS BODY)

The best way to do this is to build the timer check into 'body'
itself. That way you can be sure to always exit in a known state. If
an external timer interrupts 'body' (and I haven't yet figured out how
to do that, so it's in theory) then the state could be in some
disorder.

Of course if 'body' is something like an internal emacs call that
simply might run for a long time, this idea won't work.

If 'body' is running a /process/, however, an Emacs timer can trigger
a call to 'kill-process' or 'interrupt-process' to halt the process.

Don't know if any of this helps ...

-- 
Bob Newell
Honolulu, Hawai`i

Sent via Linux Mint 17.

Re: try-this-for ?

[Emanuel Berg]

Bob Newell wrote:

> If 'body' is running a /process/, however, an
> Emacs timer can trigger a call to
> 'kill-process' or 'interrupt-process' to halt
> the process.

But this only works for Unix processes, the
equivalent of doing

    $ program &
    $ # ...
    $ pkill -9 program

in the shell, right?

Can one do this for arbitrary Elisp
in/from Emacs?

If so, how?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[Emanuel Berg]

John Mastro wrote:

> There's `with-timeout' in timer.el, however
> (from the docstring):
>
> The timeout is checked whenever Emacs waits
> for some kind of external event (such as
> keyboard input, input from subprocesses, or
> a certain time); if the program loops without
> waiting in any way, the timeout will not
> be detected.

Right, I suspected that... so what I ask for
perhaps cannot be done because of the very
architecture? There is no preemption,
only collaboration?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Jan 18, 2018 at 03:56:44AM +0100, Emanuel Berg wrote:

[...]

> Right, I suspected that... so what I ask for
> perhaps cannot be done because of the very
> architecture? There is no preemption,
> only collaboration?

There be dragons. Or dragons and mad cows.

Up to now, Emacs Lisp's model of concurrency seems to
be explicitly cooperative. The two spots in the manual
to start with ("Special Events" [0] and, in a more
modern Emacs version "Threads" [1] hint at this. Currently
it seems to be a basic limitation of the interpreter,
although the verbiage in [1] suggests that there is an
active will to change that.

So if you want to get "signals" right deep in your
search for the next Mersenne prime, you'll have to
embed some form of "yield" [2] in your calculation.

But if you alre already "yielding" at strategic spots
in your Mersenne quest, you can as well consult your
watch (i.e. give the timer functions a chance to run),
so you don't need signals, threads or other exotica
for that [3].

Believe me, this is the easier part, and if you plan
carefully and develop some sense of aesthetics it comes
about pretty organically. The dragons are elsewhere.

Cows? I never said anything about them, oh, no.

Cheers

[0] https://www.gnu.org/software/emacs/manual/html_node/elisp/Special-Events.html#Special-Events
[1] http://git.savannah.gnu.org/cgit/emacs.git/tree/doc/lispref/threads.texi?h=emacs-26
[2] https://en.wikipedia.org/wiki/Generator_(computer_programming)
[3] What do I need threads for, then? For one, if you want
   to wield more computing power in one process: remember,
   the box you bought has at least two cores, and each core
   not dedicated to Emacs is one core wasted.

- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlpgaHwACgkQBcgs9XrR2kaDXwCdGWfdXn/4Ymm83RKSwzX8/7hl
4IoAn3ZiADkHIlI+YeU2YJV2dj6XdEsw
=3VTm
-----END PGP SIGNATURE-----

Re: try-this-for ?

[Emanuel Berg]

tomas wrote:

> Up to now, Emacs Lisp's model of concurrency
> seems to be explicitly cooperative.

"seems to be explicitly" :) ?

True concurrency isn't possible even for
multicore architectures IIUC.

The perceived concurrency, how does that work
if it isn't Unix-style preemptive with
priorities and a scheduler etc.?

For example, with `list-processes', is that
really all cooperative stuff handled by the
idle timer?

I thought the idle timer did font lock and such
lightweight not-really-necessary background
work whenever it got the chance, however that
must be something else as it isn't listed.

By the way, if Emacs cannot preempt, when the
idle timer does background work during
a lack-of-input respite, does that mean, when
you start typing again, the background work
isn't preempted either, only it is programmed
like

    (when t
      (do-something step-1)
      (check-if-OK-to-continue)
      ; ...
      (do-something step-n) )

so despite not being preemptable, perceived
concurrency is achieved as everything is so
fast anyway?

> For one, if you want to wield more computing
> power in one process: remember, the box you
> bought has at least two cores, and each core
> not dedicated to Emacs is one core wasted.

I remember writing a post on this very list how
to dedicate one core entirely to Emacs! It must
be lost somewhere in the archives.
What I remember it made Emacs run slightly
faster :) I guess one can achieve virtually the
same with nice(1) and/or not running anything
else :))

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[Emanuel Berg]

By the way, did anyone try to do a "Unemacs"
merging the POSIX process model with Emacs
which with no disrespect seems to be beyond
comparison more powerful than the idle timer
stuff. Actually they aren't mutually exclusive
as the idle timer stuff can be
incorporated/rebranded as "soft real time" :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[Emanuel Berg]

> POSIX

If you didn't know it, "Portable Operating
System Interface", ~1985. The X is for Unix and
the acronym was suggested by Richard Stallman.
The former designation was IEEE-IX so no wonder
they dropped it :)

One has to wonder about the "P" tho as it seems
portability is sort of the reason for any
standard - if it isn't portable, it isn't
a standard but a manual, right?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Jan 18, 2018 at 01:00:49PM +0100, Emanuel Berg wrote:
> tomas wrote:
> 
> > Up to now, Emacs Lisp's model of concurrency
> > seems to be explicitly cooperative.
> 
> "seems to be explicitly" :) ?

(seems to be (explicitly cooperative))

With explicitly I meant that control is not
taken away behind a process's back, but the
process has to explicitly yield control.

> True concurrency isn't possible even for
> multicore architectures IIUC.

To a certain extent yes: the processors/cores have
to cooperate to access common resources (e.g. memory),
but as long as they are operating e.g. from local
cache they do run in parallel.

> The perceived concurrency, how does that work
> if it isn't Unix-style preemptive with
> priorities and a scheduler etc.?

As far as I know, the only (implicit) yield point
in Emacs is the event loop, but many things fit
in there: waiting for key events, waiting for file
descriptors to become readable/writable (e.g.
communicating with (operating system) subprocesses,
reacting to POSIX signals, the idle timer, etc.

> For example, with `list-processes', is that
> really all cooperative stuff handled by the
> idle timer?

Not really the idle timer: this one is (probably?)
triggered from the central event loop, as the others.

But the great Eli has answered that already in emacs
devel some moons ago [1]. Turns out that what I call
"event loop" is called the "input queue". If you have
a lengthy calculation and want the other things (user
input, but also file notifications, communication
via sockets, etc) to "keep running", you have to
explicitly (that's the "explicit" part in what I
labeled as "explicitly cooperative") give a chance
to this input queue to run from time to time.
Perhaps (sit-for 0) is your friend (it would
trigger a redisplay), perhaps something else.

Cheers

[1] https://lists.gnu.org/archive/html/emacs-devel/2014-10/msg00922.html
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlpglnkACgkQBcgs9XrR2kbsIwCfXg9pI1PRDrdljTab+Tuf6VJU
/BcAn0IplpC0UqNCPzfcOqcpW0Nd7034
=+CsX
-----END PGP SIGNATURE-----

Re: try-this-for ?

[Eli Zaretskii]

> Date: Thu, 18 Jan 2018 13:43:37 +0100
> From: <tomas@tuxteam.de>
> 
> As far as I know, the only (implicit) yield point
> in Emacs is the event loop, but many things fit
> in there: waiting for key events, waiting for file
> descriptors to become readable/writable (e.g.
> communicating with (operating system) subprocesses,
> reacting to POSIX signals, the idle timer, etc.

The yield point is actually the pselect call (and its emulations on
non-Posix platforms).  On the Lisp level, this means waiting for any
kind of input: from user (keyboard/mouse), subprocesses (including
explicit calls to accept-process-output), network and serial
connections, etc.

> But the great Eli has answered that already

I'll leave the "great" part on your conscience, but regardless, AFAIR
Emanuel has me on his kill-file, so mentioning my posts won't help
him.

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Jan 18, 2018 at 05:14:18PM +0200, Eli Zaretskii wrote:

> The yield point is actually the pselect call (and its emulations on
> non-Posix platforms).  On the Lisp level, this means waiting for any
> kind of input: from user (keyboard/mouse), subprocesses (including
> explicit calls to accept-process-output), network and serial
> connections, etc.

Thanks for this lucid explanation: it corresponds more or less
to my mental model, but knowing is better than some hunch around
the belly.

> I'll leave the "great" part on your conscience

My conscience is clean (in this regard, that is).

>                                              but regardless, AFAIR
> Emanuel has me on his kill-file, so mentioning my posts won't help
> him.

Hm. He's missing out a lot, then.

Cheers and thanks
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlpgu+sACgkQBcgs9XrR2kaLcgCfbxW1uieqFBDgNeSfDhxiflOV
VHgAn3htWNrbBexFMTzJMjIz8JYxX4Zf
=Ehh7
-----END PGP SIGNATURE-----

Re: try-this-for ?

[Emanuel Berg]

tomas wrote:

>> True concurrency isn't possible even for
>> multicore architectures IIUC.
>
> To a certain extent yes: the processors/cores
> have to cooperate to access common resources
> (e.g. memory), but as long as they are
> operating e.g. from local cache they do run
> in parallel.

Yes, but I meant with respect to Emacs, can one
have, for example

    (let*((value-one (compute it))    ; on CPU1
          (value-two (compute it)) )) ; on CPU2 

execute in parallel?

> central event loop

Is this the KERNEL?

> to "keep running", you have to explicitly
> (that's the "explicit" part in what I labeled
> as "explicitly cooperative") give a chance to
> this input queue to run from time to time.
> Perhaps (sit-for 0) is your friend (it would
> trigger a redisplay), perhaps something else

The problem with yielding is it won't work to
interrupt something that has put Emacs into
a hanged state...

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Jan 18, 2018 at 01:59:17PM +0100, Emanuel Berg wrote:
> tomas wrote:
> 
> >> True concurrency isn't possible even for
> >> multicore architectures IIUC.
> >
> > To a certain extent yes: the processors/cores
> > have to cooperate to access common resources
> > (e.g. memory), but as long as they are
> > operating e.g. from local cache they do run
> > in parallel.
> 
> Yes, but I meant with respect to Emacs, can one
> have, for example
> 
>     (let*((value-one (compute it))    ; on CPU1
>           (value-two (compute it)) )) ; on CPU2 
>
> execute in parallel?

No, but see threads: they *can* execute on different CPUs
(whether Emacs will give you control over that is another
question; moreover it's a pretty new feature anyway).

> > central event loop
> 
> Is this the KERNEL?

No, I was talkint about Emacs input queue.

> > to "keep running", you have to explicitly
[...]

> The problem with yielding is it won't work to
> interrupt something that has put Emacs into
> a hanged state...

That's true about most programs out there.

Cheers
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlpgnjgACgkQBcgs9XrR2ka8RgCdFeu2XIJvDaOMwyIGsUWD2Tn+
H50An3aXGw2SOr1o1mk6/uUbsUwwdEu7
=I0xt
-----END PGP SIGNATURE-----

Re: try-this-for ?

[Emanuel Berg]

tomas wrote:

>> The problem with yielding is it won't work
>> to interrupt something that has put Emacs
>> into a hanged state...
>
> That's true about most programs out there.

Except for one: Unix.

PS. Regarding KILL files, conflicts due to
    cabin fever and sleep deprivation left
    a terrible hangover at the time but not
    even my hangovers last almost a decade, and
    besides I already posted a post saying
    I was forgiving everyone including myself.
    This was 6-8y ago. Today I try not to deal
    with guilt/apologies whatsoever anymore but
    at the time it was genuine. If anyone
    didn't read it maybe it was because I was
    still in their KILL files :) Ha ha ha!
    But anyway I have no desire to re-live any
    of that so as the saying goes, altho
    I always think it sounds a silly thing to
    say, "this is the last thing I'll say about
    it".

PPS. With Gnus, we recommend *scoring* instead
     of killing.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Jan 18, 2018 at 11:44:00PM +0100, Emanuel Berg wrote:
> tomas wrote:
> 
> >> The problem with yielding is it won't work
> >> to interrupt something that has put Emacs
> >> into a hanged state...
> >
> > That's true about most programs out there.

Woah. Communication is difficult.

I meant "user space programs" in the context of Unix (more
precisely: an unixoid operating system). More specifically
I meant "practical, useful programs" and not proofs-of-concepts.

Cheers
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlphroQACgkQBcgs9XrR2kaN8QCfSP2EZs+KXqql3RAANNFEZcKM
gzIAnRLf8LLS+jGMCsWlpZk0UNM+gzKr
=0X+a
-----END PGP SIGNATURE-----

Re: try-this-for ?

[Emanuel Berg]

tomas wrote:

> I meant "user space programs" in the context
> of Unix (more precisely: an unixoid operating
> system). More specifically I meant
> "practical, useful programs" and not
> proofs-of-concepts.

OK, but do you really have that many user space
programs that does scheduling?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Jan 19, 2018 at 09:55:35AM +0100, Emanuel Berg wrote:
> tomas wrote:
> 
> > I meant "user space programs" in the context
> > of Unix (more precisely: an unixoid operating
> > system). More specifically I meant
> > "practical, useful programs" and not
> > proofs-of-concepts.
> 
> OK, but do you really have that many user space
> programs that does scheduling?

I'm actually enhancing one I wrote for a customer ten years
ago (C, GTK2). I have written many of those. And I'm far from
a special snowflake.

Every interactive program (worth its salt) does scheduling.
Every server which has to juggle different client requests
(web server, database server, ssh daemon, you name it).

In my current case, it's a GUI program. I want it to be
responsive to the user, but concurrently listen to database
notifications, send requests to the database and to an
LDAP server, and asynchronously wait for their responses
(that is: not block the GUI while doing so). At the same
time it is polling an RFID card reader hung on a serial
interface -- every 500 ms or so please look for it, again,
without glitching the GUI or missing a database notification.

Pretty standard fare, and the model isn't that different
of what Emacs does or Exim or lighttpd. Under an unixoid,
it reduces to watching a bunch of file descriptors using
one of the asynchronous interfaces offered to you by
your trusty operating system interface (select, poll,
epoll, perhaps AIO if you're fancy), thinking hard about
whether you need some prioritization (your life is way
easier if you don't) and keeping things well organized.
You notice when you don't :-)

It's not magic. It's pretty "classical" as Unix goes,
perhaps like [1].

Cheers

[1] https://en.wikipedia.org/wiki/Advanced_Programming_in_the_Unix_Environment
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlphuZUACgkQBcgs9XrR2kby+QCfQOaumMt53kx3jqxMXUOkbFtA
dgAAn1zpnkaq3e8vizynQUkAf5GIuILz
=cKCH
-----END PGP SIGNATURE-----

Re: try-this-for ?

[Emanuel Berg]

tomas wrote:

> I'm actually enhancing one I wrote for
> a customer ten years ago (C, GTK2). I have
> written many of those. And I'm far from
> a special snowflake.

Yes, I have also written them (in C and C++)
but never thought it to be really common and
even Emacs does it in a poor-man's way compared
to Unix.

If a process cannot even be preempted one might
as well extend the definition of "scheduling" to
a OO game loop that progresses the animation
state of a bunch of sprites in a linked list...

> It's not magic. It's pretty "classical" as
> Unix goes

With Unix we think rather of PCBs, IPC, context
switches, memory protection, different
scheduling algorithms, on-the-fly and
from-the-outside changes of priorities, and
so on.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: try-this-for ?

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, Jan 19, 2018 at 11:47:55AM +0100, Emanuel Berg wrote:
> tomas wrote:
> 
> > I'm actually enhancing one I wrote for
> > a customer ten years ago (C, GTK2). I have
> > written many of those. And I'm far from
> > a special snowflake.
> 
> Yes, I have also written them (in C and C++)
> but never thought it to be really common and
> even Emacs does it in a poor-man's way compared
> to Unix.

No. It delegates this stuff to the OS, which is
(most of the time) the Right Thing to Do.

> If a process cannot even be preempted

It can be preempted, see signal. But even if you
want to get signals right, the most straightforward
way is to queue the event to take care of it at a
point where your program is ready to deal with it.
An operating system basically does the same. No magic
here, either.
>                                         one might
> as well extend the definition of "scheduling" to
> a OO game loop that progresses the animation
> state of a bunch of sprites in a linked list...

Everyone does it that way. Whether you use a linked
list or (forced by priorities) a red-black tree or
a heap doesn't change the basic concept. I once had
a program collecting messages from SCADA systems. I
started with a linked list for events and told the
customer it would hold up for about 1000 sensors in
the field. Then he came back with... 10K sensors. I
had to change to a heap implementation.

> > It's not magic. It's pretty "classical" as
> > Unix goes
> 
> With Unix we think rather of PCBs, IPC, context
> switches, memory protection, different
> scheduling algorithms, on-the-fly and
> from-the-outside changes of priorities, and
> so on.

You try to delegate that to the OS: that's what it
is for. Even modern browsers, which have a dire need
for memory protection, just have separate processes
for that.

After all, I don't want even to think how to manage
things like the MMU from user space. Dragons is not
even an adequate description for that.

Cheers
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlph1OAACgkQBcgs9XrR2kYgfgCfX2bEEoSAIQ014GI4N++2S2Q/
e58An0xhCRL4wbHg2BsvdiA3q7Z7bhgb
=TUck
-----END PGP SIGNATURE-----