[shepherd] several patches that i deem ready

[shepherd] several patches that i deem ready

[Attila Lendvai]

dear Guix, Ludo,

i have prepared the rest of my commits that were needed to hunt down the shepherd hanging bug. you can find them at:

https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila

there's some dependency among the commits, so sending them to debbugs would be either as one big series of commits, or a hopeless labirinth of patches otherwise.

therefore i recommend the following workflow instead (assuming that Ludo is pretty much the only one hacking on shepherd):

Ludo, please take a look at my branch, and cherry-pick whatever you are happy with. then based on your feedback, and the new main branch, i'll rebase and refine my commits and give you a head's up when it's ready for another merge/review.

the commits are more or less ordered in least controversial order, modulo dependencies.

the main additions are:

- a multi-layered error handler that got employed at various points in
  the codebase. this makes shepherd much more resilient, even in case
  of nested errors, and much more communicative in the log when errors
  end up happening.

- a lightweight logging infrastructure together with plenty of log
  lines throughout the codebase, and some hints in the README on how
  to turn log lines gray in emacs (i.e. easily ignorable).

looking forward to your feedback,

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“What you do speaks so loud I cannot hear what you say.”
	— Ralph Waldo Emerson (1803–1882)

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

> - a lightweight logging infrastructure together with plenty of log
> lines throughout the codebase, and some hints in the README on how
> to turn log lines gray in emacs (i.e. easily ignorable).


a quick note on the log statements: they are essentially noise when it comes to reading the code, hence the gray coloring i suggest in emacs. (although they may often serve also as "executable" comments).

i'd also like to propose to relax the 80 column limit for log lines for the same reason that i've explained above.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Happiness, whether consisting in pleasure or virtue, or both, is more often found with those who are highly cultivated in their minds and in their character, and have only a moderate share of external goods.”
	— Aristotle (BC 384–322), 'Book VII, 1323.b1'

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai <attila@lendvai.name> writes:

>> - a lightweight logging infrastructure together with plenty of log
>> lines throughout the codebase, and some hints in the README on how
>> to turn log lines gray in emacs (i.e. easily ignorable).

Are you using guile-lib's logging library for it?  I've used it in
guile-hall if you want to have an example.  We should maximize its
users, refine it and aim to have it builtin Guile at some point.

>
> a quick note on the log statements: they are essentially noise when it
> comes to reading the code, hence the gray coloring i suggest in
> emacs. (although they may often serve also as "executable" comments).
>
> i'd also like to propose to relax the 80 column limit for log lines
> for the same reason that i've explained above.

I don't think an exception is deserved here; the logging library from
guile-lib for example concatenates message strings by default, which
makes it easy to brake long messages on multiple lines.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

hi Maxim,

> > > - a lightweight logging infrastructure together with plenty of log
> > > lines throughout the codebase, and some hints in the README on how
> > > to turn log lines gray in emacs (i.e. easily ignorable).
>
>
> Are you using guile-lib's logging library for it? I've used it in
> guile-hall if you want to have an example. We should maximize its
> users, refine it and aim to have it builtin Guile at some point.


i looked at that lib first (IIRC by your recommendation), but i ended up rolling my own for the cost of two additional pages of code in shepherd. i think the main issue i had was the amount of unconditional computation that happens on the common code path, and its complexity in general, including its API.

shepherd has some non-trivial machinery regarding logging output being captured and redirected through sockets to herd and whatnot; i.e. most of the handler machinery in guile-lib's logger would be just an impedance mismatch instead of being helpful.

for those two pages it's:
 - one less external dependency
 - less resource use
 - more flexibility
 - cheaper code path when a log level is disabled at runtime
 - compile-time log level to drop entire log levels
 - and most importantly much less code complexity.

you can find the relevant commit at:

https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila

FWIW, it's a partial bort of a CL lib of mine:

https://github.com/hu-dwim/hu.dwim.logger


> > a quick note on the log statements: they are essentially noise when it
> > comes to reading the code, hence the gray coloring i suggest in
> > emacs. (although they may often serve also as "executable" comments).
> >
> > i'd also like to propose to relax the 80 column limit for log lines
> > for the same reason that i've explained above.
>
>
> I don't think an exception is deserved here; the logging library from
> guile-lib for example concatenates message strings by default, which
> makes it easy to brake long messages on multiple lines.


my ultimate goal here is to minimize the disruption of code readability. only some emacs (editor) magic and/or code formatting can help with that.

log lines are only relevant when you're debugging something, or when you're trying to understand a codebase. all other times they are essentially noise.

my proposal is what our CL team settled with: always one line per log statement, and setting the foreground color of the entire line gray in emacs.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The pursuit of commerce reconciles nations, calms wars, strengthens peace, and commutes the private good of individuals into the common benefit of all.”
	— Hugh of Saint Victor (1096–1141)

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai <attila@lendvai.name> writes:

> hi Maxim,
>
>> > > - a lightweight logging infrastructure together with plenty of log
>> > > lines throughout the codebase, and some hints in the README on how
>> > > to turn log lines gray in emacs (i.e. easily ignorable).
>>
>>
>> Are you using guile-lib's logging library for it? I've used it in
>> guile-hall if you want to have an example. We should maximize its
>> users, refine it and aim to have it builtin Guile at some point.
>
>
> i looked at that lib first (IIRC by your recommendation), but i ended
> up rolling my own for the cost of two additional pages of code in
> shepherd. i think the main issue i had was the amount of unconditional
> computation that happens on the common code path, and its complexity
> in general, including its API.
>
> shepherd has some non-trivial machinery regarding logging output being
> captured and redirected through sockets to herd and whatnot; i.e. most
> of the handler machinery in guile-lib's logger would be just an
> impedance mismatch instead of being helpful.

Since we can define a custom logger for guile-lib, I'm a bit surprised
by this, but it's true that this logging library is quite minimal/naive.

> for those two pages it's:
>  - one less external dependency
>  - less resource use
>  - more flexibility
>  - cheaper code path when a log level is disabled at runtime
>  - compile-time log level to drop entire log levels
>  - and most importantly much less code complexity.

About "cheaper code path when a log level is disabled at runtime",
perhaps it can be improved in guile-lib, but otherwise that's a nice
list.  I just wish we had a good logging library in Guile and could stop
reinventing the wheel left and right.

> you can find the relevant commit at:
>
> https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila
>
> FWIW, it's a partial bort of a CL lib of mine:
>
> https://github.com/hu-dwim/hu.dwim.logger
>
>
>> > a quick note on the log statements: they are essentially noise when it
>> > comes to reading the code, hence the gray coloring i suggest in
>> > emacs. (although they may often serve also as "executable" comments).
>>
>> > i'd also like to propose to relax the 80 column limit for log lines
>> > for the same reason that i've explained above.
>>
>>
>> I don't think an exception is deserved here; the logging library from
>> guile-lib for example concatenates message strings by default, which
>> makes it easy to brake long messages on multiple lines.
>
>
> my ultimate goal here is to minimize the disruption of code readability. only some emacs (editor) magic and/or code formatting can help with that.
>
> log lines are only relevant when you're debugging something, or when you're trying to understand a codebase. all other times they are essentially noise.
>
> my proposal is what our CL team settled with: always one line per log statement, and setting the foreground color of the entire line gray in emacs.

OK.  For levels greater than debug, they I see them as glorified
comments (executable comments as yo wrote), so I don't see a strong
reason to attempt to hide them or treat them specially.  In Python
(which strives to be readable), we typically break logging lines (which
are concatenated for free inside the parens -- default Python behavior),
and that doesn't hurt readability in my opinion, and means we can just
follow the usual style rules, keeping things simple.

Thanks for working on this, I'm sure it'll help many, myself included,
following the execution of Shepherd more easily.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

> About "cheaper code path when a log level is disabled at runtime",
> perhaps it can be improved in guile-lib, but otherwise that's a nice
> list. I just wish we had a good logging library in Guile and could stop
> reinventing the wheel left and right.


i've made my judgement that the logger in guile-lib was never applied seriously when i relized that it stores the enabled state in a hashtable (which must be looked up for every log statement).

i made sure the log statements have a unique syntax, so the underlying machinery can be replaced easily later, and then i moved on.


> OK. For levels greater than debug, they I see them as glorified
> comments (executable comments as yo wrote), so I don't see a strong
> reason to attempt to hide them or treat them specially. In Python
> (which strives to be readable), we typically break logging lines (which
> are concatenated for free inside the parens -- default Python behavior),
> and that doesn't hurt readability in my opinion, and means we can just
> follow the usual style rules, keeping things simple.


my experience is different. i found myself only ever looking at log statements when i'm debugging something, regardless of the level, and including other people's code. and then i just toggle line wrap with the press of a button.

must be related to my habit that i usually put more effort into making the code more self-documenting (readable) than i put into writing informal comments and documentation. and rethinking my "executable comment" metaphore: these log statements serve much less as comments than reporting the temporal state and program flow.

but my primary aim is to color it all gray, and i don't immediately know how to do that in emacs for multiline sexp's (i.e. balanced parens). this is the primary reason our team just kept them on one line, but the flexibility of toggling word wrap as needed is also nice: the essetial part is always within a reasonable margin, and the rest can be read when word-wrap is enabled.

if requested, then i'm willing to re-format the log statements if i can find a way to still color it all gray. it's important that logging stays out of sight while reading the code.


> Thanks for working on this, I'm sure it'll help many, myself included,
> following the execution of Shepherd more easily.


my pleasure!

in my experience when a project doesn't have proper logging, backtraces, error handling hygene, and warning-free compilation, then inefficient debugging quickly eats up more time than it would take to implement these features properly.

unfortunately, guix and guile is not very good on this front, so i found myself working on these, too. such investment rarely pays off for the first bug, but it pays off very well in the long run.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“A political situation is the manifestation of a parallel psychological problem in millions of individuals. This problem is largely unconscious (which makes it a particularly dangerous one!)”
	— Carl Jung (1875–1961), Letters, vol.1 pg. 535

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi Attila,

Attila Lendvai <attila@lendvai.name> writes:

>> About "cheaper code path when a log level is disabled at runtime",
>> perhaps it can be improved in guile-lib, but otherwise that's a nice
>> list. I just wish we had a good logging library in Guile and could stop
>> reinventing the wheel left and right.
>
>
> i've made my judgement that the logger in guile-lib was never applied
> seriously when i relized that it stores the enabled state in a
> hashtable (which must be looked up for every log statement).
>
> i made sure the log statements have a unique syntax, so the underlying
> machinery can be replaced easily later, and then i moved on.

I guess if we were serious at giving Scheme/Guile a great logging
library we would try to design it with the help of everyone as a SRFI;
it could then be integrated to Guile and any other Scheme, benefiting
the ecosystem at large.  Is this something you could be interested in
trying?  I've found the Scheme community around SRFIs very knowledgeable
and helpful.

>> OK. For levels greater than debug, they I see them as glorified
>> comments (executable comments as yo wrote), so I don't see a strong
>> reason to attempt to hide them or treat them specially. In Python
>> (which strives to be readable), we typically break logging lines (which
>> are concatenated for free inside the parens -- default Python behavior),
>> and that doesn't hurt readability in my opinion, and means we can just
>> follow the usual style rules, keeping things simple.
>
>
> my experience is different. i found myself only ever looking at log
> statements when i'm debugging something, regardless of the level, and
> including other people's code. and then i just toggle line wrap with
> the press of a button.
>
> must be related to my habit that i usually put more effort into making
> the code more self-documenting (readable) than i put into writing
> informal comments and documentation. and rethinking my "executable
> comment" metaphore: these log statements serve much less as comments
> than reporting the temporal state and program flow.
>
> but my primary aim is to color it all gray, and i don't immediately
> know how to do that in emacs for multiline sexp's (i.e. balanced
> parens). this is the primary reason our team just kept them on one
> line, but the flexibility of toggling word wrap as needed is also
> nice: the essetial part is always within a reasonable margin, and the
> rest can be read when word-wrap is enabled.

I'm sure there's a way; have you consulted in #emacs on Libera?

> if requested, then i'm willing to re-format the log statements if i
> can find a way to still color it all gray. it's important that logging
> stays out of sight while reading the code.

OK; I'll let other voices their preference (because it's down to that,
and current conventions).

>
>> Thanks for working on this, I'm sure it'll help many, myself included,
>> following the execution of Shepherd more easily.
>
>
> my pleasure!
>
> in my experience when a project doesn't have proper logging,
> backtraces, error handling hygene, and warning-free compilation, then
> inefficient debugging quickly eats up more time than it would take to
> implement these features properly.
>
> unfortunately, guix and guile is not very good on this front, so i
> found myself working on these, too. such investment rarely pays off
> for the first bug, but it pays off very well in the long run.

I wholly agree.  Sadly, I find the debugging facilities of Guile are
also lacking compared to what other programming languages have, which
further exacerbate that situation.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Zheng Junjie]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Attila,
>
> Attila Lendvai <attila@lendvai.name> writes:
>
>>> About "cheaper code path when a log level is disabled at runtime",
>>> perhaps it can be improved in guile-lib, but otherwise that's a nice
>>> list. I just wish we had a good logging library in Guile and could stop
>>> reinventing the wheel left and right.
>>
>>
>> i've made my judgement that the logger in guile-lib was never applied
>> seriously when i relized that it stores the enabled state in a
>> hashtable (which must be looked up for every log statement).
>>
>> i made sure the log statements have a unique syntax, so the underlying
>> machinery can be replaced easily later, and then i moved on.
>
> I guess if we were serious at giving Scheme/Guile a great logging
> library we would try to design it with the help of everyone as a SRFI;
> it could then be integrated to Guile and any other Scheme, benefiting
> the ecosystem at large.  Is this something you could be interested in
> trying?  I've found the Scheme community around SRFIs very knowledgeable
> and helpful.
>

maybe can see srfi-215 Central Log Exchange
[https://srfi.schemers.org/srfi-215/srfi-215.html]?

Re: [shepherd] several patches that i deem ready

[Maxim Cournoyer]

Hi,

Zheng Junjie <zhengjunjie@iscas.ac.cn> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> Hi Attila,
>>
>> Attila Lendvai <attila@lendvai.name> writes:
>>
>>>> About "cheaper code path when a log level is disabled at runtime",
>>>> perhaps it can be improved in guile-lib, but otherwise that's a nice
>>>> list. I just wish we had a good logging library in Guile and could stop
>>>> reinventing the wheel left and right.
>>>
>>>
>>> i've made my judgement that the logger in guile-lib was never applied
>>> seriously when i relized that it stores the enabled state in a
>>> hashtable (which must be looked up for every log statement).
>>>
>>> i made sure the log statements have a unique syntax, so the underlying
>>> machinery can be replaced easily later, and then i moved on.
>>
>> I guess if we were serious at giving Scheme/Guile a great logging
>> library we would try to design it with the help of everyone as a SRFI;
>> it could then be integrated to Guile and any other Scheme, benefiting
>> the ecosystem at large.  Is this something you could be interested in
>> trying?  I've found the Scheme community around SRFIs very knowledgeable
>> and helpful.
>>
>
> maybe can see srfi-215 Central Log Exchange
> [https://srfi.schemers.org/srfi-215/srfi-215.html]?

That's somewhat related, but is not an actually usable logging library,
just some standard means to dispatch them to various sinks, if I
recall my read of it.

-- 
Thanks,
Maxim

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

> i have prepared the rest of my commits that were needed to hunt down the shepherd hanging bug. you can find them at:
> 
> https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila


FWIW, here i have the guix side of the patches (they are not required for the shepherd changes):

https://codeberg.org/attila-lendvai-patches/guix/commits/branch/shepherd-guix-side

the first commit touches hurd, which i have not tested.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“But if you wish to remain slaves of bankers and pay the cost of your own slavery, let them create money.”
	— Joshua Stamp

[OT: s/Joshua/Josiah/ in sig ;-] Re: [shepherd] several patches that i deem ready

[bokr]

On +2024-01-24 17:11:28 +0000, Attila Lendvai wrote:
[...]
> -- 
> • attila lendvai
> • PGP: 963F 5D5F 45C7 DFCD 0A39
> --
> “But if you wish to remain slaves of bankers and pay the cost of your own slavery, let them create money.”
>       — Joshua Stamp
          ^^^^^^
          Josiah
    <https://en.wikipedia.org/wiki/Josiah_Stamp,_1st_Baron_Stamp>

>
Hi attila (and others who, like me, may enjoy the quotations
at the bottom of your posts :)

Should such misspellings be reported somewhere as a bug?

--
Regards,
Bengt Richter

Re: [OT: s/Joshua/Josiah/ in sig ; -] Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

> > “But if you wish to remain slaves of bankers and pay the cost of your own slavery, let them create money.”
> > — Joshua Stamp
> 
> ^^^^^^
> Josiah
> https://en.wikipedia.org/wiki/Josiah_Stamp,_1st_Baron_Stamp
> 
> Hi attila (and others who, like me, may enjoy the quotations
> at the bottom of your posts :)


your report is much appreciated, and thanks for your kind words, too! it's good to know that someone not only enjoys them, but that it has initiated further research.

it reminds me of how it all started:

years ago i found myself, that on a mailing list i was only reading the end of mail quotes from a great hacker (http://fare.tunes.org), from whom i have learned a lot on a wide range of topics. and then it struck me: i should have this too! (be the change you want to see in the world, and whatnot... :)

in that spirit, my scripts and my collection is available below (it often has quotes and references in comments, and it's grouped by topics):

https://codeberg.org/attila.lendvai/dotfiles


> Should such misspellings be reported somewhere as a bug?


an email like this is perfect. you may consider keeping it off-list though, to respect the topic of the list.

thanks again and happy hacking,

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Many people believe that evil is the presence of something. I think it’s the absence of something.”
	— Lisa Unger (1970–), 'Sliver of Truth'

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

> i have prepared the rest of my commits that
> were needed to hunt down the shepherd hanging bug.
> you can find them at:

https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various

Ludo, i would appreciate if you could give some feedback on this.

these commits are extensive (line-diff-wise) due to the added log statements, and the error handling wrapper forms. the more you work on shepherd, the more these commits rot away, and the more avoidable work/frustration it is to keep rebasing them.

i believe that these are valuable additions to shepherd, as was the reconfigure hang fix that these were needed for.

as a first phase, maybe you could cherry pick some of the commits that you find agreeable.

i'm looking forward to your feedback on how i could/should improve these to get them merged.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“When training beats education, civilization dies.”
	— C. S. Lewis (1898–1963)

Re: [shepherd] several patches that i deem ready

[Ludovic Courtès]

Hi Attila,

Attila Lendvai <attila@lendvai.name> skribis:

> i have prepared the rest of my commits that were needed to hunt down the shepherd hanging bug. you can find them at:
>
> https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila
>
> there's some dependency among the commits, so sending them to debbugs would be either as one big series of commits, or a hopeless labirinth of patches otherwise.

Yes, but OTOH, piecemeal, focused changes sent to Debbugs are easier to
review for me.  (There are 34 commits in this branch touching different
aspects.)

> therefore i recommend the following workflow instead (assuming that Ludo is pretty much the only one hacking on shepherd):
>
> Ludo, please take a look at my branch, and cherry-pick whatever you are happy with. then based on your feedback, and the new main branch, i'll rebase and refine my commits and give you a head's up when it's ready for another merge/review.
>
> the commits are more or less ordered in least controversial order, modulo dependencies.
>
> the main additions are:
>
> - a multi-layered error handler that got employed at various points in
>   the codebase. this makes shepherd much more resilient, even in case
>   of nested errors, and much more communicative in the log when errors
>   end up happening.
>
> - a lightweight logging infrastructure together with plenty of log
>   lines throughout the codebase, and some hints in the README on how
>   to turn log lines gray in emacs (i.e. easily ignorable).

I cherry-picked a couple of patches.

Some notes:

+ 94c1143 shepherd: Add tests/startup-error.sh

  Redundant with ‘tests/startup-failure.sh’ I think?

+ e802761 service: Add custom printer for <service> records.

  Good idea, but the goal is to remove GOOPS, so put aside for now.

+ af2ebec service: respawn-limit: make #f mean no limit.

  I’d rather not do that: one can use +inf.0 when needed.

+ 095e930 shepherd: Do not respawn disabled services.

  That’s already the case (see commit
  7c88d67076a0bb1d9014b3bc23ed9c68f1c702ab; maybe we hacked it
  independently in parallel).

+ dbc9150 shepherd: Increase the time range for the default respawn limit.

  This arbitrary and thus debatable, but I think the current setting
  works well, doesn’t it?

+ e03b958 support: Add logging operators.
+ 39c2e14 shepherd: add call-with-error-handling

  I like the idea: we really need those backtraces to be logged!
  There are mostly-stylistic issues that would need to be discussed
  though.  I’d like logging to be less baroque; I’m not convinced by:

    + 7183c9c shepherd: Populate the code with some log lines.

  This is exactly what I’d like to avoid—adding logging statements all
  around the code base, possibly redundant with existing logging
  statements that target users.

  What I do want though is to have “first-class logs”, pretty much like
  what we see with ‘herd log’ etc.  To me, that is much more useful than
  writing the arguments passed each and every ‘fork+exec-command’ call.

I’ll have to look further that branch.  I admit I have limited bandwidth
available and, perhaps selfishly, I like to use my free-time computing
to hack myself.

Regardless, I’d like to thank you for your continued efforts on the
Shepherd.  In one way or another, it contributes to shaping it.

Ludo’.

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

hi Ludo,


> > i have prepared the rest of my commits that were needed to hunt down the shepherd hanging bug. you can find them at:
> > 
> > https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/attila
> > 
> > there's some dependency among the commits, so sending them to debbugs would be either as one big series of commits, or a hopeless labirinth of patches otherwise.
> 
> 
> Yes, but OTOH, piecemeal, focused changes sent to Debbugs are easier to
> review for me. (There are 34 commits in this branch touching different
> aspects.)


i understand that, but cutting out some of the commits from this branch is a lot of work at best, and not possible at worst due to semantic dependencies.

e.g. how shall i implement proper error handling without being able to inspect what's happening (i.e. proper logging)?

nevertheless, i'll rebase my work on the devel branch eventually. it will be a lot of pain in itself, but if i need to reimplement/rebase stuff by hand anyway, then i'll try to further sort the commits in a least-controversial order.


> I cherry-picked a couple of patches.
> 
> Some notes:
> 
> + 94c1143 shepherd: Add tests/startup-error.sh
> 
> Redundant with ‘tests/startup-failure.sh’ I think?


one of them just returns #f from its start lambda, while the new one throws an error. they exercise different code paths in shepherd.


> + e802761 service: Add custom printer for <service> records.
> 
> 
> Good idea, but the goal is to remove GOOPS, so put aside for now.


ok, i'll get rid of it (move it away into a local kludge branch). its main purpose is to be able to simply FORMAT some service objects into the log.


> + af2ebec service: respawn-limit: make #f mean no limit.
> 
> I’d rather not do that: one can use +inf.0 when needed.


i found the respawn-limit API somewhat confusing (it requires a cons cell with two numbers). i thought #f could be a simple way to disable the respawn limit; simple both in implementation and as an API.

FWIW, it's the first time i've ever met +inf.0

but as you wish, we can manage without this commit.


> + 095e930 shepherd: Do not respawn disabled services.
> 
> That’s already the case (see commit
> 7c88d67076a0bb1d9014b3bc23ed9c68f1c702ab; maybe we hacked it
> independently in parallel).


err, hrm... i'm not sure anymore why i created that commit.

"Respawning ~a." is printed before calling START-SERVICE (that then does honor the ENABLED? flag). maybe i recorded this commit without actually checking whether the service is respawned (as opposed to merely printing an inert log message).

i'll get rid of this, but the incorrect respawning message will remain a source of confusion.


> + dbc9150 shepherd: Increase the time range for the default respawn limit.
> 
> This arbitrary and thus debatable, but I think the current setting
> works well, doesn’t it?


the current limit will not catch services whose start-to-fail time is not in the ballpark of 1 sec (5 times in 7 seconds).

the startup-to-fail time of the service i'm working with is way above 1 sec.


> + e03b958 support: Add logging operators.
> + 39c2e14 shepherd: add call-with-error-handling
> 
> I like the idea: we really need those backtraces to be logged!
> There are mostly-stylistic issues that would need to be discussed
> though. I’d like logging to be less baroque; I’m not convinced by:


what do you mean by 'baroque' here? too verbose in the source code?


> + 7183c9c shepherd: Populate the code with some log lines.
> 
> This is exactly what I’d like to avoid—adding logging statements all
> around the code base, possibly redundant with existing logging
> statements that target users.
> 
> What I do want though is to have “first-class logs”, pretty much like
> what we see with ‘herd log’ etc. To me, that is much more useful than
> writing the arguments passed each and every ‘fork+exec-command’ call.


don't they serve two entirely different purposes?

1) logs meant for the users of shepherd (aka herd log)

vs

2) logs that the shepherd and service developers need to understand shepherd's temporal behavior.


i added every logging related code in the various pursuits of hunting down specific bugs.

1. bug gets triggered
2. stare at logs, have some questions
3. add some more log statements
4. goto 1.

i'm not aware of any way to efficiently inspect the temporal behavior of a codebase other than adding explicit log statements. ideally using multiple, hierarchical log categories that can be turned on and off separately, both at runtime and at compile time.

what i added to shepherd is a super simplified, local, mock version of that (short of porting/finding a proper logging library in scheme).


> I’ll have to look further that branch. I admit I have limited bandwidth
> available and, perhaps selfishly, I like to use my free-time computing
> to hack myself.


it is of course your call how you make a tradeoff between building/fixing things by yourself, and spending your time on external contributions.

note though that such decisions will fundamentally influence the path the project takes. and often it's an unseen influence in the form of contributions that could have manifested, but in the end did not manifest.

the input i can give here is that it would be dismaying for a contributor like me to see a oneliner bugfix get accepted, while the entire work that was needed for finding that bug remains ignored (and be unavailable when looking for the next bug, or remain in a local branch that regularly takes up a lot of time to rebase).

and the icing on the cake here is that many people, who are not enthusiastic enough, would not even complain. they just quietly chose not to contribute.

this ties back into other discussions about the use of debbugs, the bandwidth issues around contributions, the various demands like the changelog format in commit messages, what is seen and what is not seen (Bastiat reference), etc.


> Regardless, I’d like to thank you for your continued efforts on the
> Shepherd. In one way or another, it contributes to shaping it.


my pleaseure. and thank you for creating/maintaining shepherd! it's a lot of fun to work with, and it makes things possible that i wouldn't even try in e.g. NixOS (where i have arrived from).

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“[…] without children and old people mixing in daily life a community has no future and no past, only a continuous present.”
	— John Taylor Gatto (1935–2018), 'Teacher of the Year Acceptance Speech'

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

hi Ludo,


> nevertheless, i'll rebase my work on the devel branch eventually. it
> will be a lot of pain in itself, but if i need to reimplement/rebase
> stuff by hand anyway, then i'll try to further sort the commits in a
> least-controversial order.


i've rebased my commits on top of the devel branch, and in the process i've reordered them into a least controversial order for your cherry-picking convenience:

https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various

i just started a wave of deeper testing after the rebase, so the more complex commits may change, but those need further work/negotiation anyway.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“We have been to the moon, we have charted the depths of the ocean and the heart of the atom, but we have a fear of looking inward to ourselves because we sense that is where all the contradictions flow together.”
	— Terence McKenna (1946–2000)

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

> i've rebased my commits on top of the devel branch, and in the process i've reordered them into a least controversial order for your cherry-picking convenience:
> 
> https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various
> 
> i just started a wave of deeper testing after the rebase, so the more complex commits may change, but those need further work/negotiation anyway.


Ludo, the first commit ('Replace stop with stop-service in power-off of the root service.') used to serve to avoid a warning, but on the 'devel' branch it is now essential:

# halt
halt: error: exception caught while executing 'power-off' on service 'root':
Unbound variable: stop

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Tyranny is defined as that which is legal for the government but illegal for the citizenry.”
	— Thomas Jefferson (1743–1826)

Re: [shepherd] several patches that i deem ready

[Attila Lendvai]

i've just force-pushed it with more conformant commit messages:

https://codeberg.org/attila-lendvai-patches/shepherd/commits/branch/various

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“If controlling another human being is the goal of parenting, then force is necessary. Fear, intimidation, threats, power-plays, and physical pain are the means of control. But if growing healthy humans is the goal, then building trust relationships, encouraging, guiding, leading, teaching, and communicating are the tools for success.”
	— L. R. Knost, 'The Problem with Punishment'