bug#35683: wishlist: addessing statefulness of .cache(s)

bug#35683: wishlist: addessing statefulness of .cache(s)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 2239 bytes --]

Hi Guix,

AFAIU issues like the two I point below are becoming a common pattern
and are *critical*

1. gnome session not starting due to state in $HOME/.cache
   http://lists.gnu.org/archive/html/guix-devel/2019-04/msg00177.html
   Message-ID: <87ef68ibfy.fsf@elephly.net>

Ricardo Wurmus:
--8<---------------cut here---------------start------------->8---
What should we do about this?  For gdm I think it would make sense to
add an activation service extension that clears the gdm user’s home
directory.  And more generally, maybe we should offer a generic cache
cleaner service.
--8<---------------cut here---------------end--------------->8---

2. X broken display transitioning from llvm6 to llvm7 in the mesa package
   http://lists.gnu.org/archive/html/guix-devel/2019-05/msg00223.html
   Message-ID: <20190511022009.nnu6szga6desvfwd@cf0>
   see also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35575

ison:
--8<---------------cut here---------------start------------->8---
Note that deleting both shader caches was required, and also if the caches get
rebuilt on a new generation and then I try to boot into an older previously
working generation then that generation will display graphics artifacts until
the caches are deleted again.

So switching between mesa compiled with llvm 6 and 7 on AMD RX 580 either
backward or forward requires manually deleting the shader caches.
--8<---------------cut here---------------end--------------->8---


AFAIU unfortunately we have application/library state all over .cache(s)
that sometimes crashes software *and* trying to fix this upstream it's
_not_ an option [1]

often users have to delete something in some .cache by guessing, "just"
to solve some strange software crash (this is common to all distros)

maybe an activation service extension proposed by Ricardo (see above)
is the right solution: I'll try to make a summary of prevoius
discussions on this topic on guix-devel to help address this (class of)
issue(s)... sorry I cannot help coding it

WDYT?

Thanks! Gio'



[1] I say this observing this class of issues since I started using free
software: am I wrong?


-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#35683: wishlist: addessing statefulness of .cache(s)

[Julien Lepiller]

Le 11 mai 2019 09:32:43 GMT+02:00, Giovanni Biscuolo <g@xelera.eu> a écrit :
>Hi Guix,
>
>AFAIU issues like the two I point below are becoming a common pattern
>and are *critical*
>
>1. gnome session not starting due to state in $HOME/.cache
>   http://lists.gnu.org/archive/html/guix-devel/2019-04/msg00177.html
>   Message-ID: <87ef68ibfy.fsf@elephly.net>
>
>Ricardo Wurmus:
>--8<---------------cut here---------------start------------->8---
>What should we do about this?  For gdm I think it would make sense to
>add an activation service extension that clears the gdm user’s home
>directory.  And more generally, maybe we should offer a generic cache
>cleaner service.
>--8<---------------cut here---------------end--------------->8---
>
>2. X broken display transitioning from llvm6 to llvm7 in the mesa
>package
>   http://lists.gnu.org/archive/html/guix-devel/2019-05/msg00223.html
>   Message-ID: <20190511022009.nnu6szga6desvfwd@cf0>
>   see also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35575
>
>ison:
>--8<---------------cut here---------------start------------->8---
>Note that deleting both shader caches was required, and also if the
>caches get
>rebuilt on a new generation and then I try to boot into an older
>previously
>working generation then that generation will display graphics artifacts
>until
>the caches are deleted again.
>
>So switching between mesa compiled with llvm 6 and 7 on AMD RX 580
>either
>backward or forward requires manually deleting the shader caches.
>--8<---------------cut here---------------end--------------->8---
>
>
>AFAIU unfortunately we have application/library state all over
>.cache(s)
>that sometimes crashes software *and* trying to fix this upstream it's
>_not_ an option [1]
>
>often users have to delete something in some .cache by guessing, "just"
>to solve some strange software crash (this is common to all distros)
>
>maybe an activation service extension proposed by Ricardo (see above)
>is the right solution: I'll try to make a summary of prevoius
>discussions on this topic on guix-devel to help address this (class of)
>issue(s)... sorry I cannot help coding it
>
>WDYT?
>
>Thanks! Gio'
>
>
>
>[1] I say this observing this class of issues since I started using
>free
>software: am I wrong?

I wonder if we could mount a tmpfs on .cache? Would that be enough?

bug#35683: wishlist: addessing statefulness of .cache(s)

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 220 bytes --]

Julien,

Julien Lepiller wrote:
> I wonder if we could mount a tmpfs on .cache? Would that be 
> enough?

Seems a shame to waste RAM like that, when we could (ab)use FUSE: 
<https://github.com/xrgtn/nullfs>.

;-)

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#35683: wishlist: addessing statefulness of .cache(s)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

Giovanni Biscuolo <g@xelera.eu> writes:

> AFAIU issues like the two I point below are becoming a common pattern
> and are *critical*

linked bug report:

- bug#35669: Mesa is failing an assertion

[...]

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#35683: wishlist: addessing statefulness of .cache(s)

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

ehlo Giovanni,

Giovanni Biscuolo wrote:
> AFAIU unfortunately we have application/library state all over 
> .cache(s)
> that sometimes crashes software *and* trying to fix this 
> upstream it's
> _not_ an option [1]

Oh.  That's… disappointing to say the least, since these are both 
upstream bugs that Guix can't fix :-(

What exactly did you ask?  What was their response?

> often users have to delete something in some .cache by guessing, 
> "just"
> to solve some strange software crash (this is common to all 
> distros)

I have never had to do this, ever.

> maybe an activation service extension proposed by Ricardo (see 
> above)
> is the right solution: I'll try to make a summary of prevoius
> discussions on this topic on guix-devel to help address this 
> (class of)
> issue(s)... sorry I cannot help coding it

We can randomly delete whole caches in the user's stead but it's 
never the ‘right’ solution.  Only the application can manage its 
caches properly.  I still hope this is possible in both cases 
here.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#35683: wishlist: addessing statefulness of .cache(s)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 2156 bytes --]

Hello Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Giovanni Biscuolo wrote:
>> AFAIU unfortunately we have application/library state all over 
>> .cache(s)
>> that sometimes crashes software *and* trying to fix this 
>> upstream it's
>> _not_ an option [1]
>
> Oh.  That's… disappointing to say the least, since these are both 
> upstream bugs that Guix can't fix :-(
>
> What exactly did you ask?  What was their response?

I did not ask upstream, sorry for the misunderstanding: I'm just sharing
my *personal* experience, and I confess I never tried to report this
class of bugs upstream, I just fixed the issue with "rm
<somewhere>/.cache/<something>"

AFAIO (as far as I'm observing) this is a common pattern, for some
current Guix bug reports too (see previously provided links)

to be clear: I'm not stating we should not report upstream and help them
:-)

>> often users have to delete something in some .cache by guessing, 
>> "just"
>> to solve some strange software crash (this is common to all 
>> distros)
>
> I have never had to do this, ever.

lucky? :-) or Very Stateless™ userland configuration?

I'm not saying I had to do this sort of cache cleaning every week, but I
had to do that Too Often™ to be accepteble to me, on multiple
installations in 10 years

[...]

> We can randomly delete whole caches in the user's stead but it's 
> never the ‘right’ solution.

I agree, but please consider that we have to manage some upstream
defects [1], sometimes :-S

> Only the application can manage its caches properly.  I still hope
> this is possible in both cases here.

OK, but meanwhile?
IMHO it's not acceptable to have critical Guix services (e.g. GDM)
blocked by similar issues

...and sometimes (often?) statefullness of .cache is not considered
upstream, so I suspect this class of upstream bugs are _not_ going to
end soon


Thank you for your contribution! Gio'



[1] having data in .cache that crashes applications and services is bad
design IMHO, let alone having configuration in .cache

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#35683: wishlist: addessing statefulness of .cache(s)

[swedebugia]

On 2019-05-11 14:59, Giovanni Biscuolo wrote:
> Hello Tobias,
> 
> Tobias Geerinckx-Rice <me@tobias.gr> writes:
...


>>> often users have to delete something in some .cache by guessing,
>>> "just"
>>> to solve some strange software crash (this is common to all
>>> distros)
>>
>> I have never had to do this, ever.

I have no memories having to do this either during the last 15 years.

...

> [1] having data in .cache that crashes applications and services is bad
> design IMHO, let alone having configuration in .cache

+1


-- 
Cheers
Swedebugia

bug#35683: wishlist: addessing statefulness of .cache(s)

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

> --8<---------------cut here---------------start------------->8---
> What should we do about this?  For gdm I think it would make sense to
> add an activation service extension that clears the gdm user’s home
> directory.  And more generally, maybe we should offer a generic cache
> cleaner service.

I don't like that workaround much.  I mean for the time being I guess it's
OK, but let's file bug reports upstream so they are aware of the problem.

Better would be if the cache directory contained a "cache-protocol-version"
file or something and make the client program heed it and make it clear the
cache if it's the wrong version, without any Guix special case (the problem
is not not Guix-specific anyway).

It's not exactly difficult.   Most of the time the bug reports just don't
get filed--and cache invalidation is always an afterthought when 
implementing a cache (sadly).

If they say no we can still keep the workaround.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#35683: wishlist: addessing statefulness of .cache(s)

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> Better would be if the cache directory contained a "cache-protocol-version"
> file or something and make the client program heed it and make it clear the
> cache if it's the wrong version, without any Guix special case (the problem
> is not not Guix-specific anyway).

I’d be surprised if applications and libraries using ~/.cache such as
Mesa didn’t have some versioning mechanism allowing them to detect stale
cache files.  Overall, Guix isn’t special in that respect.

The problem we experienced with Mesa shader caches are probably not
Guix-specific.  I think we should try to gather more info about the
stale .cache files that trigger a Mesa crash so that we can make a
useful bug report to the Mesa developers.

(I’m saying this as someone who did not experience the crash, heheh.  :-))

Thoughts?

Thanks,
Ludo’.