Problem with portable dumper

Problem with portable dumper

[Michael Albinus]

Hi,

after sleeping for months, I have decided to revive the
feature/tramp-thread-safe branch. I've merged it with master, and I've
recompiled. This results in an error:

--8<---------------cut here---------------start------------->8---
Dumping under the name bootstrap-emacs.pdmp
dumping fingerprint: 406add9509f46ad13554dfa8c2b5c4d2346e735f891479807515cd7a5226ec69
unsupported object type in dump: mutex
--8<---------------cut here---------------end--------------->8---

The error message comes from pdumper.c/dump_vectorlike, and it is likely
because I use threads, mutexes and condvars in some pre-loaded files,
like files.el or vc-hooks.el.

How could this be solved? Does dump_vectorlike needs to be extended?

Best regards, Michael.

Re: Problem with portable dumper

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Sat, 15 Feb 2020 19:45:36 +0100
> 
> Dumping under the name bootstrap-emacs.pdmp
> dumping fingerprint: 406add9509f46ad13554dfa8c2b5c4d2346e735f891479807515cd7a5226ec69
> unsupported object type in dump: mutex
> --8<---------------cut here---------------end--------------->8---
> 
> The error message comes from pdumper.c/dump_vectorlike, and it is likely
> because I use threads, mutexes and condvars in some pre-loaded files,
> like files.el or vc-hooks.el.
> 
> How could this be solved? Does dump_vectorlike needs to be extended?

Either extend dump_vectorlike, or figure out how to leave mutex
variables out of state that is dumped, I think.  Why do you need
mutexes to be dumped?

Re: Problem with portable dumper

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> Dumping under the name bootstrap-emacs.pdmp
>> dumping fingerprint:
>> 406add9509f46ad13554dfa8c2b5c4d2346e735f891479807515cd7a5226ec69
>> unsupported object type in dump: mutex
>>
>> The error message comes from pdumper.c/dump_vectorlike, and it is likely
>> because I use threads, mutexes and condvars in some pre-loaded files,
>> like files.el or vc-hooks.el.
>>
>> How could this be solved? Does dump_vectorlike needs to be extended?
>
> Either extend dump_vectorlike,

I would like to do, but I fear this is out of my capabilities :-(

> or figure out how to leave mutex variables out of state that is
> dumped, I think.  Why do you need mutexes to be dumped?

I'm working in the feature/tramp-thread-safe branch. I simply need it.

Bestz regards, Michael.

Re: Problem with portable dumper

[Stefan Monnier]

>> or figure out how to leave mutex variables out of state that is
>> dumped, I think.  Why do you need mutexes to be dumped?
> I'm working in the feature/tramp-thread-safe branch. I simply need it.

That doesn't explain why you need a mutex object to be written in the
dumped image of Emacs.

AFAICT Tramp is not even preloaded, so changing Tramp should simply not
affect the dump at all.


        Stefan

Re: Problem with portable dumper

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

> That doesn't explain why you need a mutex object to be written in the
> dumped image of Emacs.
>
> AFAICT Tramp is not even preloaded, so changing Tramp should simply not
> affect the dump at all.

The changes in files.el and vc-hooks.el are not related to Tramp. I have
changed the find-file* family of functions as well as vc-refresh-state
to run asynchronously if indicated. Tramp just profits from it.

All of this was discussed in detail months ago, when I did work on that
branch. The pdumper was introduced later, while I was away from this
work, and now I have the said problem being back.

If you want to see what happened, compare doc/emacs/files.texi,
doc/lispref/files.texi, files.el and vc-hooks.el from between master and
feature/tramp-thread-safe. etc/NEWS in the branch says also something
about (condensed).

>         Stefan

Best regards, Michael.

Re: Problem with portable dumper

[Stefan Monnier]

> The changes in files.el and vc-hooks.el are not related to Tramp. I have
> changed the find-file* family of functions as well as vc-refresh-state
> to run asynchronously if indicated. Tramp just profits from it.

Ah, I see, thanks.
Then, yes, we need to add support for mutexes to the pdumper.


        Stefan

Re: Problem with portable dumper

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Sat, 15 Feb 2020 21:58:13 +0100
> 
> The changes in files.el and vc-hooks.el are not related to Tramp. I have
> changed the find-file* family of functions as well as vc-refresh-state
> to run asynchronously if indicated. Tramp just profits from it.

Why can't you make the mutex nil before dumping and restore it at
startup?

Re: Problem with portable dumper

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> The changes in files.el and vc-hooks.el are not related to Tramp. I have
>> changed the find-file* family of functions as well as vc-refresh-state
>> to run asynchronously if indicated. Tramp just profits from it.
>
> Why can't you make the mutex nil before dumping and restore it at
> startup?

That works, thanks!

Best regards, Michael.

Re: Problem with portable dumper

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Sun, 16 Feb 2020 11:19:46 +0100
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The changes in files.el and vc-hooks.el are not related to Tramp. I have
> >> changed the find-file* family of functions as well as vc-refresh-state
> >> to run asynchronously if indicated. Tramp just profits from it.
> >
> > Why can't you make the mutex nil before dumping and restore it at
> > startup?
> 
> That works, thanks!

I'm glad my idea worked.

Btw, on a more general note: I presume that your code has a way to
disable threading in file I/O.  (If it doesn't, it should IMO, for
debugging purposes and also for situations where threading cannot be
allowed for some other reasons.)  Then you could simply arrange for
threading to be disabled during dumping, and enable it in startup.el
when Emacs starts normally.  And when threading is disabled, there
should be no need to have any mutex variables.

Re: Problem with portable dumper

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> Btw, on a more general note: I presume that your code has a way to
> disable threading in file I/O.  (If it doesn't, it should IMO, for
> debugging purposes and also for situations where threading cannot be
> allowed for some other reasons.)  Then you could simply arrange for
> threading to be disabled during dumping, and enable it in startup.el
> when Emacs starts normally.  And when threading is disabled, there
> should be no need to have any mutex variables.

For the time being, I haven't given disabling threading much
attention. There is just an additional optional argument ASYNC for the
find-file-* functions and friends.

But yes, when it will be ready for merge into master, it must be
possible to disable this in general. Debugging is one example, an Emacs
configured --without-threads another one.

Best regards, Michael.