MPS: Area scanners

MPS: Area scanners

[Gerd Möllmann]

I remember now why I thought tht scan_bc would work they way I thought,
so that I could compute a suitable scan end and so on. I read in the MPS
docs this:

  type mps_area_scan_t

  Warning: Area scanning functions are subject to the same set of
  restrictions as format scanning functions, described under Cautions.

And Cautions links to a long list containing

  7.3. Cautions

  The MPS guarantees that format methods have exclusive access to the
  object for the duration of the call. This guarantee may entail
  suspending arbitrary threads. The methods that manipulate the object
  must not perform any sort of inter-thread locking or communication.

etc.

Question is of course now, what is with the exclusive access for area
scanning functions? Is that point a member of the "set of restrictions"?

I can't answer that ATM. Maybe someone else can?

Re: MPS: Area scanners

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Emacs Devel <emacs-devel@gnu.org>
> Date: Mon, 06 May 2024 19:12:08 +0200
> 
> I remember now why I thought tht scan_bc would work they way I thought,
> so that I could compute a suitable scan end and so on. I read in the MPS
> docs this:
> 
>   type mps_area_scan_t
> 
>   Warning: Area scanning functions are subject to the same set of
>   restrictions as format scanning functions, described under Cautions.
> 
> And Cautions links to a long list containing
> 
>   7.3. Cautions
> 
>   The MPS guarantees that format methods have exclusive access to the
>   object for the duration of the call. This guarantee may entail
>   suspending arbitrary threads. The methods that manipulate the object
>   must not perform any sort of inter-thread locking or communication.
> 
> etc.
> 
> Question is of course now, what is with the exclusive access for area
> scanning functions? Is that point a member of the "set of restrictions"?
> 
> I can't answer that ATM. Maybe someone else can?

What are those scanning function you are talking about? are they all
those fix_SOMETHING functions that run off dflt_scan etc.?  If so,
these scan Lisp objects, do they?  And so, the above restrictions on
inter-thread locking or communication are relevant for those
fix_SOMETHING functions?  If so, why is inter-thread locking relevant?
I don't see anything like that in those fix_SOMETHING functions.  Did
I miss something?

Re: MPS: Area scanners

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Emacs Devel <emacs-devel@gnu.org>
>> Date: Mon, 06 May 2024 19:12:08 +0200
>> 
>> I remember now why I thought tht scan_bc would work they way I thought,
>> so that I could compute a suitable scan end and so on. I read in the MPS
>> docs this:
>> 
>>   type mps_area_scan_t
>> 
>>   Warning: Area scanning functions are subject to the same set of
>>   restrictions as format scanning functions, described under Cautions.
>> 
>> And Cautions links to a long list containing
>> 
>>   7.3. Cautions
>> 
>>   The MPS guarantees that format methods have exclusive access to the
>>   object for the duration of the call. This guarantee may entail
>>   suspending arbitrary threads. The methods that manipulate the object
>>   must not perform any sort of inter-thread locking or communication.
>> 
>> etc.
>> 
>> Question is of course now, what is with the exclusive access for area
>> scanning functions? Is that point a member of the "set of restrictions"?
>> 
>> I can't answer that ATM. Maybe someone else can?
>
> What are those scanning function you are talking about? are they all
> those fix_SOMETHING functions that run off dflt_scan etc.?  

The fix_xy functions are subroutines that can be used in scan functions.
The scan functions are

- dflt_scan, which MPS calls an object format scan function because we
  specified it in an MPS object format. The MPS object format contains a
  number of infos about our objects (see make_dflt_fmt in igc.c).

- MPS root scanning functions, which are used to scan roots. and are
  specified when creating MPS roots. Examples scan_ambig, scan_bc, ...

All the function named are in igc.c.

> If so, these scan Lisp objects, do they? 

Yes. Some objects we allocated from MPS are also not for Lisp. Doesn't
matter.

> And so, the above restrictions on inter-thread locking or
> communication are relevant for those fix_SOMETHING functions? 

Sure.

> If so, why is inter-thread locking relevant? I don't see anything like
> that in those fix_SOMETHING functions. Did I miss something?

100% certain is, and MPS says do that format scanning function are
granted exclusice access to the area of memory being scanned. So
dflt_scan and subroutines have that.

In the thread about scan_bc the question came up if root scanning
functions also have exclusive access to what they scan. I initially
believed so 100%, forgot why and then doubted the 100%, read a bit
and now believe again to 100%.

It's a bit complicated :-).

Re: MPS: Area scanners

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> I remember now why I thought tht scan_bc would work they way I thought,
> so that I could compute a suitable scan end and so on. I read in the MPS
> docs this:
>
>   type mps_area_scan_t
>
>   Warning: Area scanning functions are subject to the same set of
>   restrictions as format scanning functions, described under Cautions.
>
> And Cautions links to a long list containing
>
>   7.3. Cautions
>
>   The MPS guarantees that format methods have exclusive access to the
>   object for the duration of the call. This guarantee may entail
>   suspending arbitrary threads. The methods that manipulate the object
>   must not perform any sort of inter-thread locking or communication.
>
> etc.
>
> Question is of course now, what is with the exclusive access for area
> scanning functions? Is that point a member of the "set of restrictions"?
>
> I can't answer that ATM. Maybe someone else can?

After reading the MPS Guide (Roots, Maintaining consistency, Scanning)
again, I must say I believe that the exclusive access is granted. The
docs everywhere basically equate format scanners and root scanning
functions.

(As an aside, I'm having a hard time imagining how stack scanning, area
scanning would word if that weren't true. It doesn't have to be
barriers, it could also temporarily halt threads.)

I'd be interested in others' opinions!

Re: MPS: Area scanners

[Helmut Eller]

On Mon, May 06 2024, Gerd Möllmann wrote:

> I'd be interested in others' opinions!

The way I understand the doc is this:
  a) Area scanners are called from a signal handler.
  b) MPS is a stop-the-world design, i.e. all threads are stopped before
     an area scanner is called.

Because of a) the usual restrictions for signal handlers apply and
because b) we have exclusive access.

Re: MPS: Area scanners

[Gerd Möllmann]

Helmut Eller <eller.helmut@gmail.com> writes:

> On Mon, May 06 2024, Gerd Möllmann wrote:
>
>> I'd be interested in others' opinions!
>
> The way I understand the doc is this:
>   a) Area scanners are called from a signal handler.
>   b) MPS is a stop-the-world design, i.e. all threads are stopped before
>      an area scanner is called.
>
> Because of a) the usual restrictions for signal handlers apply and
> because b) we have exclusive access.

Thanks!