>>>> This could be fixed by adding special-handling of the default-directory
>>>> for the current buffer in 'project-buffers'.
>>> What kind of special handling? The "real" buffer-local value is hidden
>>> until the "let" exists, the global value is nil, and if the buffer is not
>>> a file-visiting one, there is no other file name to test against.
>> Additional buffer-local variable like 'buffer-default-directory' could help.
>> Or additional global variable 'global-default-directory'.  Or even
>> using the global value of the existing variable 'default-directory'.
>
> What code would use it instead of the local value of default-directory?
> Only project-related code? Or other code as well? If it's the former, we
> have an existing variable in the project package. If the latter, we'd need
> some formal description of those usage rules to proceed.

The former.  So given all the considered constraints we have to admit
there is a possibility that some rare non-project command that checks
default-directory in all buffers might get a wrong value for one buffer
when it's called immediately after 'C-x p p'.

OTOH, such a reasonable compromise can help us fix other bugs
such as bug#65558.

So here is a complete tested patch that maintains backward-compatibility
with older versions, and is localized to project.el without the need to
discuss more fundamental changes on emacs-devel, and handles 100% of
known cases such as reported in bug#58784, bug#63829, etc.