Would it help to have facilities of specifying the files in a project
by starting with an empty project, and then adding the files one by
one? Also, to be able to say that all the files in a given directory
(optionally, only files that match some shell wildcard), recursively,
should be added to a project?

Yeah! I actually suggested something like that. However, this will not fix the issue with programs looking for a specific directory for root. Such as eglot and whatever-language-server.

This would probably need a way of making the list of files/directories
in a project persistent between sessions, because currently we rely on
the filesystem or a VCS to record that.

Yeah, maybe something like that too.

Theo