I just noticed that building Emacs master from scratch, i.e. what I do 
from time to time

     git clean -xdf \
         && ./autogen.sh \
         && ./configure --with-native-compilation \
         && bear -- make \
         && make install

went down from around 8 minutes to 4:30, and everything seems to still 
be working normally.  (This is macos 12.5.1, M1 chip.)

Kudos to whoever made that happen!