For reference: If icomplete-mode is on, it has the same issue with
counsel-find-file
(from swiper/counsel, https://github.com/abo-abo/
swiper/blob/master/counsel.el). After typeing each key, it will try to
re-build the file list. If turning icomplete-mode off, it will act as ido
and will not re-build the completion list.

Best Regards,
Shuguang