find-directory-files-recursively-2

Uses make-process and :filter to parse the output
concurrently with the process.

With (read-process-output-max 4096):

start now
counts diff (75840 13 31177 60 343443 3496 4748)
counts diff (41946 1 460 0 1226494 0 8425)
counts diff (43165 1 450 0 1284214 0 8951)
counts diff (43513 1 364 0 1343316 0 10125)
counts diff (43200 1 384 0 1479048 0 9766)
counts diff (46220 1 428 0 1528863 0 10242)
counts diff (43125 1 462 0 1767068 0 8790)
counts diff (49118 1 458 0 1723271 0 10832)
counts diff (53156 1 572 0 1789919 0 10774)
counts diff (57755 1 548 0 1783286 0 12600)
counts diff (62171 1 554 0 1795216 0 13995)
counts diff (62020 1 550 0 1963255 0 13996)
counts diff (54559 1 616 0 2387308 0 10700)
counts diff (56428 1 634 0 2513219 0 11095)
counts diff (62611 1 658 0 2510756 0 12864)
counts diff (67560 1 708 0 2574312 0 13899)
counts diff (78154 1 928 0 2572273 0 14714)
counts diff (86794 1 976 0 2520915 0 17004)
counts diff (78112 1 874 0 2943548 0 15367)
counts diff (79443 1 894 0 3138948 0 15559)
counts diff (81861 1 984 0 3343764 0 15260)
counts diff (87724 1 1030 0 3430969 0 16650)
counts diff (88532 1 902 0 3591052 0 18487)
counts diff (92083 1 952 0 3769290 0 19065)
<finished\n>
Elapsed time: 1.344422s (0.747126s in 24 GCs)

And here's with (read-process-output-max 409600):

start now
counts diff (57967 1 4040 1 981912 106 7731)
counts diff (32075 1 20 0 1919096 0 10560)
counts diff (43431 1 18 0 2259314 0 14371)
counts diff (46335 1 18 0 2426290 0 15339)
counts diff (31872 1 18 0 2447639 0 10518)
counts diff (46527 1 18 0 2328042 0 15403)
counts diff (42468 1 18 0 2099976 0 14050)
counts diff (48648 1 18 0 2302713 0 16110)
counts diff (50404 1 20 0 3260921 0 16669)
counts diff (40147 1 20 0 3264463 0 13251)
counts diff (48118 1 20 0 3261725 0 15908)
counts diff (60732 1 282 0 2791003 0 16785)
counts diff (71329 1 506 0 2762237 0 17487)
counts diff (61455 1 342 0 3192771 0 16271)
counts diff (49035 1 30 0 3663715 0 16085)
counts diff (58651 1 236 0 3783888 0 16683)
counts diff (57132 1 24 0 4557688 0 18862)
counts diff (71319 1 24 0 4769891 0 23591)
<finished\n>
Elapsed time: 0.890710s (0.546486s in 18 GCs)