Re: processing a large buffer contents to a hash table

[Seweryn Kokot <sewkokot@gmail.com>]

Andreas Politz <politza@fh-trier.de> writes:

> Seweryn Kokot wrote:
>> Hello,
>>
>> I'm trying to write a function in elisp which returns word frequency of
>> a buffer content . It works but for a large file
>> (around 250 000 words) it takes 15 seconds, while a similar function in python
>> takes 4s. 
>>
>> Here is the function which process a buffer word by word and write word
>> frequency to a hash table. 
>>
>> (defun word-frequency-process-buffer ()
>>   (interactive)
>>   (let ((buffer (current-buffer)) bounds beg end word)
>> 	(save-excursion
>> 	  (goto-char (point-min))
>> 	  (while (re-search-forward "\\<[[:word:]]+\\>" nil t)
>           (word-frequency-incr (downcase (match-string 0))))))
>
> ...should do the same. (?)
> More specific, I think bounds-of-thing-at-point is
> your bottleneck. And try to compile everything.
>

Indeed it helps a lot. 
Now it also takes 4s to process this big file!

Thanks.
-- 
regards,
Seweryn