hexl-find-file is sloooow

hexl-find-file is sloooow

[Werner LEMBERG]

hexl-find-file is quite slow -- loading a 4MByte font takes more than
half a minute on my laptop...

I ask to add progress information while hexl-find-file is running.

Of course, this isn't something urgent, but perhaps it can be added to
the post-release TODO list.


    Werner

Re: hexl-find-file is sloooow

[Masatake YAMATO]

> hexl-find-file is quite slow -- loading a 4MByte font takes more than
> half a minute on my laptop...
> 
> I ask to add progress information while hexl-find-file is running.
> 
> Of course, this isn't something urgent, but perhaps it can be added to
> the post-release TODO list.

It seems that fontification takes long time.
As far as I can remember the area fontification is introduced by me.
Do you want to turn off the fortification?

Anyway, here is a progress reporter. If there is no objection,
I will install this patch in a day.

Regards,
Masatake YAMATO

2004-12-21  Masatake YAMATO  <jet@gyve.org>

	* hexl.el (hexlify-buffer): Report fontification progress.

Index: hexl.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/hexl.el,v
retrieving revision 1.90
diff -u -r1.90 hexl.el
--- hexl.el	21 Nov 2004 00:38:34 -0000	1.90
+++ hexl.el	21 Dec 2004 08:46:23 -0000
@@ -680,19 +680,26 @@
   ;; Don't decode text in the ASCII part of `hexl' program output.
   (let ((coding-system-for-read 'raw-text)
 	(coding-system-for-write buffer-file-coding-system)
-	(buffer-undo-list t))
+	(buffer-undo-list t)
+	progress pmax)
     (apply 'call-process-region (point-min) (point-max)
 	   (expand-file-name hexl-program exec-directory)
 	   t t nil (split-string hexl-options))
+    (setq pmax (point-max)
+          progress (make-progress-reporter "Hexl Buffer Fontification " 
+					   (point-min) (* 2 pmax)))
     (save-excursion
       (goto-char (point-min))
       (while (re-search-forward "^[0-9a-f]+:" nil t)
+	(progress-reporter-update progress (point))
 	(put-text-property (match-beginning 0) (match-end 0)
 			   'font-lock-face 'hexl-address-area))
       (goto-char (point-min))
       (while (re-search-forward "  \\(.+$\\)" nil t)
+	(progress-reporter-update progress (+ pmax (point)))
 	(put-text-property (match-beginning 1) (match-end 1) 
 			   'font-lock-face 'hexl-ascii-area)))
+    (progress-reporter-done progress)
     (if (> (point) (hexl-address-to-marker hexl-max-address))
 	(hexl-goto-address hexl-max-address))))

Re: hexl-find-file is sloooow

[Kim F. Storm]

Masatake YAMATO <jet@gyve.org> writes:

>> hexl-find-file is quite slow -- loading a 4MByte font takes more than
>> half a minute on my laptop...
>> 
>> I ask to add progress information while hexl-find-file is running.
>> 
>> Of course, this isn't something urgent, but perhaps it can be added to
>> the post-release TODO list.
>
> It seems that fontification takes long time.
> As far as I can remember the area fontification is introduced by me.
> Do you want to turn off the fortification?

Maybe it could be turned off for large files?
Or it could at least ask for confirmation to fontify a large buffer.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: hexl-find-file is sloooow

[Masatake YAMATO]

> >> hexl-find-file is quite slow -- loading a 4MByte font takes more than
> >> half a minute on my laptop...
> >> 
> >> I ask to add progress information while hexl-find-file is running.
> >> 
> >> Of course, this isn't something urgent, but perhaps it can be added to
> >> the post-release TODO list.
> >
> > It seems that fontification takes long time.
> > As far as I can remember the area fontification is introduced by me.
> > Do you want to turn off the fortification?
> 
> Maybe it could be turned off for large files?
> Or it could at least ask for confirmation to fontify a large buffer.

Thank you for your suggestion.
Before implementing your idea, I'd like to try my idea.

In the fontification code, `re-search-forward' is used:

    (save-excursion
      (goto-char (point-min))
      (while (re-search-forward "^[0-9a-f]+:" nil t)
	(put-text-property (match-beginning 0) (match-end 0)
			   'font-lock-face 'hexl-address-area))
      (goto-char (point-min))
      (while (re-search-forward "  \\(.+$\\)" nil t)
	(put-text-property (match-beginning 1) (match-end 1) 
			   'font-lock-face 'hexl-ascii-area)))

I think it is possible to do the same thing without `re-search-forward'.
If `re-search-forward' takes longer time than `put-text-property', 
the fontification code will be faster with my idea.

Masatake YAMATO

Re: hexl-find-file is sloooow

[Kim F. Storm]

Masatake YAMATO <jet@gyve.org> writes:

>> Maybe it could be turned off for large files?
>> Or it could at least ask for confirmation to fontify a large buffer.
>
> Thank you for your suggestion.
> Before implementing your idea, I'd like to try my idea.

But as a minimum, you could skip fontification if font-lock-mode
is not enabled in the buffer.

> I think it is possible to do the same thing without `re-search-forward'.
> If `re-search-forward' takes longer time than `put-text-property', 
> the fontification code will be faster with my idea.

But still (a lot) slower than without fontification.

I can easily distinguish fixed column output without extra decoration,
so if fontification takes more than 1-2 seconds, the benefit is too
marginal to justify the slowdown.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: hexl-find-file is sloooow

[Richard Stallman]

    It seems that fontification takes long time.
    As far as I can remember the area fontification is introduced by me.
    Do you want to turn off the fortification?

Could the fontification be made lazy?

Re: hexl-find-file is sloooow

[Masatake YAMATO]

>     It seems that fontification takes long time.
>     As far as I can remember the area fontification is introduced by me.
>     Do you want to turn off the fortification?
> 
> Could the fontification be made lazy?

I've introduced hexl-font-lock-keywords.
It becomes much faster.

However, I've met a trouble. When I do M-x hexl-find-file
on the file already loaded into emacs, hexl-font-lock-keywords
is not affected.

e.g.
	C-x C-f foo.c
	M-x hexl-find-file foo.c

Could you give me hints?

One more issue is that the regular expression used in
hexl-font-lock-keywords becomes ugly. I've given fixed column length
in it. Without fixed column length, it is difficult to fortify the
line which is not aliened to 16 bytes like:

  00132010: 0cc3 3096 9df3 59a4 327f a3fb 279d f7f7  ..0...Y.2...'...
                                                  ^Aliened. OK.
  00132020: 2500 a12a 00                             %..*.
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^Empty. Not aliened.

Masatake YAMATO

Index: hexl.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/hexl.el,v
retrieving revision 1.90
diff -u -r1.90 hexl.el
--- hexl.el	21 Nov 2004 00:38:34 -0000	1.90
+++ hexl.el	22 Dec 2004 06:39:52 -0000
@@ -111,11 +111,19 @@
 (defvar hexl-mode-old-isearch-search-fun-function)
 (defvar hexl-mode-old-require-final-newline)
 (defvar hexl-mode-old-syntax-table)
+(defvar hexl-mode-old-font-lock-keywords)
 
 (defvar hexl-ascii-overlay nil
   "Overlay used to highlight ASCII element corresponding to current point.")
 (make-variable-buffer-local 'hexl-ascii-overlay)
 
+(defvar hexl-font-lock-keywords
+  '(("^\\([0-9a-f]+:\\).\\{40\\}  \\(.+$\\)"
+     ;; "^\\([0-9a-f]+:\\).+  \\(.+$\\)"
+     (1 'hexl-address-area t t)
+     (2 'hexl-ascii-area t t)))
+  "Font lock keywords used in `hexl-mode'.")
+
 ;; routines
 
 (put 'hexl-mode 'mode-class 'special)
@@ -265,6 +273,11 @@
     (make-local-variable 'require-final-newline)
     (setq require-final-newline nil)
 
+    (make-local-variable 'hexl-mode-old-font-lock-keywords)
+    (setq hexl-mode-old-font-lock-keywords font-lock-defaults)
+    (make-local-variable 'font-lock-defaults)
+    (setq font-lock-defaults '(hexl-font-lock-keywords t))
+
     ;; Add hooks to rehexlify or dehexlify on various events.
     (add-hook 'after-revert-hook 'hexl-after-revert-hook nil t)
 
@@ -376,6 +389,7 @@
   (setq isearch-search-fun-function hexl-mode-old-isearch-search-fun-function)
   (use-local-map hexl-mode-old-local-map)
   (set-syntax-table hexl-mode-old-syntax-table)
+  (setq font-lock-defaults hexl-mode-old-font-lock-keywords)
   (setq major-mode hexl-mode-old-major-mode)
   (force-mode-line-update))
 
@@ -684,15 +698,6 @@
     (apply 'call-process-region (point-min) (point-max)
 	   (expand-file-name hexl-program exec-directory)
 	   t t nil (split-string hexl-options))
-    (save-excursion
-      (goto-char (point-min))
-      (while (re-search-forward "^[0-9a-f]+:" nil t)
-	(put-text-property (match-beginning 0) (match-end 0)
-			   'font-lock-face 'hexl-address-area))
-      (goto-char (point-min))
-      (while (re-search-forward "  \\(.+$\\)" nil t)
-	(put-text-property (match-beginning 1) (match-end 1) 
-			   'font-lock-face 'hexl-ascii-area)))
     (if (> (point) (hexl-address-to-marker hexl-max-address))
 	(hexl-goto-address hexl-max-address))))