unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Bisecting display bugs
@ 2016-07-10 23:55 Clément Pit--Claudel
  2016-07-11  1:40 ` Óscar Fuentes
  2016-07-11 14:22 ` Eli Zaretskii
  0 siblings, 2 replies; 14+ messages in thread
From: Clément Pit--Claudel @ 2016-07-10 23:55 UTC (permalink / raw)
  To: Emacs developers


[-- Attachment #1.1.1: Type: text/plain, Size: 539 bytes --]

Hey emacs-devel,

I've written up notes while bisecting a display bug (#23938) on master.  Since it didn't seem easy to identify the problem automatically from Elisp, I used a small `git bisect run' script that automatically captured, cropped, and compared screenshots of Emacs against a known-good reference.  All in all, things worked nicely: git bisect took 4 hours to complete, but it didn't require any interaction after starting.

I attached my notes.  Is there interest in including them in admin/notes/?

Cheers,
Clément.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Add-notes-about-using-git-bisect-to-pinpoint-Emacs-b.patch --]
[-- Type: text/x-diff; name="0001-Add-notes-about-using-git-bisect-to-pinpoint-Emacs-b.patch", Size: 5681 bytes --]

From 15196c30e37aa937b121abcd7d95d35e0b3350d8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 11 Jul 2016 01:47:01 +0200
Subject: [PATCH] Add notes about using `git bisect' to pinpoint Emacs bugs

* admin/notes/bisecting: New file.
* admin/notes/repo: Add pointer to admin/notes/bisecting.
---
 admin/notes/bisecting | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++
 admin/notes/repo      |   3 +-
 2 files changed, 106 insertions(+), 2 deletions(-)
 create mode 100644 admin/notes/bisecting

diff --git a/admin/notes/bisecting b/admin/notes/bisecting
new file mode 100644
index 0000000..4efb5cc
--- /dev/null
+++ b/admin/notes/bisecting
@@ -0,0 +1,105 @@
+HOW TO USE GIT BISECT TO PINPOINT EMACS BUGS  -*- outline -*-
+
+This documents explains how to use git bisect to find the commit that
+introduced an Emacs bug.  Bisecting works best if the bug is easy and
+relatively quick to reproduce in a clean ‘emacs -Q’ instance.
+
+For general information about bisecting, use ‘M-x man git-bisect’.
+This document contains quick-start instructions and Emacs-specific
+tips.
+
+* Bisecting Emacs bugs
+
+First, find a recipe that reproduces the bug in emacs -Q.  Then use
+this recipe to locate a revision in which the bug doesn't happen.  If
+it's a new bug, a not-too-recent revision could do; otherwise,
+checking for the bug previous releases of Emacs works well.
+
+Then, run the following command to start bisecting
+
+    git bisect start <buggy-revision> <good-revision>
+
+<buggy-revision> is usually HEAD in practice.
+
+Git will then check out revisions between these two bounds,; at each
+step, you need to run ‘git bisect good’ or ‘git bisect bad’ to inform
+git of the status of the current revision: ‘bad’ if it has the bug;
+good if it doesn't.
+
+If Emacs' build is broken in the current revision, use ‘git bisect
+skip’ to move to a neighboring revision.
+
+** Automating ‘git bisect’
+
+You can also write ‘git bisect run <some-shell-script>’ to automate
+the process.  ‘<some-shell-script>’ should build Emacs, run your bug
+reproduction test, and return 0 if the current revision is good, and 1
+otherwise.  Returning 125 is equivalent to doing ‘git bisect skip’.
+
+Concretely, ‘<some-shell-script>’ usually looks like this:
+
+    #!/usr/bin/env bash
+
+    # Remove leftovers from previous runs
+    git clean -xfd > /dev/null
+    # Build Emacs and skip commit if build fails
+    (./autogen.sh && ./configure --cache-file=/tmp/emacs.config.cache && make -j4) || exit 125
+
+    # Reproduce the bug, writing output somewhere
+    src/emacs -Q -l "../reproduce-the-bug.el" || exit 125
+    # Remove leftovers again
+    git clean -xfd > /dev/null
+    # Analyze output and produce exit code
+    cmp ../reference-output current-output
+
+Some caveats:
+
+- This script cleans Emacs' source directory with ‘git clean -xfd’, so
+  make sure your uncommitted changes are saved somewhere else.
+
+- You should produce the ‘../reproduce-your-bug.el’ script on your own
+  (it should check if the bug exists, and save a different output to a
+  file named ‘current-output’ based on the result of that check), as
+  well as a file ‘../reference-output’ containing the expected output
+  in the good case.
+
+** Using ‘git bisect’ to find display-related bugs
+
+Most bugs that manifest graphically can be checked for by
+programmatically inspecting text properties, but some are only
+apparent through visual inspection.  Since building Emacs takes a long
+time, it can be a pain to debug these manually.
+
+Fortunately, it's relatively easy to bisect such bugs automatically.
+Use the following template for ‘../reproduce-the-bug.el’:
+
+    (defun take-screenshot-and-exit (fname x y w h)
+      "Save a screenshot of Emacs as FNAME, then exit.
+    X and Y are the coordinates of the top-left point of the area of interest.
+    W, and H are its dimensions."
+      (let ((window-id (frame-parameter nil 'outer-window-id)))
+        (call-process "xwd" nil nil nil "-silent" "-id" window-id "-out" fname))
+      (call-process "mogrify" nil nil nil fname "-crop" (format "%dx%d+%d+%d" w h x y))
+      (kill-emacs))
+
+    (defun main ()
+      ;; Reproduce your bug here
+      …
+      ;; Force a redisplay
+      (redisplay t)
+      ;; Insert rough X, Y, W, H values below
+      (run-with-timer 0 nil #'take-screenshot-and-exit "screenshot.xwd" … … … …))
+
+    (main)
+
+This script makes a screenshot of Emacs after reproducing your bug (if
+‘xwd’ isn't available, you can use ImageMagick's ‘import’ tool,
+passing it a ‘-window’ argument where ‘xwd’ wants ‘id’).  Cropping the
+image is useful to weed out unrelated display changes; try to include
+only a small portion of the screen containing your bug.
+
+Then to produce the right exit code use ImageMagick's ‘identify’ tool:
+
+    cmp <(identify -quiet -format "%#" "../screenshot.xwd") <(identify -quiet -format "%#" "../good.xwd")
+
+‘good.xwd’ should be the image produced by your script in the good state.
diff --git a/admin/notes/repo b/admin/notes/repo
index 3ab3da7..60373bd 100644
--- a/admin/notes/repo
+++ b/admin/notes/repo
@@ -114,8 +114,7 @@ again.
 
 * Bisecting
 
-This is a semi-automated way to find the revision that introduced a bug.
-Browse 'git help bisect' for technical instructions.
+See admin/notes/bisecting.
 
 * Maintaining ChangeLog history
 
-- 
2.9.0


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply related	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2017-04-17 13:43 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-07-10 23:55 Bisecting display bugs Clément Pit--Claudel
2016-07-11  1:40 ` Óscar Fuentes
2016-07-11 14:22 ` Eli Zaretskii
2016-07-11 17:46   ` Óscar Fuentes
2016-07-11 17:54     ` Eli Zaretskii
2016-07-11 18:05       ` Óscar Fuentes
2016-07-22 18:55   ` Clément Pit--Claudel
2016-07-23  8:51     ` Eli Zaretskii
2016-07-23 13:22       ` Clément Pit--Claudel
2017-04-16 15:04         ` Clément Pit--Claudel
2017-04-16 15:32           ` Noam Postavsky
2017-04-16 20:57             ` Clément Pit-Claudel
2017-04-17 13:13               ` Herring, Davis
2017-04-17 13:43                 ` Teemu Likonen

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).