From: "Clément Pit--Claudel" <clement.pit@gmail.com>
To: Emacs developers <emacs-devel@gnu.org>
Subject: Bisecting display bugs
Date: Mon, 11 Jul 2016 01:55:31 +0200 [thread overview]
Message-ID: <5782E073.6000505@gmail.com> (raw)
[-- Attachment #1.1.1: Type: text/plain, Size: 539 bytes --]
Hey emacs-devel,
I've written up notes while bisecting a display bug (#23938) on master. Since it didn't seem easy to identify the problem automatically from Elisp, I used a small `git bisect run' script that automatically captured, cropped, and compared screenshots of Emacs against a known-good reference. All in all, things worked nicely: git bisect took 4 hours to complete, but it didn't require any interaction after starting.
I attached my notes. Is there interest in including them in admin/notes/?
Cheers,
Clément.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: 0001-Add-notes-about-using-git-bisect-to-pinpoint-Emacs-b.patch --]
[-- Type: text/x-diff; name="0001-Add-notes-about-using-git-bisect-to-pinpoint-Emacs-b.patch", Size: 5681 bytes --]
From 15196c30e37aa937b121abcd7d95d35e0b3350d8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20Pit--Claudel?= <clement.pitclaudel@live.com>
Date: Mon, 11 Jul 2016 01:47:01 +0200
Subject: [PATCH] Add notes about using `git bisect' to pinpoint Emacs bugs
* admin/notes/bisecting: New file.
* admin/notes/repo: Add pointer to admin/notes/bisecting.
---
admin/notes/bisecting | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++
admin/notes/repo | 3 +-
2 files changed, 106 insertions(+), 2 deletions(-)
create mode 100644 admin/notes/bisecting
diff --git a/admin/notes/bisecting b/admin/notes/bisecting
new file mode 100644
index 0000000..4efb5cc
--- /dev/null
+++ b/admin/notes/bisecting
@@ -0,0 +1,105 @@
+HOW TO USE GIT BISECT TO PINPOINT EMACS BUGS -*- outline -*-
+
+This documents explains how to use git bisect to find the commit that
+introduced an Emacs bug. Bisecting works best if the bug is easy and
+relatively quick to reproduce in a clean ‘emacs -Q’ instance.
+
+For general information about bisecting, use ‘M-x man git-bisect’.
+This document contains quick-start instructions and Emacs-specific
+tips.
+
+* Bisecting Emacs bugs
+
+First, find a recipe that reproduces the bug in emacs -Q. Then use
+this recipe to locate a revision in which the bug doesn't happen. If
+it's a new bug, a not-too-recent revision could do; otherwise,
+checking for the bug previous releases of Emacs works well.
+
+Then, run the following command to start bisecting
+
+ git bisect start <buggy-revision> <good-revision>
+
+<buggy-revision> is usually HEAD in practice.
+
+Git will then check out revisions between these two bounds,; at each
+step, you need to run ‘git bisect good’ or ‘git bisect bad’ to inform
+git of the status of the current revision: ‘bad’ if it has the bug;
+good if it doesn't.
+
+If Emacs' build is broken in the current revision, use ‘git bisect
+skip’ to move to a neighboring revision.
+
+** Automating ‘git bisect’
+
+You can also write ‘git bisect run <some-shell-script>’ to automate
+the process. ‘<some-shell-script>’ should build Emacs, run your bug
+reproduction test, and return 0 if the current revision is good, and 1
+otherwise. Returning 125 is equivalent to doing ‘git bisect skip’.
+
+Concretely, ‘<some-shell-script>’ usually looks like this:
+
+ #!/usr/bin/env bash
+
+ # Remove leftovers from previous runs
+ git clean -xfd > /dev/null
+ # Build Emacs and skip commit if build fails
+ (./autogen.sh && ./configure --cache-file=/tmp/emacs.config.cache && make -j4) || exit 125
+
+ # Reproduce the bug, writing output somewhere
+ src/emacs -Q -l "../reproduce-the-bug.el" || exit 125
+ # Remove leftovers again
+ git clean -xfd > /dev/null
+ # Analyze output and produce exit code
+ cmp ../reference-output current-output
+
+Some caveats:
+
+- This script cleans Emacs' source directory with ‘git clean -xfd’, so
+ make sure your uncommitted changes are saved somewhere else.
+
+- You should produce the ‘../reproduce-your-bug.el’ script on your own
+ (it should check if the bug exists, and save a different output to a
+ file named ‘current-output’ based on the result of that check), as
+ well as a file ‘../reference-output’ containing the expected output
+ in the good case.
+
+** Using ‘git bisect’ to find display-related bugs
+
+Most bugs that manifest graphically can be checked for by
+programmatically inspecting text properties, but some are only
+apparent through visual inspection. Since building Emacs takes a long
+time, it can be a pain to debug these manually.
+
+Fortunately, it's relatively easy to bisect such bugs automatically.
+Use the following template for ‘../reproduce-the-bug.el’:
+
+ (defun take-screenshot-and-exit (fname x y w h)
+ "Save a screenshot of Emacs as FNAME, then exit.
+ X and Y are the coordinates of the top-left point of the area of interest.
+ W, and H are its dimensions."
+ (let ((window-id (frame-parameter nil 'outer-window-id)))
+ (call-process "xwd" nil nil nil "-silent" "-id" window-id "-out" fname))
+ (call-process "mogrify" nil nil nil fname "-crop" (format "%dx%d+%d+%d" w h x y))
+ (kill-emacs))
+
+ (defun main ()
+ ;; Reproduce your bug here
+ …
+ ;; Force a redisplay
+ (redisplay t)
+ ;; Insert rough X, Y, W, H values below
+ (run-with-timer 0 nil #'take-screenshot-and-exit "screenshot.xwd" … … … …))
+
+ (main)
+
+This script makes a screenshot of Emacs after reproducing your bug (if
+‘xwd’ isn't available, you can use ImageMagick's ‘import’ tool,
+passing it a ‘-window’ argument where ‘xwd’ wants ‘id’). Cropping the
+image is useful to weed out unrelated display changes; try to include
+only a small portion of the screen containing your bug.
+
+Then to produce the right exit code use ImageMagick's ‘identify’ tool:
+
+ cmp <(identify -quiet -format "%#" "../screenshot.xwd") <(identify -quiet -format "%#" "../good.xwd")
+
+‘good.xwd’ should be the image produced by your script in the good state.
diff --git a/admin/notes/repo b/admin/notes/repo
index 3ab3da7..60373bd 100644
--- a/admin/notes/repo
+++ b/admin/notes/repo
@@ -114,8 +114,7 @@ again.
* Bisecting
-This is a semi-automated way to find the revision that introduced a bug.
-Browse 'git help bisect' for technical instructions.
+See admin/notes/bisecting.
* Maintaining ChangeLog history
--
2.9.0
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
next reply other threads:[~2016-07-10 23:55 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-10 23:55 Clément Pit--Claudel [this message]
2016-07-11 1:40 ` Bisecting display bugs Óscar Fuentes
2016-07-11 14:22 ` Eli Zaretskii
2016-07-11 17:46 ` Óscar Fuentes
2016-07-11 17:54 ` Eli Zaretskii
2016-07-11 18:05 ` Óscar Fuentes
2016-07-22 18:55 ` Clément Pit--Claudel
2016-07-23 8:51 ` Eli Zaretskii
2016-07-23 13:22 ` Clément Pit--Claudel
2017-04-16 15:04 ` Clément Pit--Claudel
2017-04-16 15:32 ` Noam Postavsky
2017-04-16 20:57 ` Clément Pit-Claudel
2017-04-17 13:13 ` Herring, Davis
2017-04-17 13:43 ` Teemu Likonen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5782E073.6000505@gmail.com \
--to=clement.pit@gmail.com \
--cc=emacs-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).