config problems and bisecting an org-mode configuration file

config problems and bisecting an org-mode configuration file

[Sharon Kimble]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]


I have three problems with my config at the moment -

- an inability to close emacs using its internal commands, and
- an inability to right-click on an url in gnus to either open that url
  in emacs-w3m, or copy it to the clipboard for use in another browser, and
- an inability to use +foo+ to strike-through something to effectively
  show that its commented out.

I know the theory behind 'bisecting my init file', except mine is held
in an org-mode file (called 'config.org') which is called by my init.el
which states -

--8<---------------cut here---------------start------------->8---
(package-initialize)
(require 'ob-tangle)
(org-babel-load-file "~/.emacs.d/config.org")

;; Don't load old .elc files when the .el file is newer
(setq load-prefer-newer t)
--8<---------------cut here---------------end--------------->8---

So do I bisect my 'config.org', or the 'config.el' please?

'config.org' is 570.7 kb, and its tangled config.el is 440.0 kb.

- 'config.org' has 17773 lines of active code (active means everything
  that is to be tangled)
- 'config.el' has 12024 lines of code.

So, which do I bisect please? And any hints/ideas which could make it
simpler/easier to do please?

Its further complicated by me using Debian which sets its own startup files before emacs is starting to load my config, so ideally I'd like to not use them at all.

I've run 'emacs -Q' which loaded and then was able to be closed via its internal closing command.
I've run 'emacs --debug-init' which loaded correctly but then wasn't able to be closed via the internal command.
I've run 'M-x bug-hunter-file' on 'config.el' which showed zero problems.

So all in all, its looking like something is wrong with my config, and
it needs to be sorted out this weekend, so any ideas please folks?

Thanks
Sharon.
-- 
A taste of linux = http://www.sharons.org.uk
TGmeds = http://www.tgmeds.org.uk
DrugFacts = https://www.drugfacts.org.uk  
Debian 9.0, fluxbox 1.3.5-2, emacs 25.1.1, org-mode 9.0.7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Sharon Kimble wrote:

> I have three problems with my config at the
> moment -
>
> - an inability to close emacs using its
> internal commands

What do you mean ... ? Because you have Gnus
running and it won't save activity?
Or something else?

    (defun emacs-quit-no-confirm ()
      (interactive)
      (when (gnus-alive-p) (gnus-group-exit))
      (save-buffers-kill-terminal t) ) ; silently save all

> and - an inability to
> right-click on an url in gnus to either open
> that url in emacs-w3m

Do you mean URL:s that for some reasons are
not buttons?

> or copy it to the clipboard for use in
> another browser

If it is a literal URL, you can do

    (defun kill-url ()
      (interactive)
      (kill-new (thing-at-point 'url)) )
    )

> and - an inability to use +foo+ to
> strike-through something to effectively show
> that its commented out.

You want a word that is enclosed within plus
signs to take on a specific face? You can do

    (font-lock-add-keywords 'emacs-lisp-mode
      '(
        ("\\+.*\\+" . font-lock-comment-face)
        )
      t) ; APPEND

(Change mode and face to your liking.)

> it needs to be sorted out this weekend

Oh, no! Get to work everyone :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Sharon Kimble]

[-- Attachment #1: Type: text/plain, Size: 2250 bytes --]

Emanuel Berg <moasen@zoho.com> writes:

> Sharon Kimble wrote:
>
>> I have three problems with my config at the
>> moment -
>>
>> - an inability to close emacs using its
>> internal commands
>
> What do you mean ... ? Because you have Gnus
> running and it won't save activity?
> Or something else?
>
>     (defun emacs-quit-no-confirm ()
>       (interactive)
>       (when (gnus-alive-p) (gnus-group-exit))
>       (save-buffers-kill-terminal t) ) ; silently save all

I meant that I was unable to close emacs using its internal commands of
'C-x C-c' and had to kill it aggressively by either its PID or pkill.

>
>> and - an inability to
>> right-click on an url in gnus to either open
>> that url in emacs-w3m
>
> Do you mean URL:s that for some reasons are
> not buttons?

Hyperlinks in gnus emails.

>
>> or copy it to the clipboard for use in
>> another browser
>
> If it is a literal URL, you can do
>
>     (defun kill-url ()
>       (interactive)
>       (kill-new (thing-at-point 'url)) )
>     )
>
>> and - an inability to use +foo+ to
>> strike-through something to effectively show
>> that its commented out.

In org-mode if you put +foo+ then foo has been strike-throughed, and I
had lost this ability.

>
> You want a word that is enclosed within plus
> signs to take on a specific face? You can do
>
>     (font-lock-add-keywords 'emacs-lisp-mode
>       '(
>         ("\\+.*\\+" . font-lock-comment-face)
>         )
>       t) ; APPEND
>
> (Change mode and face to your liking.)
>
>> it needs to be sorted out this weekend
>
> Oh, no! Get to work everyone :)

Bwwwwrrrp!!! (That's a raspberry for you! :)

It was an admonition for me that I *needed* to get this sorted this
weekend. And I have almost done.

- Inability to close properly - WORKS
- inability to use org-mode strike-through by using multiple +'s - WORKS
- Ability to right-click a hyperlink - not sorted. It seems to be a
  problem with avy-menu, so I've raised a bug on their site.

Thanks
Sharon.
-- 
A taste of linux = http://www.sharons.org.uk
TGmeds = http://www.tgmeds.org.uk
DrugFacts = https://www.drugfacts.org.uk  
Debian 9.0, fluxbox 1.3.5-2, emacs 25.1.1, org-mode 9.0.7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Sharon Kimble wrote:

> I meant that I was unable to close emacs using
> its internal commands of 'C-x C-c' and had to
> kill it aggressively by either its PID
> or pkill. [...]
> Hyperlinks in gnus emails. [...]
> In org-mode if you put +foo+ then foo has been
> strike-throughed, and I had lost
> this ability.

OK, so all of these problems are caused by your
own configurations? Man, you must really be an
aggressive Elisper :) Well, good luck working it
out.

Try to identify areas in your source that seem
suspicious and then comment them out and
restart.

Either that, ot comment out half your code.
If it resolves a problem, you know it is in
that half; and if it doesn't, then it is in the
other :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Sharon Kimble]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

Emanuel Berg <moasen@zoho.com> writes:

> Sharon Kimble wrote:
>
>> I meant that I was unable to close emacs using
>> its internal commands of 'C-x C-c' and had to
>> kill it aggressively by either its PID
>> or pkill. [...]
>> Hyperlinks in gnus emails. [...]
>> In org-mode if you put +foo+ then foo has been
>> strike-throughed, and I had lost
>> this ability.
>
> OK, so all of these problems are caused by your
> own configurations? Man, you must really be an
> aggressive Elisper :) Well, good luck working it
> out.
>
> Try to identify areas in your source that seem
> suspicious and then comment them out and
> restart.

All done and sorted, the first two problems were in my custom.el and
have been resolved, and the last problem was because I had
ace-popup-menu and avy-menu installed. After email conversations with
the writer of both programmes, I've deleted ace-popup-menu and I've now
got my right-click back again. So all is now working properly.

>
> Either that, ot comment out half your code.
> If it resolves a problem, you know it is in
> that half; and if it doesn't, then it is in the
> other :)

And that's what I did, with very good support form Drew giving me very
explicit instructions which helped resolve the problem by showing me,
eventually, where the problem was. All kudos to Drew!

Thanks
Sharon.
-- 
A taste of linux = http://www.sharons.org.uk
TGmeds = http://www.tgmeds.org.uk
DrugFacts = https://www.drugfacts.org.uk  
Debian 9.0, fluxbox 1.3.5-2, emacs 25.1.1, org-mode 9.0.7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Sharon Kimble wrote:

> And that's what I did, with very good support
> form Drew giving me very explicit
> instructions which helped resolve the problem
> by showing me, eventually, where the problem
> was. All kudos to Drew!

This is his favorite method. I think he even
got a name for it - "binary search", right?

While there is no denying its advantages, with
your situation, it seems you have a lot of
configs and probably you should make the code
more modular, so changes in one part is easy to
pinpoint and have less or no impact on
everything else.

If you are using a module, say module M, you
can create a file for configs for that module
alone, say my-M.el, which has (require 'M) as
the first line, and then load that file from
.emacs.

Incidentally, this doesn't prevent binary
search, tho - it won't be perfectly binary
anymore :) - as you can then just comment out
the line(s) in .emacs that loads the module(s).

If you byte-compile the code, the compiler will
help you determine what code interferes with
what other, and you can remedy that by moving
around stuff and adding more pairs of `require'
and `provide'.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Robert Thorpe]

When Emacs configs are small they're reasonably similar to configurations for
other programs.

However, when they grow they are essentially like programs.  They're
software development problems in their own right.

In that case I think it's useful to apply the methods of large-scale
software development.  Version control, for example.

Perhaps you do that though.

BR,
Robert Thorpe

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Robert Thorpe wrote:

> When Emacs configs are small they're
> reasonably similar to configurations for
> other programs.
>
> However, when they grow they are essentially
> like programs. They're software development
> problems in their own right.

Yeah, I'm mean there is a silly perceived
hierarchy to it. One guy makes a lousy PC-game
no one ever plays and he makes a living out of
it, another guy writes Elisp and is rich at
heart... At some point you realize it doesn't
really matter, just live and do whatever
you want.

> In that case I think it's useful to apply the
> methods of large-scale software development.
> Version control, for example.
>
> Perhaps you do that though.

I don't think Sharon does that but she could,
only from what we've heard it sounds more like
everything is tangled up. If so, rather that
continue with it until it breaks and then roll
back to a sound state and start pushing all
over again, I'd put my investment into breaking
it up into neat units of code so that for
example configuring org-mode won't break Gnus
article mode and even basic
Emacs functionality!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Sharon Kimble]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> When Emacs configs are small they're reasonably similar to configurations for
> other programs.
>
> However, when they grow they are essentially like programs.  They're
> software development problems in their own right.
>
> In that case I think it's useful to apply the methods of large-scale
> software development.  Version control, for example.
>
> Perhaps you do that though.

Not quite in version control, although its backed up hourly, but as my
config is in org-mode every block of code is timestamped when it was
inserted or changed, so it is possible to navigate to specific modules
either with Imenu, or the date.

Thanks
Sharon.
-- 
A taste of linux = http://www.sharons.org.uk
TGmeds = http://www.tgmeds.org.uk
DrugFacts = https://www.drugfacts.org.uk  
Debian 9.0, fluxbox 1.3.5-2, emacs 25.1.1, org-mode 9.0.7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: config problems and bisecting an org-mode configuration file

[Sharon Kimble]

[-- Attachment #1: Type: text/plain, Size: 1906 bytes --]

Emanuel Berg <moasen@zoho.com> writes:

> Sharon Kimble wrote:
>
>> And that's what I did, with very good support
>> form Drew giving me very explicit
>> instructions which helped resolve the problem
>> by showing me, eventually, where the problem
>> was. All kudos to Drew!
>
> This is his favorite method. I think he even
> got a name for it - "binary search", right?
>
> While there is no denying its advantages, with
> your situation, it seems you have a lot of
> configs and probably you should make the code
> more modular, so changes in one part is easy to
> pinpoint and have less or no impact on
> everything else.
>
> If you are using a module, say module M, you
> can create a file for configs for that module
> alone, say my-M.el, which has (require 'M) as
> the first line, and then load that file from
> .emacs.

My config is an org-mode file, with almost all code blocks in their own
module, plus I'm also timestamping each module so its also possible to
navigate using either Imenu or my date-stamps.

> Incidentally, this doesn't prevent binary
> search, tho - it won't be perfectly binary
> anymore :) - as you can then just comment out
> the line(s) in .emacs that loads the module(s).
>
> If you byte-compile the code, the compiler will
> help you determine what code interferes with
> what other, and you can remedy that by moving
> around stuff and adding more pairs of `require'
> and `provide'.

Every change to my config.org and the file is auto-compiled so that when
I restart emacs I don't have to wait for config.el to be generated.
Sometimes its a bit tiresome waiting for it to be generated but overall
it works very well.

Thanks
Sharon.
-- 
A taste of linux = http://www.sharons.org.uk
TGmeds = http://www.tgmeds.org.uk
DrugFacts = https://www.drugfacts.org.uk  
Debian 9.0, fluxbox 1.3.5-2, emacs 25.1.1, org-mode 9.0.7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Sharon Kimble wrote:

> Every change to my config.org and the file is
> auto-compiled so that when I restart emacs
> I don't have to wait for config.el to be
> generated. Sometimes its a bit tiresome
> waiting for it to be generated but overall it
> works very well.

auto-compile, what exactly is that? Is is
compilation whenever you save?

It sounds a bit overkill since compilation is
mostly an optimization thing (and a way to get
suggestions how to improve the code), and
because changes to the code are most often
small, it would seem to be enough to have the
big body compiled after the change but put into
effect only upon restart, and what has changed
and needs to be changed for the session just
re-evaluated to immediate gain (e.g., the defun
that has been modified).

Use `C-x C-e' which is `eval-last-sexp' or, if
you feel the need to re-evaluate the entire
file

    (defun load-this-file ()
      (interactive)
      (load-file (buffer-file-name)) )

You can also do

    M-x load-file RET RET

to load the current file.

That said, even auto-compile on save should not
take a lot of time! Some of my Elisp files are
moderately long (the longest is 376 lines), and
when I do byte-compile in the zhell after
changing this and only this file, it takes only
2.581 seconds!

Because most of my other files are much shorter,
byte-compile is virtually instantaneous.

Even if we make room for a small overhead when
it is done with auto-compile, it should be
*fast*. So again, keep your code in different
files not exceeding say 100 lines, and it
should be fast enough. (There are exceptions of
course when the file includes tons of
documentation or if the code is of a "list"
nature, e.g. an all but endless list of keys or
faces or ...)

Try this in your config dir

   for f in *.el; do wc -l $f; done | sort -n -r

What are your top results?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Emanuel Berg wrote:

> Try this in your config dir
>
>    for f in *.el; do wc -l $f; done | sort -n
> -r
>
> What are your top results?

Here is a more neat shell function with AWK
that'll compute the mean value

    elisp-mean () {
        for f in *.el; do
            wc -l $f
        done | awk '{ sum += $1 } END { print int(sum/NR) + 1 }'
    }

After attempting to get it to work for an
embarrassingly long time, now I now mine is 55!

:)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Here is an even better way I found on the web
(see commented-out URL) with some very minor
changes. Do

    elisp-lines | numbers

I have

    items  sum   low  high  median  mean
    71     3891  6    376   41      55

:)

elisp-lines () {
    for f in *.el; do
        wc -l $f
    done
}

numbers () {
    # thank you: https://unix.stackexchange.com/a/13779
    echo "items\tsum\tlow\thigh\tmedian\tmean"
    sort -n | awk '
          BEGIN {
            c = 0;
            sum = 0;
          }
          $1 ~ /^[0-9]*(\.[0-9]*)?$/ {
            a[c++] = $1;
            sum += $1;
          }
          END {
            ave = int(sum/c) + 1;
            if ( (c % 2) == 1 ) {
              median = a[ int(c/2) ];
            } else {
              median = ( a[c/2] + a[c/2-1] ) / 2;
            }
            OFS="\t";
            print c, sum, a[0], a[c-1], median, ave
          }
        '
}

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: config problems and bisecting an org-mode configuration file

[Emanuel Berg]

Emanuel Berg wrote:

>     items sum low high median mean 71 3891 6
> 376 41 55

Oups! I forgot I have some stuff in
directories. My code is more modular than even
I thought! Then I have

    items	sum	low	high	median	mean
    97	5604	6	376	42	58

(Hm - the tabs doesn't seem to be interpreted
consistently in the Emacs shell. It looks good
in my VT with tmux and zsh tho.)

Here is how I altered the zsh to do a recursive
search

    elisp-lines () {
        for f in **/*.el; do
            wc -l $f
        done
    }

It is the extra **/ on the second line. It is
a zsh feature - perhaps it has made its way to
bash by now? - figure it out, or
"debruiez-vous" (?) or whatever they say in the
FFL...

-- 
underground experts united
http://user.it.uu.se/~embe8573