bug#27748: 26.0.50; doc strings should be in DOC file

[Ken Raeburn <raeburn@raeburn.org>]

There are a bunch of doc strings from the preloaded Lisp files that do not wind up in the DOC file.  Presumably this means they wind up in the emacs executable image itself, saved away as extra string objects that GC needs to track.  (In the big-elc-file work Stefan started and I'm experimenting with, these doc strings wind up in the dumped Lisp environment file, and need to be parsed and saved away at load time.)

1. defcustom doc strings from files compiled with lexical binding.

   For example, files.el (lexical bindings) defines
   delete-auto-save-files but it doesn't show up in the DOC file;
   files.elc starts with an initial byte-code blob which includes the
   symbol delete-auto-save-files and its doc string in the constants
   array.

   On the other hand, custom.el (dynamic bindings) declares
   custom-theme-directory, the .elc file dumps out the doc string in a
   #@... block before a separate top-level call to
   custom-declare-variable, and since this is what make-docfile looks
   for, the doc string winds up in the DOC file.

2. In isearch, CL macro expansion results in symbols like
   isearch--state-forward--cmacro having function definitions that
   include the two doc strings "Access slot \"forward\" of
   `(isearch--state (:constructor nil) (:copier nil) (:constructor
   isearch--get-state (&aux (string isearch-string) (message
   isearch-message) (point (point)) (success isearch-success) (forward
   isearch-forward) (other-end isearch-other-end) (word
   isearch-regexp-function) (error isearch-error) (wrapped
   isearch-wrapped) (barrier isearch-barrier) (case-fold-search
   isearch-case-fold-search) (pop-fun (if isearch-push-state-function
   (funcall isearch-push-state-function))))))' struct CL-X." and
   "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC (with
   "\n\n(fn CL-X)" appended) as the documentation for the function
   isearch--state-forward.

   It appears that, as far as the Emacs help system is concerned,
   isearch--state-*--cmacro functions are undocumented.

   The --cmacro functions generate cl-block forms that include the
   original doc string, since it's treated as part of the body, but it's
   not clear to me whether it has any use there at all, or if it could
   just be discarded, or perhaps looked up via the non --cmacro symbols
   at run time if it is of use.

3. Undocumented functions, strange as it sounds... in files.el, function
   file-name-non-special is defined with no documentation.  In
   files.elc, the bytecode object constructed has a doc string "\n\n(fn
   OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
   separate string that make-docfile can pick up.

To “reproduce”…

Delete or “chmod 0” the DOC file before starting Emacs.  Use ielm to look at the symbol property lists of delete-auto-save-files and custom-theme-directory; the former has a string for its variable-documentation property, and the latter has a number.  Look at the function definitions of isearch--state-forward--cmacro and file-name-non-special and see the doc strings in them.  Examine the .elc files and DOC.






In GNU Emacs 26.0.50 (build 2, x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G1217))
 of 2017-07-18 built on bang.local
Repository revision: 0083123499cc29e301c197218d3809b225675e57
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
Loading ~/lib/elisp/sue.elc...done
Loading desktop...done
Warning: desktop file appears to be in use by PID 83193.
Using it may cause conflicts.  Use it anyway? (y or n) n
Desktop file in use; not loaded.
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured features:
RSVG IMAGEMAGICK DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  desktop-save-mode: t
  global-hi-lock-mode: t
  hi-lock-mode: t
  which-function-mode: t
  icomplete-mode: t
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr warnings emacsbug message subr-x puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader add-log desktop frameset cus-start
cus-load kr-defs hi-lock which-func imenu icomplete iso-transl
smart-quotes easy-mmode tramp tramp-compat tramp-loaddefs trampver shell
pcomplete comint ansi-color ring parse-time format-spec advice cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs server time sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils finder-inf package easymenu epg-config
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib time-date tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 261240 12410)
 (symbols 48 26239 2)
 (miscs 40 57 285)
 (strings 32 47062 2197)
 (string-bytes 1 1448035)
 (vectors 16 42658)
 (vector-slots 8 784892 12266)
 (floats 8 70 90)
 (intervals 56 241 0)
 (buffers 992 12))