org-mode and 'intangible text property

org-mode and 'intangible text property

[ishikawa]

 org-mode and 'intangible text property

 Hello,

 I have a question about org-mode.el.
 (sorry for the lengthy posting, I wanted to list the
 background of the issue in one e-mail.)

 I am a user of Emacs over the years (20+ years).
 I have been using outline mode and have come to realize that org-mode
 seems to offer additional benefits by reading blogs and others.

 However, when I tried to use org-mode, I hit a snug.

 I tried to input Japanese string using a Japanese input system written
 in Emacs Lisp called tamago/egg.  But when I tried to type in, say, a
 first-level heading in org-mode, the string completely disappears (not
 even a trace of it) after I hit the return (to finalize the Japanese
 input.)

 The cause of the problem seems to be as follows:
 The use of 'intangible property of org-mode collided with the
 use of 'intangible property of the Japanese input method
 implemented in Emacs Lisp.

 Basically, the problem is that org-mode removes the 'intangible
 property set by egg/tamago Japanese input system without
 discrimination at all is what someone wrote in his blog. (org-mode
 seems to use 'intangible internally.)

 The Japanese input system, egg/tamago, was a rather popular system in
 late 1980 and 1990's. It now sees many competitive Japanese input
 systems, but there are emacs die hards who can't live without it when
 it comes to typing Japanese into Emacs buffer.

 Although egg/tamago is in strictly maintenance mode, it is difficult
 for me (and others who have grown up to use this input system
 figuratively speaking) to ditch this egg/tamago input system if I can
 avoid it.
 [No major update has been done in the last 4-5 years to tamago/egg and
 its major web site is now torn down. Only the packages that are in the
 major linux distributions, FreeBSD, NetBSD packaging systems are more
 or less used by many, with some patches floating around. For example,
 when emacs 23.4 appeared, finally we have to replace or use defalias
 to fix the missing last-command-char variable in egg/tamago package.
 There *ARE* people willing to maintain egg/tamago to keep it alive in 
 the face of such
 issues for sure, mine included.]

 On the other hand, if org-mode is a great replacement for
 outline-mode, then I would certainly like to try it out
 (and still use egg/tamago for input if possible.)


 Org mode seems to use 'intangible text property one way or the other.
 igrep mode search within org subdirectory of lisp directory of emacs
 24.3 turned up the following hits (shown at the end of this e-mail).

 One of the macro's found in  org-mac.el is as follows. : 
 org-maybe-intangible
 This is quoted a few lines below.

 The document string for this macro suggests that org-mode had an issue
 with flyspell.el due to the intangible text property use. But now the
 particular usage seems to be avoided by the introduction of macros.

 ;; FIXME: Slated for removal.  Current Org mode does not support Emacs 
 < 22
 (defmacro org-maybe-intangible (props)
   "Add '(intangible t) to PROPS if Emacs version is earlier than Emacs 
 22.
 In Emacs 21, invisible text is not avoided by the command loop, so the
 intangible property is needed to make sure point skips this text.
 In Emacs 22, this is not necessary.  The intangible text property has
 led to problems with flyspell.  These problems are fixed in 
 flyspell.el,
 but we still avoid setting the property in Emacs 22 and later.
 We use a macro so that the test can happen at compilation time."
   (if (< emacs-major-version 22)
       `(append '(intangible t) ,props)
     props))


 HOWEVER there are still about a dozen references to
 'intangible property in org-mode emacs-lisp code.

 I wonder if such avoidance as in the case of flyspell.el can be made
 for the conflict of usage of 'intangible with egg/tamago input routine 
 written in Emacs-Lisp.

 Someone in Japan who faced this incompatibility problem of org-mode
 and egg/tamago input method stated that he (I assume he) substituted
 all the references to "intangible" in org-mode elisp code with
 "org-intangible". Now, as far as he is concerned, the Japanese input 
 now
 works, and he has not bumped into a case where cursor moves into an
 awkward (or should not happen) position in org-mode (which should have
 been avoided by the use of 'intangible property in org-mode).

 His drastic surgery may be the last straw if no alternative is
 available.  (His blog in Japanese is available here:
 http://d.hatena.ne.jp/grandVin/20090129/1233226833
 (dated January 2009)

 Also, it is quoted in an independent someone's emacs wiki written in 
 Japanese
 prominently.  http://www.tsukune.org/skk/memo/index.php?emacs#n27db692

 At least you can see there are Japanese users who are bitten by the
 problem and want to solve it somehow.

 Does anyone in the know can evaluate the perspective of removing the
 references to 'intangible in the org code?  If, due to the conflict
 with flyspell.el, some usages of intangible could be removed, can some
 remaining references be removed, too?  (Well, I am not sure of
 this. Probably intangible is used to make sure that the cursor does
 not move, say, into a next cell in a table.)

 OR, more important to me, can someone suggest a manner how a 
 conflicting use of
 'intangible can be somehow managed?

 Thank you for reading so far.

 I am outlining a rough idea how this could be handled. But maybe I am 
 wrong.

 Any tips will be appreciated.

 TIA


 PS:  my idea is to use defadvise
 function to redefine the meanings of the following functions both in
 org-mode and when egg/tamago Japanese input is in effect,

 - set-text-properties
 - remove-text-properties
 - add-text-properties (may not be necessary according to the usage 
 pattern.)
 - get-text-property
 - put-text-property

 These functions are used to set/add or reference the text property in
 org-mode (and egg/tamago Japanese input method.)

 My idea is roughly as follows:

 Let us keep the both forms of 'intangible property PLUS the value pair 
 for
 org-mode and egg/tamago as part of the text property, and
 set/remove and return only the property set in the same mode.  Let us 
 use
 org-mode-intangible and egg-mode-intangible for these internal forms in 
 the explanation below.

 For example, when remove-text-properties is called and 'intangible is
 about to be removed, and if it is called within org-mode, delete the
 intangible property for org-mode, i.e., org-mode-intangible becomes
 nil.  if it is called from egg/tamago input mode, intangible property
 for egg-mode, i.e., egg-mode-intangible becomes nil.
 (If it is called outside these modules, the behavior of the function 
 remains the same.)

 When get-text-property is called for 'intangible, check what was set
 in each mode previously, and return the value for the matching mode.
 I found that, for org-mode, we can simply return t or nil for 
 'intangible.
 For egg-mode, we probably need to return 'its-part-1, 'its-part-2 or 
 nil depending on
 what was set before.
 (If it is called outside these modules, the behavior of the function 
 remains the same.)

 Now the difficult part is to set 'intangible that is shown to the rest
 of the Emacs Lisp world (outside org-mode and egg/tamago input mode).
 'intangible property seems to be used by display code written in C and
 so we must produce something that is visible to such routines.
 We need to set 'intangible to non-nil or nil depending on what was set
 in org-mode and egg-mode (probably the LOGICAL OR of what was set
 previously in both modes).

 I have no idea if my idea above works or not.

 The drastic surgery of replacing 'intangible in org-mode lisp code to
 'org-intangible mentioned in someone's blog is a desire to keep the 
 operation of
 the particular Japanese input system intact over the possible rare 
 misbehavior of
 org-mode.  But if we can fix both elisp codes somehow to make
 them compatible as in the case of flyspell.el, that will be wonderful.

 Any tips will be appreciated.


 Here is the search for "intangible" in  org-mode Emacs Lisp code.

 grep -n  -e intangible *.el /dev/null
 org-colview.el:248:     (overlay-put ov 'intangible t)
        CI's comment:  this usage may be a little tricky to handle.
 
 org-macs.el:151:(defmacro org-maybe-intangible (props)
 org-macs.el:152:  "Add '(intangible t) to PROPS if Emacs version is 
 earlier than Emacs 22.
 org-macs.el:154:intangible property is needed to make sure point skips 
 this text.
 org-macs.el:155:In Emacs 22, this is not necessary.  The intangible 
 text property has
 org-macs.el:160:      `(append '(intangible t) ,props)

        CI's comment: the above is the org-maybe-intangible defintion.
 org-macs.el:239:(defconst org-rm-props '(invisible t face t keymap t 
 intangible t mouse-face t
           CI's comment:
 	  org-rm-props is used in the following
 	  places, but their usage are as args to
 	  the few set of functions:

 	  grep -n  -e org-rm-props ~/repos/emacs-24.3/lisp/org/*.el /dev/null
           ~/repos/emacs-24.3/lisp/org/org-macs.el:239:(defconst 
 org-rm-props '(invisible t face t keymap t intangible t mouse-face t
           ~/repos/emacs-24.3/lisp/org/org-macs.el:247:	(and s 
 (remove-text-properties 0 (length s) org-rm-props s))
 	  CI's comment: the above is used as an arg to remove-text-properties)
 	  ~/repos/emacs-24.3/lisp/org/org-macs.el:254:in `org-rm-props'."

 	  The above is the usage in the following: the following
 	  processing may suggest one way to deal with the subtle issue
 	  of usage conflicts.
 
 	(defsubst org-no-properties (s &optional restricted)
 	  "Remove all text properties from string S.
 	When RESTRICTED is non-nil, only remove the properties listed
 	in `org-rm-props'."
 	  (if (fboundp 'set-text-properties)
 	      (set-text-properties 0 (length s) nil s)
 	    (if restricted
 		(remove-text-properties 0 (length s) org-rm-props s)
 	      (set-text-properties 0 (length s) nil s)))
 	  s)

 	  ~/repos/emacs-24.3/lisp/org/org-macs.el:258:	(remove-text-properties 
 0 (length s) org-rm-props s)
 	  ~/repos/emacs-24.3/lisp/org/org.el:8018:  (remove-text-properties 0 
 (length s) org-rm-props s)
 	  CI's comment: the above is used as an arg to remove-text-properties)
 	  The above usages are as args to remove-text-properties.
 

 org-src.el:314:				'(display nil invisible nil intangible nil))
     (CI's comment: this is used as an arg to remove-text-properties)

 org-src.el:472:				'(display nil invisible nil intangible nil))
     (CI's comment: the above is used as an arg to 
 remove-text-properties)

 org-table.el:1894:					       display t intangible t))
     (CI's comment: the above is used as an arg to 
 remove-text-properties)

 org-table.el:1923:					  intangible t))
     (CI's comment: the above is used as an arg to 
 remove-text-properties)

 org.el:5301:  '(mouse-face highlight keymap invisible intangible 
 help-echo org-linked-text htmlize-link))

     CI's comment: the above is used in  (defconst org-nonsticky-props
   '(mouse-face highlight keymap invisible intangible help-echo 
 org-linked-text htmlize-link))

 org.el:5328:				'(display t invisible t intangible t))
     (CI's comment: the above is used as an arg to 
 remove-text-properties)

 org.el:5370:				    '(display t invisible t intangible t))
     (CI's comment: the above is used as an arg to 
 remove-text-properties)

 org.el:5389:					'(display t invisible t intangible t)))
     (CI's comment: the above is used as an arg to 
 remove-text-properties)


 org.el:5490:	     (ip (org-maybe-intangible
      CI's comment org-maybe-intangible does not use 'intangible in
      emacs 24, I think.

 org.el:5572:      (add-text-properties s e (org-maybe-intangible 
 '(invisible org-cwidth)))
      CI's comment org-maybe-intangible does not use 'intangible in
      emacs 24, I think.

 org.el:5987:					 invisible t intangible t
      CI's comment: the above is used as an arg to
      remove-text-properties,
      i.e.   (remove-text-properties beg end
 			    '(mouse-face t keymap t org-linked-text t
 					 invisible t intangible t
 					 org-no-flyspell t org-emphasis t))
 

 org.el:10194:	(unless (get-text-property (match-end group) 'intangible) 
 ; Emacs 21
      CI's comment: checking with the get-text-property at the position.

 Igrep finished (matches found) at Thu Apr 18 16:45:43


 [] Searching the usage of "intangible" in tamago/egg

 ./egg-cnv.el:185:(defmacro egg-bunsetsu-info () ''intangible)

 ./its.el:326:				     'intangible         'its-part-2
     CI's comment: The above is part of set-text-properties as in

       (set-text-properties 0 1 (list 'read-only          t
 				     'invisible          'egg
 				     'intangible         'its-part-2
 				     'its-cursor         cursor
 				     'point-entered      'egg-enter/leave-fence
 				     'point-left         'egg-enter/leave-fence
 				     'modification-hooks '(egg-modify-fence))
 			   str)
       (insert str)

 ./its.el:350:;; intangible intangible
 ./its.el:354:  (let ((open-props '(its-start t intangible its-part-1))
 	  CI's comment: open-props is used subsequently
 	  as an argument to add-text-properties.
 	  (add-text-properties p p1 open-props)

  ./its.el:355:	(close-props '(rear-nonsticky t its-end t intangible 
 its-part-2))
 	  CI's comment: close-props is used subsequently
 	  as an argument to add-text-properties.
       (add-text-properties p1 (point) close-props)

 ./its.el:388:    (put-text-property 0 (length str) 'intangible 
 'its-part-1 str)
        CI's comment : the above is as argument to put-text-property.
 ./its.el:579:			       'intangible 'its-part-1)
        CI's comment: the above is as argument to add-text-property

     (add-text-properties 0 (length output)
 			 (list 'its-map its-current-map
 			       'its-syl newsyl
 			       'egg-lang its-current-language
 			       'read-only t
 			       'intangible 'its-part-1)
 			 output)

 ./its.el:895:    (put-text-property 0 (- end beg) 'intangible 
 'its-part-1 str)
 ./its.el:903:    (put-text-property 0 (- end beg) 'intangible 
 'its-part-2 str)
 ./its.el:1167:    (remove-text-properties 0 len '(intangible nil) 
 source)
 ./its.el:1197:	  (add-text-properties 0 len '(read-only t intangible 
 its-part-1) source)
 ./its.el:1199:      (add-text-properties 0 len '(read-only t intangible 
 its-part-2) source)
 ./its.el:1274:  (and (eq (get-text-property (point) 'intangible) 
 'its-part-2)
 ./its.el:1527:	    (put-text-property 0 len 'intangible 'its-part-1 
 old-str)


 TIA

Re: org-mode and 'intangible text property

[Stefan Monnier]

> org-mode and 'intangible text property

You might want to send this directly to the Org maintainers rather than
here where most readers are just as helpless as you are.

FWIW, most uses of `intangible' are ill-advised, BTW.


        Stefan

Re: org-mode and 'intangible text property

[ISHIKAWA,chiaki]

(2013/04/19 13:16), Stefan Monnier wrote:
>> org-mode and 'intangible text property
> 
> You might want to send this directly to the Org maintainers rather than
> here where most readers are just as helpless as you are.
> 
> FWIW, most uses of `intangible' are ill-advised, BTW.
> 
> 
>          Stefan
> 
> 

Will do. I just found a org support mailing list.
Thank you for the pointer. I also thought that
intangible and invisible usage in org-mode can be simplified.
But I will investigate in org-mode mailing list.

Thank you again.

Chiaki