Correct use of text properties?

Correct use of text properties?

[Joshua Lambert]

[-- Attachment #1: Type: text/plain, Size: 2480 bytes --]

I am trying to use text properties to make library MARC records more
readable in Emacs. Is the function below correctly written? It is meant to
simply replace codepoints 29 and 30 with newlines, using text properties.
It seems to visually accomplish what I want, but line numbers and some
cursor motions don't treat the newlines like true newlines. Is that
expected behavior?

(The files use codepoints 29-31 in ASCII or UTF8 so they may not display
properly.)

#+begin_src emacs-lisp
(defun marc-add-newlines ()
  "Replace the display text property of all   with newline."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (while (re-search-forward " " nil 1)
      (put-text-property (- (point) 1) (point) 'display "\n")))
  (save-excursion
    (goto-char (point-min))
    (while (re-search-forward " " nil 1)
      (put-text-property (- (point) 1) (point) 'display "\n"))))
#+end_src

Following is what a sample MARC record looks like, though MARC files
don't have newlines. So, if you test this, use the attached file.

#+begin_example MARC transmission record
00608nam 22001817a
45000010012000000050017000120080041000290490009000700400013000790740014
00092086002300106245009400129260002600223500003300249500003500282710006
900317949004000386 tmp97483713 20221114095123.0 221114s2022
xxu f000 0 eng d   aMOUE   aMvI cMvI   a1017-A-07 0  aY 4.F
76/1:117-105 04 aThe Burma Crisis, One Year After the Coup, Hearing,
Serial No. 117-105, February 17, 2022   a[S.l : bs.n., c2022?]
 aShipping List #: 2023-0021-P   aShipping List Date: 11/02/2022 1
 aUnited States. bCongress. bHouse. bCommittee on Foreign
Affairs.  1 aY 4.F 76/1:117-105 lSMVUS s- t1 h60
#+end_example

Following is the visual result after executing the function above.

#+begin_example
00608nam 22001817a
45000010012000000050017000120080041000290490009000700400013000790740014
00092086002300106245009400129260002600223500003300249500003500282710006
900317949004000386
tmp97483713
20221114095123.0
221114s2022
xxu f000 0 eng d
   aMOUE
   aMvI cMvI
   a1017-A-07
0  aY 4.F 76/1:117-105
04 aThe Burma Crisis, One Year After the Coup, Hearing, Serial No.
117-105, February 17, 2022
   a[S.l : bs.n., c2022?]
   aShipping List #: 2023-0021-P
   aShipping List Date: 11/02/2022
1  aUnited States. bCongress. bHouse. bCommittee on Foreign Affairs.
 1 aY 4.F 76/1:117-105 lSMVUS s- t1 h60
#+end_example

I'm relatively new to Emacs, elisp, and programming in general so
thanks for your help.

J. Lambert

[-- Attachment #2: sample-marc.mrc --]
[-- Type: application/marc, Size: 582 bytes --]

Re: Correct use of text properties?

[Eli Zaretskii]

> From: Joshua Lambert <jlambert.lis.tech@gmail.com>
> Date: Wed, 1 Mar 2023 22:21:14 -0600
> 
> I am trying to use text properties to make library MARC records more
> readable in Emacs. Is the function below correctly written? It is meant to
> simply replace codepoints 29 and 30 with newlines, using text properties.
> It seems to visually accomplish what I want, but line numbers and some
> cursor motions don't treat the newlines like true newlines. Is that
> expected behavior?

Probably.  But you don't describe the behavior you found surprising,
so it is hard to tell whether what you see is expected.  Please tell
more about the behavior you observe that surprised you.

> (The files use codepoints 29-31 in ASCII or UTF8 so they may not display
> properly.)
> 
> #+begin_src emacs-lisp
> (defun marc-add-newlines ()
>   "Replace the display text property of all   with newline."
>   (interactive)
>   (save-excursion
>     (goto-char (point-min))
>     (while (re-search-forward " " nil 1)
>       (put-text-property (- (point) 1) (point) 'display "\n")))
>   (save-excursion
>     (goto-char (point-min))
>     (while (re-search-forward " " nil 1)
>       (put-text-property (- (point) 1) (point) 'display "\n"))))
> #+end_src

If all you want is to break long lines on space characters, I suggest
to try visual-line-mode instead: it does that automatically in
low-level display code.  Did you try that?

Re: Correct use of text properties?

[Joshua Lambert]

[-- Attachment #1: Type: text/plain, Size: 1687 bytes --]

The pasted code may not have come through due to my copying and pasting
non-printing characters. Attached is the function in its own file.

On Thu, Mar 2, 2023 at 1:16 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Joshua Lambert <jlambert.lis.tech@gmail.com>
> > Date: Wed, 1 Mar 2023 22:21:14 -0600
> >

> > It seems to visually accomplish what I want, but line numbers and some
> > cursor motions don't treat the newlines like true newlines. Is that
> > expected behavior?
>
> Probably.  But you don't describe the behavior you found surprising,
> so it is hard to tell whether what you see is expected.  Please tell
> more about the behavior you observe that surprised you.

In the modeline and with display-line-numbers-mode the line number is always 1.
Also, pointer motions such as move-end-of-line and forward-paragraph jump to
the end of the file. Since I was adding newline characters as text properties, I
thought the line numbers might increment as I moved point down past those
newlines.

> If all you want is to break long lines on space characters, I suggest
> to try visual-line-mode instead: it does that automatically in
> low-level display code.  Did you try that?
>
I have to break the lines in specific places so this won't work. I only want to
break lines when there is a codepoint 29 or 30 in the file.

Thanks for your time and attention. Many if not all programs that manipulate
MARC records break/transform the data structure for human consumption and
then later remake it. I expected to have to do that in Emacs and then discovered
the display tables and text properties and thought I would explore those.

J. Lambert

[-- Attachment #2: marc-add-newlines.el --]
[-- Type: application/octet-stream, Size: 491 bytes --]

Re: Correct use of text properties?

[Eli Zaretskii]

> From: Joshua Lambert <jlambert.lis.tech@gmail.com>
> Date: Thu, 2 Mar 2023 09:25:20 -0600
> Cc: help-gnu-emacs@gnu.org
> 
> > Probably.  But you don't describe the behavior you found surprising,
> > so it is hard to tell whether what you see is expected.  Please tell
> > more about the behavior you observe that surprised you.
> 
> In the modeline and with display-line-numbers-mode the line number is always 1.

That is expected.  If you want to count screen lines, try setting
display-line-numbers to the value 'visual'.  However, that will
display numbers relative to the current line, so maybe this is not
what you want.

> Also, pointer motions such as move-end-of-line and forward-paragraph jump to
> the end of the file.

This is also expected.

> Since I was adding newline characters as text properties, I
> thought the line numbers might increment as I moved point down past those
> newlines.

No, these properties only affect the display, not how Emacs counts
lines or moves point: those are still done on the actual buffer text.

Re: Correct use of text properties?

[Joshua Lambert]

Thank you. That explains what I wanted to know.

J. Lambert

On Thu, Mar 2, 2023 at 9:48 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Joshua Lambert <jlambert.lis.tech@gmail.com>
> > Date: Thu, 2 Mar 2023 09:25:20 -0600
> > Cc: help-gnu-emacs@gnu.org
> >
> > > Probably.  But you don't describe the behavior you found surprising,
> > > so it is hard to tell whether what you see is expected.  Please tell
> > > more about the behavior you observe that surprised you.
> >
> > In the modeline and with display-line-numbers-mode the line number is always 1.
>
> That is expected.  If you want to count screen lines, try setting
> display-line-numbers to the value 'visual'.  However, that will
> display numbers relative to the current line, so maybe this is not
> what you want.
>
> > Also, pointer motions such as move-end-of-line and forward-paragraph jump to
> > the end of the file.
>
> This is also expected.
>
> > Since I was adding newline characters as text properties, I
> > thought the line numbers might increment as I moved point down past those
> > newlines.
>
> No, these properties only affect the display, not how Emacs counts
> lines or moves point: those are still done on the actual buffer text.
>

Re: Correct use of text properties?

[Michael Heerdegen]

Joshua Lambert <jlambert.lis.tech@gmail.com> writes:

> I have to break the lines in specific places so this won't work. I
> only want to break lines when there is a codepoint 29 or 30 in the
> file.

You might then want to consider to try longlines-mode: it allows to
control at which characters it adds virtual line breaks (see
`longlines-break-chars').

Michael.