Stylus drawing input?

Stylus drawing input?

[Dov Grobgeld]

Hello,

I wonder if anyone has ever worked on stylus (or mouse actually)
drawing input for emacs, which stores a "path" [*] of the stylus input
events?

Here are some possible use cases for such support:

In PDF-Mode there is an annotation option that allows attaching text
to various points in the document. It would be a nice addition to be
able to scribble with a stylus on top of the document and leaving
behind a "path" annotation as well.

Another use for such a mode would be for the android port, where we
may send the path to a path-to-text algorithm, for stylus text input.

When giving a presentation, it is often quicker to scribble on top of
the slides to illustrate a point, than it is to edit the source text
of the presentation.

Thanks!

[1] A path is here defined as a sequence of moveto xy, lineto xy,
(curveto(?), closepath) commands. Possibly with with additional vertex
attributes like time, pressure, tilt, etc.

Re: Stylus drawing input?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I wonder if anyone has ever worked on stylus (or mouse actually)
  > drawing input for emacs, which stores a "path" [*] of the stylus input
  > events?

Which system platforms support this sort of input?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Stylus drawing input?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]


Richard Stallman <rms@gnu.org> writes:

>   > I wonder if anyone has ever worked on stylus (or mouse actually)
>   > drawing input for emacs, which stores a "path" [*] of the stylus input
>   > events?
>
> Which system platforms support this sort of input?

I remember trying out an input in emacs for drawing letters that then
got inserted into the buffer, but I now searched for an hour and did not
find it again.

Just writing here to note that such a thing existed and it was pretty
awesome to just run a command, draw a picture in emacs, and have it
inserted into the current text.

Things I stumbled upon on the way:

- draw in emacs: https://github.com/misohena/el-easydraw
- draw svgs in emacs: https://elpa.gnu.org/devel/sketch-mode.html

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Stylus drawing input?

[Peter]

> I remember trying out an input in emacs for drawing letters that then
> got inserted into the buffer, but I now searched for an hour and did not
> find it again.
> Just writing here to note that such a thing existed and it was pretty
> awesome to just run a command, draw a picture in emacs, and have it
> inserted into the current text.

strokes.el (part of Emacs) could maybe be used to do this, when
capturing a new stroke it keeps track of the full path (and even can
generate a small image showing the path).

Best wishes,
Peter

Re: Stylus drawing input?

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

> Which system platforms support this sort of input?

GNU/Linux platforms running the X Window System.

Re: Stylus drawing input?

[Eli Zaretskii]

> From: "Dr. Arne Babenhauserheide" <arne_bab@web.de>
> Cc: Dov Grobgeld <dov.grobgeld@gmail.com>, emacs-devel@gnu.org
> Date: Sat, 14 Jan 2023 09:17:24 +0100
> 
> > Which system platforms support this sort of input?
> 
> I remember trying out an input in emacs for drawing letters that then
> got inserted into the buffer, but I now searched for an hour and did not
> find it again.
> 
> Just writing here to note that such a thing existed and it was pretty
> awesome to just run a command, draw a picture in emacs, and have it
> inserted into the current text.

Do you mean artist-mode, perhaps?  Or strokes.el?

Re: Stylus drawing input?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]


Eli Zaretskii <eliz@gnu.org> writes:

>> From: "Dr. Arne Babenhauserheide" <arne_bab@web.de>
>> Cc: Dov Grobgeld <dov.grobgeld@gmail.com>, emacs-devel@gnu.org
>> Date: Sat, 14 Jan 2023 09:17:24 +0100
>> 
>> > Which system platforms support this sort of input?
>> 
>> I remember trying out an input in emacs for drawing letters that then
>> got inserted into the buffer, but I now searched for an hour and did not
>> find it again.
>> 
>> Just writing here to note that such a thing existed and it was pretty
>> awesome to just run a command, draw a picture in emacs, and have it
>> inserted into the current text.
>
> Do you mean artist-mode, perhaps?  Or strokes.el?

Strokes it is!   
M-x strokes-compose-complex-stroke

Thank you!

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Stylus drawing input?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > - draw in emacs: https://github.com/misohena/el-easydraw

Is this a useful package that would be nice to have in ELPA?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Stylus drawing input?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Which system platforms support this sort of input?

  > GNU/Linux platforms running the X Window System.

That's good,  I was concerned it might be only MacOS or something
similar.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Stylus drawing input?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 829 bytes --]


Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > - draw in emacs: https://github.com/misohena/el-easydraw
>
> Is this a useful package that would be nice to have in ELPA?

I think so, yes. I did not try it yet, but it seems to simplify a
documentation workflow alot, and it inlines the SVG data directly into
the link, so the document stays self-contained.

It looks similar to drawing with ditaa, but without the external
dependency, because there is no compile-step (it’s just SVG data).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Stylus drawing input?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

PS: Here is a website with a description that does not rely on unfree
    platforms: https://misohena.jp/blog/2021-09-21-emacs-easy-draw.html

The clearest description is a gif, though:
https://misohena.jp/blog/wp-content/uploads/2021-09-21-edraw-example.gif

Richard Stallman <rms@gnu.org> writes:
>   > - draw in emacs: https://github.com/misohena/el-easydraw
>
> Is this a useful package that would be nice to have in ELPA?


-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Stylus drawing input?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 403 bytes --]

On Mon, Jan 16, 2023 at 08:45:55AM +0100, Dr. Arne Babenhauserheide wrote:
> PS: Here is a website with a description that does not rely on unfree
>     platforms: https://misohena.jp/blog/2021-09-21-emacs-easy-draw.html
> 
> The clearest description is a gif, though:
> https://misohena.jp/blog/wp-content/uploads/2021-09-21-edraw-example.gif

Thanks for the non-github link!

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Stylus drawing input?

[Dov Grobgeld]

On X11, to read stylus input, we need to use XInput extension.

Gdk, used in gtk, has a good abstraction to it that allows doing code
like the following to distinguish between mouse and stylus input in
its motion notify callback, and in this case extract the pressure.

  int on_motion_notify_event(GdkEventMotion *event)
  {
    // The pressure value is:
    //   found=false for mouse motion
    //   0 for proximity stylus motion
    //   >0 for the pen touching the pad
    double pressure=-1;
    int found = gdk_event_get_axis((GdkEvent*)event,
GDK_AXIS_PRESSURE, &pressure);
    GdkDeviceTool *tool = gdk_event_get_device_tool((GdkEvent*)event);

    // For mouse the tool isn't defined, so we set a default -1
    GdkDeviceToolType tool_type = (GdkDeviceToolType)(-1);
    if (tool)
      // true for stylus only
      tool_type = gdk_device_tool_get_tool_type (tool);

    printf("motion_notify: tool_type=%d,
x=%.3f},y=%.3f},found_stylus=%d,pressure=%.3f\n",
          (int)tool_type,
          event->x, event->y, found, pressure);
    return TRUE;
  }

To support styluses (i.e. to distinguish them from mouse events) we
need something similar in emacs.

Regards,


On Mon, Jan 16, 2023 at 10:27 AM <tomas@tuxteam.de> wrote:
>
> On Mon, Jan 16, 2023 at 08:45:55AM +0100, Dr. Arne Babenhauserheide wrote:
> > PS: Here is a website with a description that does not rely on unfree
> >     platforms: https://misohena.jp/blog/2021-09-21-emacs-easy-draw.html
> >
> > The clearest description is a gif, though:
> > https://misohena.jp/blog/wp-content/uploads/2021-09-21-edraw-example.gif
>
> Thanks for the non-github link!
>
> Cheers
> --
> t

Re: Stylus drawing input?

[Po Lu]

Emacs 29 already has native support for version 2 of the the X Input extension and its various input devices.

See the node Touchscreen Events in the Emacs Lisp reference manual.  This is not yet supported on the PGTK build.

On January 24, 2023 5:15:58 AM GMT+08:00, Dov Grobgeld <dov.grobgeld@gmail.com> wrote:
>On X11, to read stylus input, we need to use XInput extension.
>
>Gdk, used in gtk, has a good abstraction to it that allows doing code
>like the following to distinguish between mouse and stylus input in
>its motion notify callback, and in this case extract the pressure.
>
>  int on_motion_notify_event(GdkEventMotion *event)
>  {
>    // The pressure value is:
>    //   found=false for mouse motion
>    //   0 for proximity stylus motion
>    //   >0 for the pen touching the pad
>    double pressure=-1;
>    int found = gdk_event_get_axis((GdkEvent*)event,
>GDK_AXIS_PRESSURE, &pressure);
>    GdkDeviceTool *tool = gdk_event_get_device_tool((GdkEvent*)event);
>
>    // For mouse the tool isn't defined, so we set a default -1
>    GdkDeviceToolType tool_type = (GdkDeviceToolType)(-1);
>    if (tool)
>      // true for stylus only
>      tool_type = gdk_device_tool_get_tool_type (tool);
>
>    printf("motion_notify: tool_type=%d,
>x=%.3f},y=%.3f},found_stylus=%d,pressure=%.3f\n",
>          (int)tool_type,
>          event->x, event->y, found, pressure);
>    return TRUE;
>  }
>
>To support styluses (i.e. to distinguish them from mouse events) we
>need something similar in emacs.
>
>Regards,
>
>
>On Mon, Jan 16, 2023 at 10:27 AM <tomas@tuxteam.de> wrote:
>>
>> On Mon, Jan 16, 2023 at 08:45:55AM +0100, Dr. Arne Babenhauserheide wrote:
>> > PS: Here is a website with a description that does not rely on unfree
>> >     platforms: https://misohena.jp/blog/2021-09-21-emacs-easy-draw.html
>> >
>> > The clearest description is a gif, though:
>> > https://misohena.jp/blog/wp-content/uploads/2021-09-21-edraw-example.gif
>>
>> Thanks for the non-github link!
>>
>> Cheers
>> --
>> t
>

Re: Stylus drawing input?

[Dov Grobgeld]

Thanks. I need some guidance on how feasible the following is and how
I would go about doing it.

1. How do I draw vector graphics object in an emacs buffer. I.e.
line-to, move-to, close-path, fill etc. Do I need to generate the
pixels on my own (can be done e.g. with the agg library or with
libcairo) and then place the image as on overlay (how?), or is there
higher lever interface? Is it accessible from elisp or from C?
2. How can I trace the x,y input coordinates of mouse movement. E.g.
I'd like to run a command to enter capture mode, and then catch
subsequent motion events of the mouse, until the mouse button is
raised. The resulting object should be a list of motion coordinates.
3. How would I do the same for a stylus while catching additional pressure data?
4. Given some vector graphics objects drawn on the screen, how would I
do mouse hit detection, so that doing a mouse click on the object,
returns the object id? (This I can do "offline" though, by drawing a
"label image" of the buffer contents on mouse press end then finding
the label of the object under the mouse.)

Thanks!

On Tue, Jan 24, 2023 at 1:47 AM Po Lu <Luangruo@yahoo.com> wrote:
>
> Emacs 29 already has native support for version 2 of the the X Input extension and its various input devices.
>
> See the node Touchscreen Events in the Emacs Lisp reference manual.  This is not yet supported on the PGTK build.
>
> On January 24, 2023 5:15:58 AM GMT+08:00, Dov Grobgeld <dov.grobgeld@gmail.com> wrote:
> >On X11, to read stylus input, we need to use XInput extension.
> >
> >Gdk, used in gtk, has a good abstraction to it that allows doing code
> >like the following to distinguish between mouse and stylus input in
> >its motion notify callback, and in this case extract the pressure.
> >
> >  int on_motion_notify_event(GdkEventMotion *event)
> >  {
> >    // The pressure value is:
> >    //   found=false for mouse motion
> >    //   0 for proximity stylus motion
> >    //   >0 for the pen touching the pad
> >    double pressure=-1;
> >    int found = gdk_event_get_axis((GdkEvent*)event,
> >GDK_AXIS_PRESSURE, &pressure);
> >    GdkDeviceTool *tool = gdk_event_get_device_tool((GdkEvent*)event);
> >
> >    // For mouse the tool isn't defined, so we set a default -1
> >    GdkDeviceToolType tool_type = (GdkDeviceToolType)(-1);
> >    if (tool)
> >      // true for stylus only
> >      tool_type = gdk_device_tool_get_tool_type (tool);
> >
> >    printf("motion_notify: tool_type=%d,
> >x=%.3f},y=%.3f},found_stylus=%d,pressure=%.3f\n",
> >          (int)tool_type,
> >          event->x, event->y, found, pressure);
> >    return TRUE;
> >  }
> >
> >To support styluses (i.e. to distinguish them from mouse events) we
> >need something similar in emacs.
> >
> >Regards,
> >
> >
> >On Mon, Jan 16, 2023 at 10:27 AM <tomas@tuxteam.de> wrote:
> >>
> >> On Mon, Jan 16, 2023 at 08:45:55AM +0100, Dr. Arne Babenhauserheide wrote:
> >> > PS: Here is a website with a description that does not rely on unfree
> >> >     platforms: https://misohena.jp/blog/2021-09-21-emacs-easy-draw.html
> >> >
> >> > The clearest description is a gif, though:
> >> > https://misohena.jp/blog/wp-content/uploads/2021-09-21-edraw-example.gif
> >>
> >> Thanks for the non-github link!
> >>
> >> Cheers
> >> --
> >> t
> >

Re: Stylus drawing input?

[Po Lu]

Dov Grobgeld <dov.grobgeld@gmail.com> writes:

> Thanks. I need some guidance on how feasible the following is and how
> I would go about doing it.
>
> 1. How do I draw vector graphics object in an emacs buffer. I.e.
> line-to, move-to, close-path, fill etc. Do I need to generate the
> pixels on my own (can be done e.g. with the agg library or with
> libcairo) and then place the image as on overlay (how?), or is there
> higher lever interface?

Emacs doesn't have any built in vector graphics capability.  But the
image route could probably work, yes.

> 2. How can I trace the x,y input coordinates of mouse movement. E.g.
> I'd like to run a command to enter capture mode, and then catch
> subsequent motion events of the mouse, until the mouse button is
> raised. The resulting object should be a list of motion coordinates.

I suggest reading through looking the entirety of (elisp)Input Events.
It will explain everything better than I can.

> 3. How would I do the same for a stylus while catching additional
> pressure data?

I will see what interfaces the X Input extension provides to access that
information and ack.

> 4. Given some vector graphics objects drawn on the screen, how would I
> do mouse hit detection, so that doing a mouse click on the object,
> returns the object id? (This I can do "offline" though, by drawing a
> "label image" of the buffer contents on mouse press end then finding
> the label of the object under the mouse.)

I think you had better head to the nearest library (or perhaps book
store) for this one.  That is outside of the scope of this list.

Re: Stylus drawing input?

[Eli Zaretskii]

> From: Dov Grobgeld <dov.grobgeld@gmail.com>
> Date: Tue, 24 Jan 2023 10:03:02 +0200
> Cc: emacs-devel@gnu.org, tomas@tuxteam.de
> 
> Thanks. I need some guidance on how feasible the following is and how
> I would go about doing it.
> 
> 1. How do I draw vector graphics object in an emacs buffer. I.e.
> line-to, move-to, close-path, fill etc. Do I need to generate the
> pixels on my own (can be done e.g. with the agg library or with
> libcairo) and then place the image as on overlay (how?), or is there
> higher lever interface? Is it accessible from elisp or from C?
> 2. How can I trace the x,y input coordinates of mouse movement. E.g.
> I'd like to run a command to enter capture mode, and then catch
> subsequent motion events of the mouse, until the mouse button is
> raised. The resulting object should be a list of motion coordinates.
> 3. How would I do the same for a stylus while catching additional pressure data?
> 4. Given some vector graphics objects drawn on the screen, how would I
> do mouse hit detection, so that doing a mouse click on the object,
> returns the object id? (This I can do "offline" though, by drawing a
> "label image" of the buffer contents on mouse press end then finding
> the label of the object under the mouse.)

Does the svg.el library help with some of the above?

Re: Stylus drawing input?

[Dov Grobgeld]

Thanks! I had a look at it and it seems that it shows the rendering
part of issue number 1. I.e. by expressing the vector graphics as svg
we can generate an image and insert it into the buffer. It might also
solve number 4, if I can figure out how to make it draw antialiased in
a single color.

But I still don't understand how emacs can display such an image as an
"overlay" on top of other buffer contents. E.g. if I have an RGBA
image (or svg object) containing the stroke of the stylus. How can I
display that "above" the pdf display window? Does emacs support the
notion of layers?

(Having the notion of layers would also be nice for doing
presentations in emacs, as you can then have some static graphics as a
background image.)

Regards,
Dov



On Tue, Jan 24, 2023 at 2:27 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Dov Grobgeld <dov.grobgeld@gmail.com>
> > Date: Tue, 24 Jan 2023 10:03:02 +0200
> > Cc: emacs-devel@gnu.org, tomas@tuxteam.de
> >
> > Thanks. I need some guidance on how feasible the following is and how
> > I would go about doing it.
> >
> > 1. How do I draw vector graphics object in an emacs buffer. I.e.
> > line-to, move-to, close-path, fill etc. Do I need to generate the
> > pixels on my own (can be done e.g. with the agg library or with
> > libcairo) and then place the image as on overlay (how?), or is there
> > higher lever interface? Is it accessible from elisp or from C?
> > 2. How can I trace the x,y input coordinates of mouse movement. E.g.
> > I'd like to run a command to enter capture mode, and then catch
> > subsequent motion events of the mouse, until the mouse button is
> > raised. The resulting object should be a list of motion coordinates.
> > 3. How would I do the same for a stylus while catching additional pressure data?
> > 4. Given some vector graphics objects drawn on the screen, how would I
> > do mouse hit detection, so that doing a mouse click on the object,
> > returns the object id? (This I can do "offline" though, by drawing a
> > "label image" of the buffer contents on mouse press end then finding
> > the label of the object under the mouse.)
>
> Does the svg.el library help with some of the above?

Re: Stylus drawing input?

[Eli Zaretskii]

> From: Dov Grobgeld <dov.grobgeld@gmail.com>
> Date: Tue, 24 Jan 2023 17:55:15 +0200
> Cc: Luangruo@yahoo.com, emacs-devel@gnu.org, tomas@tuxteam.de
> 
> But I still don't understand how emacs can display such an image as an
> "overlay" on top of other buffer contents.

It cannot.  Emacs treats images as large characters, so all we are
capable of doing is display images in-line.

> (Having the notion of layers would also be nice for doing
> presentations in emacs, as you can then have some static graphics as a
> background image.)

That would need a complete redesign of the current display engine,
AFAIU.  Patches welcome.