unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
* avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
@ 2022-08-20 19:59 Aleix Conchillo Flaqué
  2022-08-20 21:48 ` Maxime Devos
  0 siblings, 1 reply; 7+ messages in thread
From: Aleix Conchillo Flaqué @ 2022-08-20 19:59 UTC (permalink / raw)
  To: guile-user, guile-devel

Hi there,

I have a GuileScript example that it's basically a web server written in
Guile that returns a basic HTML and also a <script> with some code
generated with GuileScript. Something like this:

------------------------------------------------
(use-modules ((language guilescript compile) #:prefix gs:) ....)

(define js-fibonacci
  '(begin
     (define (fib n)
       (if (<= n 1)
           1
           (+ (fib (- n 2)) (fib (- n 1)))))

     (define (computeFibonacci)
       (let ((elem (js-invoke document 'getElementById "number"))
             (n (js-ref elem 'value)))
         (alert (+ "Fibonacci of " n " is " (fib n)))))

     (alert "Let's compute some fibonacci numbers!")))

(define js-script
  (receive (js _)
      (gs:compile js-fibonacci)
    js))

;;
;; This is the web page.
;;
(define (main-form request body)
  `(html
    (head (title "Fibonacci GuileScript Server")
          (script ,js-script))
    (body
     (input (@ (id "number") (type "text") (size "50") (value "")))
     (button (@ (type "button") (onclick "computeFibonacci()")) "Compute
Fibonacci"))))
------------------------------------------------

However, I'm not able to find a way to avoid character encoding/escaping
and the generated code inside <script> will always have "&lt;", etc. And
<script> is a place where encodings can be avoided. This is true for both
Guile and guile-lib's (htmlprag), even though htmlprag's escapes less
characters (e.g. double quotes).

One way I found to solve this was to have <script src="fib.js"> and then
have a handler for fib.js that would just return the transpiled string. But
it's not as nice, it's extra work and it's also an additional roundtrip to
the server.

Has anyone ran into this issue? Would is make sense to add a keyword
argument to (sxml->xml)? For example, (sxml->xml SXML PORT #:escape #f).

Best,

Aleix


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
  2022-08-20 19:59 avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html Aleix Conchillo Flaqué
@ 2022-08-20 21:48 ` Maxime Devos
  2022-08-21  0:05   ` Aleix Conchillo Flaqué
  0 siblings, 1 reply; 7+ messages in thread
From: Maxime Devos @ 2022-08-20 21:48 UTC (permalink / raw)
  To: Aleix Conchillo Flaqué, guile-user, guile-devel


[-- Attachment #1.1.1: Type: text/plain, Size: 1327 bytes --]

The GuileScript looks nice, for interested readers, see 
<https://github.com/aconchillo/guilescript>

On 20-08-2022 21:59, Aleix Conchillo Flaqué wrote:
> However, I'm not able to find a way to avoid character 
> encoding/escaping and the generated code inside <script> will always 
> have "&lt;", etc. And <script> is a place where encodings can be 
> avoided. This is true for both Guile and guile-lib's (htmlprag), even 
> though htmlprag's escapes less characters (e.g. double quotes).
>
> One way I found to solve this was to have <script src="fib.js"> and 
> then have a handler for fib.js that would just return the transpiled 
> string. But it's not as nice, it's extra work and it's also an 
> additional roundtrip to the server.
>
> Has anyone ran into this issue? Would is make sense to add a keyword 
> argument to (sxml->xml)? For example, (sxml->xml SXML PORT #:escape #f).

Having unescaped < in <script>...</script> does not seem valid XML to me.

Does escaping &lt; actually cause problems?  If so, you could look into 
declaring your web page as XHTML, with an appropriate doctype and 
Content-Type.

I would expect XHTML to accept &lt; and reject unescaped <, being XML 
and not the messy syntax of HTML, but it's not something I have tried 
out myself.

Greetings,
Maxime.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
  2022-08-20 21:48 ` Maxime Devos
@ 2022-08-21  0:05   ` Aleix Conchillo Flaqué
  2022-08-21  5:09     ` tomas
  2022-08-21 10:16     ` Maxime Devos
  0 siblings, 2 replies; 7+ messages in thread
From: Aleix Conchillo Flaqué @ 2022-08-21  0:05 UTC (permalink / raw)
  To: Maxime Devos; +Cc: guile-user, guile-devel

Hi Maxime,

On Sat, Aug 20, 2022 at 2:48 PM Maxime Devos <maximedevos@telenet.be> wrote:
>
> The GuileScript looks nice, for interested readers, see <
https://github.com/aconchillo/guilescript>
>
> On 20-08-2022 21:59, Aleix Conchillo Flaqué wrote:
>
> However, I'm not able to find a way to avoid character encoding/escaping
and the generated code inside <script> will always have "&lt;", etc. And
<script> is a place where encodings can be avoided. This is true for both
Guile and guile-lib's (htmlprag), even though htmlprag's escapes less
characters (e.g. double quotes).
>
> One way I found to solve this was to have <script src="fib.js"> and then
have a handler for fib.js that would just return the transpiled string. But
it's not as nice, it's extra work and it's also an additional roundtrip to
the server.
>
> Has anyone ran into this issue? Would is make sense to add a keyword
argument to (sxml->xml)? For example, (sxml->xml SXML PORT #:escape #f).
>
> > Having unescaped < in <script>...</script> does not seem valid XML to
me.
>

According to the spec, embedding inline content in the <script> tag should
conform to the language defined by the "type" attribute (defaults to
javascript). So, I would expect you could put any string that conforms to
JS.

"""
When used to include dynamic scripts, the scripts may either be embedded
inline or may be imported from an external file using the src attribute. If
the language is not that described by "text/javascript", then the type
attribute must be present, as described below. Whatever language is used,
the contents of the script element must conform with the requirements of
that language's specification
"""

> Does escaping &lt; actually cause problems?  If so, you could look into
declaring your web page as XHTML, with an appropriate doctype and
Content-Type.
>
> I would expect XHTML to accept &lt; and reject unescaped <, being XML and
not the messy syntax of HTML, but it's not something I have tried out
myself.
>

It does, browsers (at least Chrome) don't interpret that correctly, since
it's not valid JavaScript.

In any case, I just went by using my work around of loading the JS as a
separate file:

https://github.com/aconchillo/guilescript/blob/master/examples/fibonacci-server.scm

Best,

Aleix


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
  2022-08-21  0:05   ` Aleix Conchillo Flaqué
@ 2022-08-21  5:09     ` tomas
  2022-08-21 10:16     ` Maxime Devos
  1 sibling, 0 replies; 7+ messages in thread
From: tomas @ 2022-08-21  5:09 UTC (permalink / raw)
  To: guile-devel

[-- Attachment #1: Type: text/plain, Size: 1713 bytes --]

On Sat, Aug 20, 2022 at 05:05:22PM -0700, Aleix Conchillo Flaqué wrote:
> Hi Maxime,
> 
> On Sat, Aug 20, 2022 at 2:48 PM Maxime Devos <maximedevos@telenet.be> wrote:
> >
> > The GuileScript looks nice, for interested readers, see <
> https://github.com/aconchillo/guilescript>
> >
> > On 20-08-2022 21:59, Aleix Conchillo Flaqué wrote:
> >
> > However, I'm not able to find a way to avoid character encoding/escaping
> and the generated code inside <script> will always have "&lt;", etc. And
> <script> is a place where encodings can be avoided. This is true for both
> Guile and guile-lib's (htmlprag), even though htmlprag's escapes less
> characters (e.g. double quotes).

Indeed. <script> is a monster in this endless tragicomedy which is HTML.

Here's what the WHATWG has to say [1] on it:

    "The easiest and safest way to avoid the rather strange restrictions
     described in this section is to always escape an ASCII case-insensitive
     match for "<!--" as "\x3C!--", "<script" as "\x3Cscript", and
     "</script" as "\x3C/script" when these sequences appear in literals
     in scripts [...], and to avoid writing code that uses such constructs
     in expressions."

So basically, what you have to do to preprocess your script is to find out
whether the offending sequences (i.e. "<!--" [I'd have forgotten that one],
and "<script" [why?] and finally "</script") with their (javastring-) escaped
counterparts whenever they appear in javascript strings... and not use them
otherwise.

If your language is not javascript... good luck.

Cheers

[1] https://html.spec.whatwg.org/multipage/scripting.html#restrictions-for-contents-of-script-elements

-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
  2022-08-21  0:05   ` Aleix Conchillo Flaqué
  2022-08-21  5:09     ` tomas
@ 2022-08-21 10:16     ` Maxime Devos
  2022-08-21 10:23       ` tomas
  2022-08-21 22:34       ` Aleix Conchillo Flaqué
  1 sibling, 2 replies; 7+ messages in thread
From: Maxime Devos @ 2022-08-21 10:16 UTC (permalink / raw)
  To: Aleix Conchillo Flaqué; +Cc: guile-user, guile-devel


[-- Attachment #1.1.1.1: Type: text/plain, Size: 2362 bytes --]

On 21-08-2022 02:05, Aleix Conchillo Flaqué wrote:

> According to the spec, embedding inline content in the <script> tag 
> should conform to the language defined by the "type" attribute 
> (defaults to javascript). So, I would expect you could put any string 
> that conforms to JS.
>
> """
> When used to include dynamic scripts, the scripts may either be 
> embedded inline or may be imported from an external file using the src 
> attribute. If the language is not that described by "text/javascript", 
> then the type attribute must be present, as described below. Whatever 
> language is used, the contents of the script element must conform with 
> the requirements of that language's specification

I am proposing to use XHTML (which is XML), not HTML. HTML's special 
parsing quirks are irrelevant here.

> It does, browsers (at least Chrome) don't interpret that correctly, 
> since it's not valid JavaScript.
As <script> ... </script> is XML, the XML parser  (not the HTML parser, 
this is XHTML!) will decode the &lt; inside the <script>...</script>, 
the result _after decoding_ is valid JavaScript.  In XML, <script> is 
not special -- everything is parsed the same way in XML.

Anyway, it seems to work for me, both in icecat and ungoogled-chromium:

> (use-modules (web server))
> (define document
> "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
> <html xmlns=\"http://www.w3.org/1999/xhtml\">
> <head>
> <title>Test</title>
> </head>
> <body>
> <script type=\"text/javascript\">
> console.log(\"&lt;Hi!&gt;\");
> </script>
> </body>
> </html>")
>
> (define (handler request request-body)
>   (values '((content-type application/xhtml+xml))
>       document))
>
> (run-server handler 'http)
-- on the console, <Hi!> is logged, not &lt;Hi!&gt;.

If I replace &lt; by < and &gt; by > to make it 'valid JavasScript' as 
you appear to be proposing, I get a parsing error:

>
>       (ungoogled-chromium)
>       This page contains the following errors:
>
> error on line 8 at column 17: error parsing attribute name
>
>
>       Below is a rendering of the page up to the first error.
>
>
and

> (icecat):
> XML Parsing Error: not well-formed
> Location: http://localhost:8080/
> Line Number 8, Column 17:
> console.log("<Hi!>");
> ----------------^

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 3859 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
  2022-08-21 10:16     ` Maxime Devos
@ 2022-08-21 10:23       ` tomas
  2022-08-21 22:34       ` Aleix Conchillo Flaqué
  1 sibling, 0 replies; 7+ messages in thread
From: tomas @ 2022-08-21 10:23 UTC (permalink / raw)
  To: guile-devel

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

On Sun, Aug 21, 2022 at 12:16:54PM +0200, Maxime Devos wrote:
> On 21-08-2022 02:05, Aleix Conchillo Flaqué wrote:
> 
> > According to the spec, embedding inline content in the <script> tag
> > should conform to the language defined by the "type" attribute (defaults
> > to javascript). So, I would expect you could put any string that
> > conforms to JS.
> > 
> > """
> > When used to include dynamic scripts, the scripts may either be embedded
> > inline or may be imported from an external file using the src attribute.
> > If the language is not that described by "text/javascript", then the
> > type attribute must be present, as described below. Whatever language is
> > used, the contents of the script element must conform with the
> > requirements of that language's specification
> 
> I am proposing to use XHTML (which is XML), not HTML. HTML's special parsing
> quirks are irrelevant here.

Yes, the problem evaporates with XHTML. If you are at the generating side
(like here), that's definitely an option.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html
  2022-08-21 10:16     ` Maxime Devos
  2022-08-21 10:23       ` tomas
@ 2022-08-21 22:34       ` Aleix Conchillo Flaqué
  1 sibling, 0 replies; 7+ messages in thread
From: Aleix Conchillo Flaqué @ 2022-08-21 22:34 UTC (permalink / raw)
  To: Maxime Devos; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

Thank you Maxime,

On Sun, Aug 21, 2022 at 3:16 AM Maxime Devos <maximedevos@telenet.be> wrote:

> On 21-08-2022 02:05, Aleix Conchillo Flaqué wrote:
>
> According to the spec, embedding inline content in the <script> tag should
> conform to the language defined by the "type" attribute (defaults to
> javascript). So, I would expect you could put any string that conforms to
> JS.
>
> """
> When used to include dynamic scripts, the scripts may either be embedded
> inline or may be imported from an external file using the src attribute. If
> the language is not that described by "text/javascript", then the type
> attribute must be present, as described below. Whatever language is used,
> the contents of the script element must conform with the requirements of
> that language's specification
>
> I am proposing to use XHTML (which is XML), not HTML. HTML's special
> parsing quirks are irrelevant here.
>
> It does, browsers (at least Chrome) don't interpret that correctly, since
> it's not valid JavaScript.
>
> As <script> ... </script> is XML, the XML parser  (not the HTML parser,
> this is XHTML!) will decode the &lt; inside the <script>...</script>, the
> result _after decoding_ is valid JavaScript.  In XML, <script> is not
> special -- everything is parsed the same way in XML.
>
>
Got it! I didn't know that was treated differently. I just tried it and it
works perfectly and it's exactly what I wanted:

https://github.com/aconchillo/guilescript/commit/c959ceff15e8e9fdf81cc59a754ed66e6bb53cc0

I avoided the <?xml?> declaration. I know that's mandatory in XML
documents, but it seemed to work without it.

Thanks again!

Aleix

[-- Attachment #2: Type: text/html, Size: 3473 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2022-08-21 22:34 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-20 19:59 avoid character encoding/escaping in sxml->xml or htmlprag's sxml->html Aleix Conchillo Flaqué
2022-08-20 21:48 ` Maxime Devos
2022-08-21  0:05   ` Aleix Conchillo Flaqué
2022-08-21  5:09     ` tomas
2022-08-21 10:16     ` Maxime Devos
2022-08-21 10:23       ` tomas
2022-08-21 22:34       ` Aleix Conchillo Flaqué

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).