Converting strings of hexadecimal numbers to the respective bytes

Converting strings of hexadecimal numbers to the respective bytes

[florian]

I was wondering if it is possible to convert a string, say '0a', which
represents a two-digit hexadecimal number, to the corresponding byte.
The solution does not seem far away (something like char-to-string),
but surprisingly, Elisp does not seem to deal much with hexadecimal
numbers.

What I mean is: if it were a three-digit decimal representation, you
could say

   (char-to-string (string-to-number "010"))

instead, but I don't seem to see such a possibility for a hexadecimal
number.

The question may be very dumb, but could Elisp do such a thing at all?

Thanks for any enlightenment ...

Florian

Re: Converting strings of hexadecimal numbers to the respective bytes

[Juanma Barranquero]

On Tue, Apr 7, 2009 at 00:22, florian <lorian@fsavigny.de> wrote:

> What I mean is: if it were a three-digit decimal representation, you
> could say
>
>   (char-to-string (string-to-number "010"))
>
> instead, but I don't seem to see such a possibility for a hexadecimal
> number.

  (string-to-number string &optional base)

  Parse string as a decimal number and return the number.
  This parses both integers and floating point numbers.
  It ignores leading spaces and tabs.

  If base, interpret string as a number in that base.  If base isn't
  present, base 10 is used.  base must be between 2 and 16 (inclusive).
  If the base used is not 10, floating point is not recognized.

so

  (char-to-string (string-to-number "0a" 16)) => "\n"

    Juanma

Is X'ABCD' notation of general interest? (was: Converting strings of hexadecimal numbers to the respective bytes)

[florian]

Juanma,

>   (string-to-number string &optional base)

many thanks. I swear I did RTFM, or rather rummage it, but I obviously
missed that important bit.

Your hint was just what I lacked, and I used it to program two small
functions for converting strings in X'ABCD' notation into their binary
form and vice versa, along with two functions to do the same with file
contents (i.e. file contents to X'ABCD' and vice versa).

The X'ABCD' notation represents arbitrary bytes as their two-digit
hexadecimal representation, which produces strings that have double
the size and can be passed over the command line (SQLite uses this).
Does anybody think that functions to handle that (I have round-trip
tested them) would be of general interest, so that posting them on
gnu.emacs.sources would make sense? (Just don't want to clutter public
space needlessly.)

Many thanks again, and best regards,

Florian

Re: Is X'ABCD' notation of general interest? (was: Converting strings of hexadecimal numbers to the respective bytes)

[Juanma Barranquero]

On Thu, Apr 9, 2009 at 12:05, florian <lorian@fsavigny.de> wrote:

> many thanks. I swear I did RTFM, or rather rummage it, but I obviously
> missed that important bit.

Don't worry, I can't count the number of times I've read and re-read
an info page, only to miss the important bit (or sometimes, and more
embarrassingly, a whole paragraph ;-)

> Does anybody think that functions to handle that (I have round-trip
> tested them) would be of general interest, so that posting them on
> gnu.emacs.sources would make sense? (Just don't want to clutter public
> space needlessly.)

Just post them. That's what gnu.emacs.sources is for.

    Juanma

Re: Is X'ABCD' notation of general interest?

[Thien-Thi Nguyen]

() florian <lorian@fsavigny.de>
() Thu, 9 Apr 2009 03:05:03 -0700 (PDT)

   The X'ABCD' notation represents arbitrary bytes as their
   two-digit hexadecimal representation, which produces strings
   that have double the size and can be passed over the command
   line (SQLite uses this).  Does anybody think that functions to
   handle that (I have round-trip tested them) would be of general
   interest, so that posting them on gnu.emacs.sources would make
   sense? (Just don't want to clutter public space needlessly.)

You may be interested in knowing that Emacs can `read' hex if the
number has the "#x" prefix.

(let ((standard-input "#x0a"))
  (read))
=> 10

Something like the following (completely untested) code might be
faster than using `string-to-number':

(defvar small "#x......"
  "Fixed-size buffer holding a small hex number to be `read'.")

(defun next-small-X-quote-hex-quote-number ()
  "Return the next parsed X'HEX-NUMBER' or nil."
  (let ((ok (- (length small) 2))  ;; -2 for "#x"
        b e)
    (when (re-search-forward "X'[0-9a-fA-F]+'" nil t)
      (setq b (+ 2 (match-beginning 0))
            e (1- (point)))
      (if (not (< ok (- e b)))
          ;; "fast path" (one hopes, one wonders)
          (do ((w 2 (1+ w)))
              ((= b e) (let ((standard-input small))
                         (read)))
            (aset small w (char-before (incf b))))
        ;; fallback for big strings
        (string-to-number (buffer-substring b e) 16)))))

This is the limit (performance wise) if the buffer is not to be
modified.  If the buffer can be modified, you can go even faster
if you rewrite the "X'" as "x#" directly and set `standard-input'
to `(current-buffer)' (plus usual performance tweaks).  Of course,
if speed matters not, then no worries, feel free to ignore...

thi

Re: Is X'ABCD' notation of general interest?

[Nikolaj Schumacher]

Thien-Thi Nguyen <ttn@gnuvola.org> wrote:

> Something like the following (completely untested) code might be
> faster than using `string-to-number':

If you try to micro-optimize, you really should do some
benchmarking... ;)

`string-to-number' is a C function.  It is nearly impossible to write
anything faster in Lisp.  On my system `string-to-number' is more than 10x
faster than your function.


regards,
Nikolaj Schumacher

Re: Is X'ABCD' notation of general interest?

[Thien-Thi Nguyen]

() Nikolaj Schumacher <me@nschum.de>
() Sun, 12 Apr 2009 23:25:58 +0200

   If you try to micro-optimize, you really should do some
   benchmarking... ;)

   `string-to-number' is a C function.  It is nearly impossible to write
   anything faster in Lisp.  On my system `string-to-number' is more than 10x
   faster than your function.

Cool.

thi