Fix the destroyed disk image partition table in a raw image file using Emacs.

Fix the destroyed disk image partition table in a raw image file using Emacs.

[Hongyi Zhao]

I've two disk image files whose partuuid info are shown as below based
on the method discussed here on
<https://unix.stackexchange.com/questions/419104/what-is-partuuid-from-blkid-when-using-msdos-partition-table>:

$ dd if=openwrt-19.07.7-x86-64-combined-ext4.img bs=1 count=4 skip=440
2>/dev/null | od -t x4 -An
eb3ae1bb
$ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
761b36b0

In order to change the partuuid of the 2nd disk image to the value of
the 1st, I do the following operation:

$ dd if=openwrt-19.07.7-x86-64-combined-ext4.img of=disk.img bs=1
count=4 skip=440 seek=440
4+0 records in
4+0 records out
4 bytes copied, 0.000195385 s, 20.5 kB/s
$ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
eb3ae1bb

But it turns out that the disk.img will be destroyed by the above
operation. So, the only usable info for now is the following data
snippet extracted from the corresponding place of the original image:

$ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
761b36b0

It's well known that Emacs is an 8-bit clean editor which makes it
safe to edit binary files. So, I want to whether I can fix the above
destroyed disk image file with Emacs for my scenario.

Regards,
HY
-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Polytechnic University of Science and Technology engineering
NO. 552 North Gangtie Road, Xingtai, China

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Jean Louis]

* Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11 16:35]:
> It's well known that Emacs is an 8-bit clean editor which makes it
> safe to edit binary files. So, I want to whether I can fix the above
> destroyed disk image file with Emacs for my scenario.

There is hexl library:

;; This package implements a major mode for editing binary files.  It uses
;; a program called hexl, supplied with the GNU Emacs distribution, that
;; can filter a binary into an editable format or from the format back into
;; binary.  For full instructions, invoke `hexl-mode' on an empty buffer and
;; do M-x `describe-mode'.

But I guess you need `hexl' program.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://rms-support-letter.github.io/

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Dan Hitt]

On Sun, Apr 11, 2021 at 6:41 AM Jean Louis <bugs@gnu.support> wrote:

> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11 16:35]:
> > It's well known that Emacs is an 8-bit clean editor which makes it
> > safe to edit binary files. So, I want to whether I can fix the above
> > destroyed disk image file with Emacs for my scenario.
>
> There is hexl library:
>
> ;; This package implements a major mode for editing binary files.  It uses
> ;; a program called hexl, supplied with the GNU Emacs distribution, that
> ;; can filter a binary into an editable format or from the format back into
> ;; binary.  For full instructions, invoke `hexl-mode' on an empty buffer
> and
> ;; do M-x `describe-mode'.
>
> But I guess you need `hexl' program.
>
> --
> Jean
>

Hi Jean,

hexl-mode works for me on my xubuntu 20.04 system, and i do not have a
program called 'hexl'.

Of course, i don't know if my system is secretly using some other program
which does the same thing as hexl.

dan


>
> Take action in Free Software Foundation campaigns:
> https://www.fsf.org/campaigns
>
> Sign an open letter in support of Richard M. Stallman
> https://rms-support-letter.github.io/
>
>
>

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

On Sun, Apr 11, 2021 at 10:26:26AM -0700, Dan Hitt wrote:
> On Sun, Apr 11, 2021 at 6:41 AM Jean Louis <bugs@gnu.support> wrote:
> 
> > * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11 16:35]:
> > > It's well known that Emacs is an 8-bit clean editor which makes it
> > > safe to edit binary files. So, I want to whether I can fix the above
> > > destroyed disk image file with Emacs for my scenario.
> >
> > There is hexl library:
> >
> > ;; This package implements a major mode for editing binary files.  It uses
> > ;; a program called hexl, supplied with the GNU Emacs distribution, that
> > ;; can filter a binary into an editable format or from the format back into
> > ;; binary.  For full instructions, invoke `hexl-mode' on an empty buffer
> > and
> > ;; do M-x `describe-mode'.
> >
> > But I guess you need `hexl' program.
> >
> > --
> > Jean
> >
> 
> Hi Jean,
> 
> hexl-mode works for me on my xubuntu 20.04 system, and i do not have a
> program called 'hexl'.

Look around somewhere (Debian here)

  /usr/lib/emacs/<version>/x86_64-linux-gnu/hexl

Replace <version> by your Emacs version and x86_64-linux-gnu by some
appropriate architecture string.

> Of course, i don't know if my system is secretly using some other program
> which does the same thing as hexl.

It's all in the open ;-)

Hint: apt-file search finds such things, even if you haven't installed
a package. Package apt-file. Very much recommended. 

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Dan Hitt]

On Sun, Apr 11, 2021 at 11:45 AM <tomas@tuxteam.de> wrote:

> On Sun, Apr 11, 2021 at 10:26:26AM -0700, Dan Hitt wrote:
> > On Sun, Apr 11, 2021 at 6:41 AM Jean Louis <bugs@gnu.support> wrote:
> >
> > > * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11 16:35]:
> > > > It's well known that Emacs is an 8-bit clean editor which makes it
> > > > safe to edit binary files. So, I want to whether I can fix the above
> > > > destroyed disk image file with Emacs for my scenario.
> > >
> > > There is hexl library:
> > >
> > > ;; This package implements a major mode for editing binary files.  It
> uses
> > > ;; a program called hexl, supplied with the GNU Emacs distribution,
> that
> > > ;; can filter a binary into an editable format or from the format back
> into
> > > ;; binary.  For full instructions, invoke `hexl-mode' on an empty
> buffer
> > > and
> > > ;; do M-x `describe-mode'.
> > >
> > > But I guess you need `hexl' program.
> > >
> > > --
> > > Jean
> > >
> >
> > Hi Jean,
> >
> > hexl-mode works for me on my xubuntu 20.04 system, and i do not have a
> > program called 'hexl'.
>
> Look around somewhere (Debian here)
>
>   /usr/lib/emacs/<version>/x86_64-linux-gnu/hexl
>
> Replace <version> by your Emacs version and x86_64-linux-gnu by some
> appropriate architecture string.
>

OK, i guess i do have it after all :)

Thanks for pointing out the path, and the suggestion to use apt-file.

One thing i notice about hexl, is that although it must surely be a gnu
program, based on its path, it does not understand either the --help or the
--version command line options.

Anyhow, i hope the OP can use hexl-mode if he can't find any better, more
specific tools.

dan


> > Of course, i don't know if my system is secretly using some other program
> > which does the same thing as hexl.
>
> It's all in the open ;-)
>
> Hint: apt-file search finds such things, even if you haven't installed
> a package. Package apt-file. Very much recommended.
>
> Cheers
>  - t
>

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Stefan Monnier]

>> It's well known that Emacs is an 8-bit clean editor which makes it
>> safe to edit binary files. So, I want to whether I can fix the above
>> destroyed disk image file with Emacs for my scenario.
> There is hexl library:

As well as the `nhexl-mode` available from GNU ELPA.


        Stefan

GNU ELPA problem (was: Re: Fix the destroyed disk image partition table in a raw image file using Emacs.)

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier wrote:

>> There is hexl library:
>
> As well as the `nhexl-mode` available from GNU ELPA.

This is what happens when I enter GNU ELPA and MELPA:

  Importing package-keyring.gpg...done
  Saving file /home/incal/ooa/dork.el...
  Wrote /home/incal/ooa/dork.el
  Package refresh done
  Packages that can be upgraded: 14; type ‘U’ to mark for upgrading.
  package-menu--mark-upgrades-1: Wrong type argument: package-desc, nil

I now believe this is a configuration/extension bug introduced
by YT, however the only thing I did with the intention of
changing anything around that/this area is some trivial Elisp
[last], so do you have any idea based on the above messages
where to look or what to look for? TIA.

Other than that it works fine what I can see.

;;; -*- lexical-binding: t -*-
;;;
;;; this file:
;;;   http://user.it.uu.se/~embe8573/emacs-init/elpa.el
;;;   https://dataswamp.org/~incal/emacs-init/elpa.el

(require 'close)
(require 'package)
(require 'scroll)
(require 'super)

(push '("melpa" . "https://melpa.org/packages/") package-archives)

(setq-default tabulated-list-use-header-line nil)

(let ((the-map package-menu-mode-map))
  (set-close-key        the-map)
  (set-pane-scroll-keys the-map)
  (set-vertical-keys    the-map) )

(defalias 'elpa  #'list-packages)
(defalias 'melpa #'list-packages)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Gregor Zattler]

Hi Hongyi Zhao,
* Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11; 21:34]:
> $ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
> 761b36b0
>
> It's well known that Emacs is an 8-bit clean editor which makes it
> safe to edit binary files. So, I want to whether I can fix the above
> destroyed disk image file with Emacs for my scenario.

If your emacs is able to handle the disk image in memory
(which probably means you need a 64bit emacs and enough RAM
installed) you could open the file of the disk image with

find-file-literally

   Visit file FILENAME with no conversion of any kind.

   Format conversion and character code conversion are both
   disabled, and multibyte characters are disabled in the
   resulting buffer.  The major mode used is Fundamental
   mode regardless of the file name, and local variable
   specifications in the file are ignored.  Automatic
   uncompression and adding a newline at the end of the file
   due to require-final-newline is also disabled.

And then the first thing to do would be

binary-overwrite-mode

   When Binary Overwrite mode is enabled, printing
   characters typed in replace existing text.  Newlines are
   not treated specially, so typing at the end of a line
   joins the line to the next, with the typed character
   between them.  Typing before a tab character simply
   replaces the tab with the character typed.  C-q replaces
   the text at the cursor, just as ordinary typing
   characters do.


You then can move point in the disk image to the desired
place and type in whatever you want and later save the
buffer to its file.


Ciao; Gregor
--
 -... --- .-. . -.. ..--.. ...-.-

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Hongyi Zhao]

On Mon, Apr 12, 2021 at 3:18 AM Gregor Zattler <telegraph@gmx.net> wrote:
>
> Hi Hongyi Zhao,
> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11; 21:34]:
> > $ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
> > 761b36b0
> >
> > It's well known that Emacs is an 8-bit clean editor which makes it
> > safe to edit binary files. So, I want to whether I can fix the above
> > destroyed disk image file with Emacs for my scenario.
>
> If your emacs is able to handle the disk image in memory
> (which probably means you need a 64bit emacs and enough RAM
> installed) you could open the file of the disk image with
>
> find-file-literally
>
>    Visit file FILENAME with no conversion of any kind.
>
>    Format conversion and character code conversion are both
>    disabled, and multibyte characters are disabled in the
>    resulting buffer.  The major mode used is Fundamental
>    mode regardless of the file name, and local variable
>    specifications in the file are ignored.  Automatic
>    uncompression and adding a newline at the end of the file
>    due to require-final-newline is also disabled.
>
> And then the first thing to do would be
>
> binary-overwrite-mode
>
>    When Binary Overwrite mode is enabled, printing
>    characters typed in replace existing text.  Newlines are
>    not treated specially, so typing at the end of a line
>    joins the line to the next, with the typed character
>    between them.  Typing before a tab character simply
>    replaces the tab with the character typed.  C-q replaces
>    the text at the cursor, just as ordinary typing
>    characters do.
>
>
> You then can move point in the disk image to the desired
> place and type in whatever you want and later save the
> buffer to its file.

Thank you so much for telling me this feature of Emacs.

Regards
-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Polytechnic University of Science and Technology engineering
NO. 552 North Gangtie Road, Xingtai, China

Re: Fix the destroyed disk image partition table in a raw image file using Emacs.

[Hongyi Zhao]

On Mon, Apr 12, 2021 at 3:18 AM Gregor Zattler <telegraph@gmx.net> wrote:
>
> Hi Hongyi Zhao,
> * Hongyi Zhao <hongyi.zhao@gmail.com> [2021-04-11; 21:34]:
> > $ dd if=disk.img bs=1 count=4 skip=440 2>/dev/null | od -t x4 -An
> > 761b36b0
> >
> > It's well known that Emacs is an 8-bit clean editor which makes it
> > safe to edit binary files. So, I want to whether I can fix the above
> > destroyed disk image file with Emacs for my scenario.
>
> If your emacs is able to handle the disk image in memory
> (which probably means you need a 64bit emacs and enough RAM
> installed) you could open the file of the disk image with
>
> find-file-literally
>
>    Visit file FILENAME with no conversion of any kind.
>
>    Format conversion and character code conversion are both
>    disabled, and multibyte characters are disabled in the
>    resulting buffer.  The major mode used is Fundamental
>    mode regardless of the file name, and local variable
>    specifications in the file are ignored.  Automatic
>    uncompression and adding a newline at the end of the file
>    due to require-final-newline is also disabled.
>
> And then the first thing to do would be
>
> binary-overwrite-mode
>
>    When Binary Overwrite mode is enabled, printing
>    characters typed in replace existing text.  Newlines are
>    not treated specially, so typing at the end of a line
>    joins the line to the next, with the typed character
>    between them.  Typing before a tab character simply
>    replaces the tab with the character typed.  C-q replaces
>    the text at the cursor, just as ordinary typing
>    characters do.
>
>
> You then can move point in the disk image to the desired
> place and type in whatever you want and later save the
> buffer to its file.
>
>
> Ciao; Gregor
> --
>  -... --- .-. . -.. ..--.. ...-.-

If the 'conv=notrunc' option of 'dd' is used, the following is a pure
*nix tools based restore method given on comp.unix.shell newsgroup by
Helmut Waitzmann. For the more detailed discussion, see
<https://groups.google.com/g/comp.unix.shell/c/-4jQ-6tkOgs/m/FZl_96n-AQAJ>:

<quote>
[The 'conv=notrunc' option of 'dd']

>Assuming I have used this option, what exact restore command should
>I use?


First you should check, that the command


{
printf '%s\n' 'ibase=16' &&
printf '%s\n' b0 36 1b 76 |
LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
} | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
xargs -E '' -- printf '%b' |
od -v -A n -t x4

yields the output '761b36b0'.  If that's the case, you could have
done

{
printf '%s\n' 'ibase=16' &&
printf '%s\n' b0 36 1b 76 |
LC_ALL=POSIX tr -- '[:lower:]' '[:upper:]'
} | bc | xargs -E '' -- printf \''\\%.4o'\''\n' |
xargs -E '' -- printf '%b' |
dd of=disk.img bs=1 seek=440 conv=notrunc

to restore the original partuuid.
</quote>

To be frank, the internal logic of the above code is still unclear to
me. I post it here for your reference.

Regards
-- 
Assoc. Prof. Hongyi Zhao <hongyi.zhao@gmail.com>
Theory and Simulation of Materials
Hebei Polytechnic University of Science and Technology engineering
NO. 552 North Gangtie Road, Xingtai, China