Anyone here ever use Scalpel to recover deleted files ?

[eppur_se_muova]

This discussion thread was locked by steve2470 (a host of the Computer Help and Support group).

I'm a little confused by the instructions:

# For each file type, the configuration file describes the file's
# extension, whether the header and footer are case sensitive, the
# min/maximum file size, and the header and footer for the file. The
# footer field is optional, but extension, case sensitivity, size, and
# footer are required.

So ... is the footer field optional or not ? Or is it the header ?

Is there something better than scalpel ?

[left-of-center2012]

1. Nope - never heard of it

Often when I can't figure something out, I'll Google a question.

"How do I _____ on Scalpel ?"

[eppur_se_muova]

2. That's kind of how I found out about scalpel ...

apparently, it hasn't been updated in a long time, so maybe no one's using it anymoe.

[gvstn]

3. You're on Linux?

I'm not familiar with any recovery software for Linux but if you know how to burn a CD there are recovery programs that will run from CD (or possibly USB but CD is most reliable). I have an old version of Minitools Power Data recovery that will recover for free if you would want that to try.

I just had a major recovery problem and tried about 10 recovery programs. I don't know what the problem was but I eventually got the files back but it took 24 hours and all the filenames were lost. I haven't had a recovery problem like that in 10 years of so. But they were all Windows programs. The minitools is the only one I can think of that will create a bootable disk.

[eppur_se_muova]

4. I've been searching the Web already ...

PhotoRec and TestDisk seem to be the tools most often used. The problem is "the usual" one ... I can't recover the partition or its directory (tried debugfs with no luck) but I can recover a buttload of files, thousands of them (~1700 directories with 500 files each), mostly filled with log files from updates and such, but with all filenames lost. Just trying to delete, say, all .html files gets me an "arguments list too long" error. I need a recovery method that extracts info from the files automagically, and scalpel (maybe some later programs built on top of it ? anybody know ?) seems to offer a way to do that. I still don't know if it will work with the files I want most -- they were created by a "niche" program, and don't have standard extensions.

BTW, I've booted this computer off a separate partition, so live CD's and boot CDs aren't really needed. The partition happened to have Scientific Linux on it -- not my first choice, but I tried it out a few months ago and it was still on the partition. And yes, I know to keep the corrupted disk unmounted to prevent further data loss. I have about three weeks to work on this before I will have to start rerunning a job over from scratch rather than trying to recover the checkpoint file, and a few weeks after that before there's no further point at all, so I'm taking the time to explore my options thoroughly.

For anyone in a similar situation, a good starting point is https://help.ubuntu.com/community/DataRecovery . Note scalpel is one of the options listed, but instructions for most are minimal, eg: "Use any data carving tool to search the output image for files." Well, data carving tools is what I'm trying to find. Instructions tend to be written at the level of veteran file system gurus, which doesn't help me much. Something with a simple point-and-click interface and multiple undelete options, like the old Norton Utilities for Macintosh would be great -- that was from what, 1990 ?

[gvstn]

5. Your problem is exactly like mine--complete destruction of the file table.

The things I tried with no success at even recovering raw data were:

TestDisk is usually fairly good at restoring a partition have you tried that already? PhotoRec didn't work either,also usually pretty good.

Partition Wizard (Windows program is not bad at recovering some partitions. I think you can create a bootable disk if you don't run Windows at all.)

PowerData Recovery (http://www.mediafire.com/download/i4d4e2allk4ynd7/pdr6nolimitfree.exe This a free version of the bootable disk--You must select "I am a Home USER" when starting the program or it won't be free to recover your files.) Is fairly simple point click. Guide: http://www.powerdatarecovery.com/damaged-partition-recovery.html

I think the above three might be worth a try to rule them out.

I can't remember the one that finally worked (I torrented a program) but it only gave me exactly what you currently have with the thousands of log files etc. It happened to be the partition where I keep media files so I just saved the biggest movie and music files and dumped the rest.

[eppur_se_muova]

6. TestDisk/PhotoRec is what got me the 850K (or so) nameless files.

Thanks for the link -- I'm still trying out other options from that link I posted earlier, so there's still a lot to try. I just wish I could find some *success* stories from other people so I could know what is most likely to work.

It make a big difference if you can save a disk image, but I don't *really* need to do that -- I do have a 1TB drive I can use to creates a few copies of that partition before going ahead and finishing the reinstall. I've just got too many problem computers to deal with them all at once.

[gvstn]

7. Good Luck.

Like I said it was the worst partition destruction I have had in 15 years. It happened instantly and no program could properly recover the file system index (?). All the unnamed files are a nightmare and more than a human can comprehend.

Maybe a Linux forum could get you better help with Scalpel. It just sounds like you are down to reading RAW data and trying to reconstruct files.

***Gparted is the weakest of the partition programs but you could that a quick try but if TestDisk couldn't help I doubt Gparted will see anything.

[w0nderer]

8. A little

I've used scalpel a little (still do)

Unless you are looking for a File Format (ff) not in the conf file already
forget about the footer

http://www.linux-magazine.com/Online/Features/Recovering-Deleted-Files-with-Scalpel
has an ok intro into how to add file formats

Scalpel is now (i believe) a part of "the sleuth kit" (tsk) http://forensicswiki.org/wiki/The_Sleuth_Kit
http://www.sleuthkit.org/

which can use autopsy
http://www.sleuthkit.org/autopsy/desc.php

-----------------------
now...____FROM MY UNDERSTANDING____ (didn't verify this with src code) YMMV maintain backups
the footer controls the end of the filecarving
ie....header is the intro part of a file and footer the end

hit header for mp3
start copy to recovered mp3
hit footer for mp3...stop copy

as the one page above shows on how to do headers...you can also do footers (hexdump em) find a pattern, put it in
----
something better?
any tool is as good as the user, scalpel is pretty good, especially with tsk and autopsy

for windows winhex (with the expensive forensic licence) is good, so is encasa

for linux
google filecarving or forensic distribution (depending on purpose)
if you need to snag out an area between inode ranges ifile and ffile from tsk can help


at this point i'm assuming you have copied the area or are mounting it ReadOnly
if this still is a live file system...depending on circumstances you could be way out of luck


hope this helps

[eppur_se_muova]

9. Thanks, looks like some useful info in that article.

I tried downloading Sleuth Kit and Autopsy, had a little trouble because I'm not used to RHEL-based distros doing things "their way". Will try again later.

That article suggests later apps based on Scalpel don't incorporate all its features, which seems to be my experience so far. I'm going to need to add custom headers to the list.

Thanks again for the info. If I can recover these files I'll save a few CPU-weeks of calculation, otherwise lost to a moment's confusion about which computer I was working on.

[w0nderer]

10. :-) slackware based

so i'm used to 'compile from source'



what ever you do, the less you run hdd you need to rescue from , the better
even better
image it to another hdd
ddrescue is good for that
make one image file then work against it


working against a live file system is like pulling a fingerprint from someone playing with playdoh

[eppur_se_muova]

11. Well, "compile from source" drives me nuts.

It turns out that Scalpel requires two other packages to compile ... both available from one guy ... on different Web sites. Of course, those packages require other packages .... five just to "configure" one library ... and after downloading those, of course I just get another error message (autopoint - command not found) when I try to run configure.

When you're trying to do an emergency recovery, you need tools that can be downloaded in one go, so you can focus on recovering the files, not diddling the recovery tools. R-Linux looks like it *should* do that, but it doesn't have all the commands the manual says it should. Apparently the manual for the freeware version was "edited down" from the manual for the $$$ version, and they "accidentally" did a sloppy job of it.

There are at least two packages worth looking at, both $$$ ware. If I had known to buy them at the time I bought the computer, I would have. Can't afford them now, so I keep trying to find something free that will work, and that doesn't require an experienced programmer & sysadmin's expertise to get running.

[w0nderer]

12. compile from source shouldn't be done WHEN you have to rescue

it should be done before, and then burned to a rescue disc
or on the side on another machine and burned to a rescue disc

anything else is kinda like calling AAA and getting a membership when you are on the roadside broken down

compile from source (any forensic or data recovery starts before it's needed by preparing for the task)
btw..if you can come up with a way, to create a binary download (non interpreted (perl lua python) ) that is
universally applicable to different linux systems(arm/ppc/x86-32/64/sparc // then add in the other hw configuration, scsi, sata, pata) you'll beat everyone in the linux community

that being said,i didn't suggest that YOU compile from source. i pointed out that i came from that area.

for forensic or datarecovery use one of the big live cd / dvd rescue discs..they usually have scalpel
and ddrescue (and dd3rescue or any of the forensically enhanced dd's) or at least dd which will let you image the hdd, from which image you'll then work

kali linux being one(live cd/dvd) that has some forensic tools (and others) (doesn't at last check have scalpel but does have foremost which does pretty much the same) it is not 'beginner easy to use'

http://www.howtogeek.com/howto/15761/recover-data-like-a-forensics-expert-using-an-ubuntu-live-cd/
is step by step on ubuntu

i've tried to drop a fast intro course on you here in these posts
however data rescue, forensics are not 'average tasks' in general, the are programmer/specialist/sysadmin tasks (primarily the 2 latter)

[eppur_se_muova]

13. Yeah, I'm aware of the situation in that last sentence. Not average user stuff.

One post on a Web site dealing with a similar query suggested that this is a job for pros. Unfortunately, that means $$$, and since this is all-nonprofit stuff, that's out of the question.

I wish I had known what tools it would have been nice to have on hand beforehand, but since that doesn't come with the OS, and it's not common knowledge ... there's just no equivalent to Norton Utilities for Linux, or I would have bought it. Long in the past, I recovered numerous files with the old NUM under the old Mac OS. With OSX, I've gotten "Disk Tools cannot repair this disk" three times, and there's nowhere to go from there. (Norton System Tools for Macintosh fell behind the times, AFAICT).

[w0nderer]

14. is this a linux box or a mac os x box?

there are some mac os x progs that'll do data recovery and file carving

my last os x box is antique 10.3.9 but on it i used an older version of macforensicslab (subrosa soft) not free
(in fact..rather expensive)

either way principle is the same
image drive
copy image elsewhere
backup image
work on copy of image with filecarver or hexeditor

helix http://www.e-fense.com/products.php find the helix3 link for the free one
is a pretty good for forensics...probably better than kali for forensics (kali beats it on pentest), but both are worth toolkit disks
(old intro to helix)

if it's a mac and it's new enough to have x86 processor (not ppc) it should be able to run either cd/dvd from x86-64 or a mac image of helix or kali

norton utils..blast from the past
disked (disk hex editor) is another from the old days (also not on mac or linux)

[eppur_se_muova]

20. Autopsy is now supported by CAINE -- interface is fairly easy.

http://www.caine-live.net/

Only problem is, I've recovered some files with foremost (precursor of scalpel) that Autopsy couldn't see at all.

Thanks for the links from earlier, they really helped, but it took me awhile to sort through all those new tools.

[a2liberal]

15. Do you know generally about the file format

of the program whose files you're looking for? And how large are they? I would suggest grepping for "magic" strings that are likely to be in the file on the whole drive/image, and if you have hits use dd to pull out that block and surrounding blocks. If the file is small enough and/or not fragmented, you may get lucky and find it all together.

(One way to find out about the file format is to create a new file in the program, and then run hexdump -C on it to look at it)

As for your argument too long problem trying to delete html files, try xargs. However, if your file format wasn't supported by PhotoRec I doubt it'll be in the files it found. TestDisk may have an undelete that recovers files regardless of type (don't remember, but I have actually restored filesystems wholesale with it on lucky occasions)

[eppur_se_muova]

16. Thanks, that's how I seem to have found some of my files ...

... still trying to recover all of them (there's a lot -- hundreds).

Fortunately, I have two functioning Linux partitions on the same disk as the one that got trashed -- and all about the same size, so block size is the same. The files I am looking for all have precisely the same length, and I can check their contents by running the program that created them (slow, but effective). I found a viable hex search string by comparing several files generated by the same program on one of the bootable partitions.

I've fallen back to using the older program Foremost -- the instructions (and configuration file!) are apparently the original from which those of Scalpel were modified, minus the confusing mistake. And Foremost compiled with nary a glitch. Just need some more time and I think I'll have recovered all I can, and can resume my projects.

[a2liberal]

17. Awesome!

Glad you found a way to recover some of them at least

[Vamksery]

22. data recovery tools for iPhone

Do you have other suggestions ? before use the apple data recovery tools ,we also need to download a new version of iTunes on computer ?

[gopiscrap]

23. welc

[fleabiscuit]

19. No clue from me, but have you tried Ars Technica forums? They have helped me a lot.

Open forum - https://arstechnica.com/civis/