Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Sheep
Jul 24, 2003

fletcher posted:

HP N40L! Too small for your 8 drive requirement though.

The case is good but the CPU in the N40L (or N54L, the more expensive model) is a dual core processor from 2011 that runs at 1.5Ghz (2.2 or N54L). For the about the same price you can get a Gen10 with an Opteron X3421 that gives you 4 cores at 2Ghz (turbo to 3.4). It's like having three N40Ls in one box.

I think I paid like $350 on Amazon for a Gen10 last year and it has been a dream. I still keep my N54L because the form factor is nice and it just works, but it's useless for anything remotely CPU intensive, including writing to LUKS encrypted drives via SMB and such.

Edit: Here's a passmark comparison.

Sheep fucked around with this message at 16:07 on Mar 9, 2019

Adbot
ADBOT LOVES YOU

astral
Apr 26, 2004

Nam Taf posted:

You care way too much about this file list for the amount of backups you held of it.

WWWWWWWWWWWWWWWWWW posted:

It's an 8 TB. I bought it like three months ago. I hadn't gotten around to setting up a RAID6 with it yet.

There's a good saying that's applicable here...

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.
you gonna help me or not

redeyes
Sep 14, 2002

by Fluffdaddy
Send the drive in for data recovery: https://www.300dollardatarecovery.com/data-recovery-prices/

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.
As I said before I'm not spending $500-$1000 on getting data back when all I need are the file names when I'm one mouse click away from obtaining that if that goddamn EDB analyzer website wasn't messed up

astral
Apr 26, 2004

WWWWWWWWWWWWWWWWWW posted:

you gonna help me or not

Sure! It's: "RAID is not backup!"

bobfather
Sep 20, 2001

I will analyze your nervous system for beer money

WWWWWWWWWWWWWWWWWW posted:

As I said before I'm not spending $500-$1000 on getting data back when all I need are the file names when I'm one mouse click away from obtaining that if that goddamn EDB analyzer website wasn't messed up

Your stuff is gone, you can’t remember what any of it was, nor did you watch any of it before it was lost. It was just an unwatched collection, so it doesn’t matter that it’s gone. Go to the top 250 IMDB page and base your future collection on that.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.
Yeah I'm literally one mouse click away from finding out what all my lost data was so I'll pass on the useless passive-aggressive suggestions, thanks. Legit thanks to the non-assholes that actually tried to help me though.

bobfather
Sep 20, 2001

I will analyze your nervous system for beer money

WWWWWWWWWWWWWWWWWW posted:

Yeah I'm literally one mouse click away from finding out what all my lost data was so I'll pass on the useless passive-aggressive suggestions, thanks. Legit thanks to the non-assholes that actually tried to help me though.

My friend, the help you need goes beyond PowerShell.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

bobfather posted:

My friend, the help you need goes beyond PowerShell.

I know, I bought the piece of software that can do it just fine; I know it because the trial version showed me, but the drat company went out of business so I couldn't register it.

If you mean mental help, I completely agree

mystes
May 31, 2006

If that program just looks at the windows search index, aren't there other ways to get that data?

Moey
Oct 22, 2010

I LIKE TO MOVE IT

Sheep posted:

The case is good but the CPU in the N40L (or N54L, the more expensive model) is a dual core processor from 2011 that runs at 1.5Ghz (2.2 or N54L). For the about the same price you can get a Gen10 with an Opteron X3421 that gives you 4 cores at 2Ghz (turbo to 3.4). It's like having three N40Ls in one box.

I think I paid like $350 on Amazon for a Gen10 last year and it has been a dream. I still keep my N54L because the form factor is nice and it just works, but it's useless for anything remotely CPU intensive, including writing to LUKS encrypted drives via SMB and such.

Edit: Here's a passmark comparison.

I shoved a e3-1275 into my Gen8 microserver. Gotta add some 30mm fans to the passive heatsink if you are gonna push it tho.

BlankSystemDaemon
Mar 13, 2009



mystes posted:

If that program just looks at the windows search index, aren't there other ways to get that data?
INDXParse can be used to parse $I30 files (which contain much the same values as what's found in the Master File Table, and which are generated by Windows Search - unfortunately that requires that someone has used Windows Search at some point). Alternatively, the Master File Table can sometimes be recovered from an NTFS drive without having to recover the data itself, since it always exists (unlike $I30 files) and since it's usually written in more than one place.

The proper way to do it is to run 'recoverdisk' in FreeBSD (or ddrescue in most Linux distributions, I think?) as it works at a level below the filesystem, and just tries to read the disk forever until you stop it or it has produced a disk image without any sectors which return error codes other than 0. Then you can mount that image and use the above tools or simply copy whichever files still work.

BlankSystemDaemon fucked around with this message at 11:31 on Mar 10, 2019

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

mystes posted:

If that program just looks at the windows search index, aren't there other ways to get that data?

That's all it does and that's all I need, yep.

I found one other program that does it, and it's free no less, but as the creator admits, "it often crashes on large EDB files" and my EDB file is 14 gigs, so out of two programs I found, one crashes, and the other works but the company went out of business so I can't buy the license. :negative:

If it helps uncover this mystery or something, the data it needs to read is from inside SystemIndex_PropertyStore and then 33-System_Item_Url and 11-System_FileName. It's all there, it's just that the freeware version won't show me everything.

I'd pay $85 just for the goddamn crack of the program.

I live in a country where I am lucky to get 100kbps downloads, so I bring my external hard drives when I go to visit my cousin in America, and we download stuff the entire ~month that I am there. He also transfers a bunch of stuff he'd think I love. Sometimes we even do this over the mail. This is a major reason why I don't know what's on the drives, but while also being important that I know what's on them. I know it's a unique situation, and seems dumb, but I would really like to get these file names.

G-Prime
Apr 30, 2003

Baby, when it's love,
if it's not rough it isn't fun.
https://github.com/libyal/libesedb/...format.asciidoc There's the file format specification. Write a parser and extract it. If somebody else had a bulletproof solution for you, they'd likely have given it by now.

Edit: I think this could have come off harsher than I meant it to. Losing data sucks, we all know it. You didn't have backups, and while you were working on setting up RAID, RAID isn't a backup either. The overwhelming majority of people in this thread have had a catastrophic data loss and understand how much it sucks. That said, your options are find a tool that can solve your problem (and clearly, you haven't, because you can't get a licensed version of the tool that MAY solve it, and the other tool just plain can't deal with a DB that large), write something yourself that can solve it, pay for data recovery, or suck it up and move on with life. Option 1 hasn't worked out, if you're capable of option 2 I would recommend considering it, if you actually want the data back option 3 is the best course of action.

Just go for option 4.

G-Prime fucked around with this message at 18:43 on Mar 10, 2019

mystes
May 31, 2006

WWWWWWWWWWWWWWWWWW posted:

That's all it does and that's all I need, yep.

I found one other program that does it, and it's free no less, but as the creator admits, "it often crashes on large EDB files" and my EDB file is 14 gigs, so out of two programs I found, one crashes, and the other works but the company went out of business so I can't buy the license. :negative:

If it helps uncover this mystery or something, the data it needs to read is from inside SystemIndex_PropertyStore and then 33-System_Item_Url and 11-System_FileName. It's all there, it's just that the freeware version won't show me everything.

I'd pay $85 just for the goddamn crack of the program.

I live in a country where I am lucky to get 100kbps downloads, so I bring my external hard drives when I go to visit my cousin in America, and we download stuff the entire ~month that I am there. He also transfers a bunch of stuff he'd think I love. Sometimes we even do this over the mail. This is a major reason why I don't know what's on the drives, but while also being important that I know what's on them. I know it's a unique situation, and seems dumb, but I would really like to get these file names.
If you have the windows.edb file available, can't you just look at it in ESEDatabaseView? I just tried and if you look at the SystemIndex_PropertyStore table, the paths of all the files appear to be in the 4443-System_ItemPathDisplay column, but maybe your file is too big for it to work?

mystes fucked around with this message at 20:56 on Mar 10, 2019

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

G-Prime posted:

https://github.com/libyal/libesedb/...format.asciidoc There's the file format specification. Write a parser and extract it. If somebody else had a bulletproof solution for you, they'd likely have given it by now.

Edit: I think this could have come off harsher than I meant it to. Losing data sucks, we all know it. You didn't have backups, and while you were working on setting up RAID, RAID isn't a backup either. The overwhelming majority of people in this thread have had a catastrophic data loss and understand how much it sucks. That said, your options are find a tool that can solve your problem (and clearly, you haven't, because you can't get a licensed version of the tool that MAY solve it, and the other tool just plain can't deal with a DB that large), write something yourself that can solve it, pay for data recovery, or suck it up and move on with life. Option 1 hasn't worked out, if you're capable of option 2 I would recommend considering it, if you actually want the data back option 3 is the best course of action.

Just go for option 4.

Thank you. While I've done things like running python scripts and stuff like that, I've never wrote a parser before and I tried googling how to do that and I am not sure it's something I can figure out. Not out of laziness, but out of a genuine inability to understand it.

mystes posted:

If you have the windows.edb file available, can't you just look at it in ESEDatabaseView? I just tried and if you look at the SystemIndex_PropertyStore table, the paths of all the files appear to be in the 4443-System_ItemPathDisplay column, but maybe your file is too big for it to work?


Thank you for the reply.

Yeah, that program is the one that crashes on me when I try to load it. It says it doesn't work well with large DB files. Mine is 14GB :(

Windows Search Index Analyzer is the one that doesn't crash, but I can only get the first 100 file names because the company went out of business and won't let me buy the license

mystes
May 31, 2006

WWWWWWWWWWWWWWWWWW posted:

Thank you for the reply.

Yeah, that program is the one that crashes on me when I try to load it. It says it doesn't work well with large DB files. Mine is 14GB :(

Windows Search Index Analyzer is the one that doesn't crash, but I can only get the first 100 file names because the company went out of business and won't let me buy the license
If you can install python and the python libesedb bindings (you can probably install them somehow in windows, otherwise in wsl or booting from an ubuntu usb stick run "apt-get install python3-libesedb").

Run the following (with python3 if using wsl/ubuntu), changing the paths in the top two lines as appropriate:
code:
edb_file = "/path/to/Windows.edb"
output_file = "/path/to/output_file.txt"

import pyesedb
d = pyesedb.open("/home/jrothenberg/samba/Windows.edb")
t = [t for t in d.tables if t.name=="SystemIndex_PropertyStore"][0]
cnum = [i for i,c in enumerate(t.columns) if c.name=="4443-System_ItemPathDisplay"][0]
f = open(output_file, "w")
for r in t.records:
	f.write(r.get_value_data_as_string(cnum))
	f.write("\n")
f.close()
I don't know whether this will work on huge files but at least it won't try to display the whole thing which should help.

Edit: I hardcoded the number 4443 but I have no idea whether that depends on windows version or something so it might need to be changed or not hardcoded if that doesn't work.

mystes fucked around with this message at 22:00 on Mar 10, 2019

Takes No Damage
Nov 20, 2004

The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far.


Grimey Drawer
Is there a cloud backup solution that's recommended for FreeNAS in particular? I see it comes with a few plugins already available like CrashPlan and Amazon S3, but I also haven't heard great things about FreeNAS plugin stability in general (mostly with their Plex flavor, however). Really just looking for something to backup a couple TB offsite in case the house burns down, trying to figure out what would have the best interface directly with FreeNAS. Can I assume any solution would come with scheduling and version-ing?

Hed
Mar 31, 2004

Fun Shoe
I also want to know the answer but in my opinion CrashPlan software is really bad and requires a lot of janitoring to keep working.

Rooted Vegetable
Jun 1, 2002

Hed posted:

I also want to know the answer but in my opinion CrashPlan software is really bad and requires a lot of janitoring to keep working.

I'm actually sat here wondering why I'm giving it $10/mo to use half my ram ..
Despite it doing a restore as I speak.

I'm actually wondering about setting up an off-site backup at my MILs

Sheep
Jul 24, 2003

Heners_UK posted:

I'm actually sat here wondering why I'm giving it $10/mo to use half my ram ..
Despite it doing a restore as I speak.

Because Java.

The Linux client is better (and has no arbitrary upload/download speed caps).

Rooted Vegetable
Jun 1, 2002

Sheep posted:

Because Java.

The Linux client is better (and has no arbitrary upload/download speed caps).

Wait, what Linux client? I'm running it in Docker on Linux but it's still the JRE using GUI version

Chilled Milk
Jun 22, 2003

No one here is alone,
satellites in every home
I use the built in Cloud Credentials/Cloud Sync to copy off to my OneDrive and S3. Getting the OneDrive credentials is kind of a pain but it's a free TB of space. I backup my music there, because it fits and I don't particularly care if Microsoft scans all my CD rips and concert bootlegs.
Then my important stuff (docs, photos, etc) goes to S3. I should switch to B2 since it's cheaper but I already had a personal AWS account setup.

Sheep
Jul 24, 2003

Heners_UK posted:

Wait, what Linux client? I'm running it in Docker on Linux but it's still the JRE using GUI version

I mean it still uses Java on the Linux client but it's not quite as garbage as the Windows client because there aren't (what appear to be) arbitrary bandwidth caps. The Linux client will totally saturate my gigabit link both up and down whereas the Windows client goes at maybe a fifteenth of the speed.

You can run it headless though with port forwarding and some other shenanigans but honestly I find it easier to just use X forwarding because effort.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

mystes posted:

If you can install python and the python libesedb bindings (you can probably install them somehow in windows, otherwise in wsl or booting from an ubuntu usb stick run "apt-get install python3-libesedb").

Run the following (with python3 if using wsl/ubuntu), changing the paths in the top two lines as appropriate:
code:
edb_file = "/path/to/Windows.edb"
output_file = "/path/to/output_file.txt"

import pyesedb
d = pyesedb.open("/home/jrothenberg/samba/Windows.edb")
t = [t for t in d.tables if t.name=="SystemIndex_PropertyStore"][0]
cnum = [i for i,c in enumerate(t.columns) if c.name=="4443-System_ItemPathDisplay"][0]
f = open(output_file, "w")
for r in t.records:
	f.write(r.get_value_data_as_string(cnum))
	f.write("\n")
f.close()
I don't know whether this will work on huge files but at least it won't try to display the whole thing which should help.

Edit: I hardcoded the number 4443 but I have no idea whether that depends on windows version or something so it might need to be changed or not hardcoded if that doesn't work.

Thank you so, so much.

I have to keep Python 2.x installed for some reason or another (I think Sonarr or something like that? I forget; I set it up years ago). Can I install 3.0 on my Windows machine to run your script, or will it mess up whatever is using Python 2.x?

apropos man
Sep 5, 2016

You get a hundred and forty one thousand years and you're out in eight!
Are you using a Linux variant for the recovery? Don't remove python2. There is lots of stuff relying on it and your OS will start doing weird things.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

apropos man posted:

Are you using a Linux variant for the recovery? Don't remove python2. There is lots of stuff relying on it and your OS will start doing weird things.

Well the post said I could use Windows and Python 3.x to use that script to read my Windows search indexing .EDB file; that's what I was asking about.

I could do the thumb drive thing to boot from a Linux distro to read the Windows search indexing .EDB file but I can't do that when remote desktoping from my work computer :twisted:

mystes
May 31, 2006

WWWWWWWWWWWWWWWWWW posted:

Well the post said I could use Windows and Python 3.x to use that script to read my Windows search indexing .EDB file; that's what I was asking about.

I could do the thumb drive thing to boot from a Linux distro to read the Windows search indexing .EDB file but I can't do that when remote desktoping from my work computer :twisted:
You could use python2 if you just change "open" to "file", but the bigger problem is that it doesn't look like there are prebuilt binaries for the libesedb library for windows, so you would have to compile it which would probably be a pain on windows.

If you use ubuntu or WSL you can just install the package.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

mystes posted:

You could use python2 if you just change "open" to "file", but the bigger problem is that it doesn't look like there are prebuilt binaries for the libesedb library for windows, so you would have to compile it which would probably be a pain on windows.

If you use ubuntu or WSL you can just install the package.

OK so just download a bootable version of ubuntu onto a USB drive, reboot my computer with the .EDB file on it, and run your code and that's it? I've never used a Linux before (unless you count me rooting my Android).

mystes
May 31, 2006

WWWWWWWWWWWWWWWWWW posted:

OK so just download a bootable version of ubuntu onto a USB drive, reboot my computer with the .EDB file on it, and run your code and that's it? I've never used a Linux before (unless you count me rooting my Android).
It should be something like this:

1) Reboot into the ubuntu usb stick and select "try ubuntu" or something like that from the menu to get a normal ubuntu desktop
2) Open a terminal
3) Run "sudo apt-get update" and then "sudo apt-get install python3-libesedb" and type "Y" when prompted
4) Save the python code to somewhere, say /home/ubuntu/dumpedb.py
5) Open the hard drive from the file explorer and find the full path of the edb file (you can right click on it and view the properties to get the path of the folder it's in and then add the filename to it; be aware that linux is case sensitive)
6) Edit the paths in dumpedb.py based on where the edb file is and where you want the output to go (if the edb file is on your boot drive and you are using fast boot on windows the ntfs volume may get mounted read only so you might have to save the edb file to the usb drive or you can can just temporarily write it to /home/ubuntu/whatever.txt and then copy it wherever)
7) Run python3 dumpedb.py

mystes fucked around with this message at 17:09 on Mar 11, 2019

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

mystes posted:

It should be something like this:

1) Reboot into the ubuntu usb stick and select "try ubuntu" or something like that from the menu to get a normal ubuntu desktop
2) Open a terminal
3) Run "sudo apt-get update" and then "sudo apt-get install python3-libesedb" and type "Y" when prompted
4) Save the python code to somewhere, say /home/ubuntu/dumpedb.py
5) Open the hard drive from the file explorer and find the full path of the edb file (you can right click on it and view the properties to get the path of the folder it's in and then add the filename to it; be aware that linux is case sensitive)
6) Edit the paths in dumpedb.py based on where the edb file is and where you want the output to go (if the edb file is on your boot drive and you are using fast boot on windows the ntfs volume may get mounted read only so you might have to save the edb file to the usb drive or you can can just temporarily write it to /home/ubuntu/whatever.txt and then copy it wherever)
7) Run python3 dumpedb.py

I love you so much. I will have time to do this on Wednesday (working 16s tonight and tomorrow) but I will let you know how it goes. I really appreciate it.

thebigcow
Jan 3, 2001

Bully!

WWWWWWWWWWWWWWWWWW posted:

OK so just download a bootable version of ubuntu onto a USB drive, reboot my computer with the .EDB file on it, and run your code and that's it? I've never used a Linux before (unless you count me rooting my Android).

You can also just install Windows Subsystem for Linux and your choice of a handful of distros right off the Windows Store.

https://docs.microsoft.com/en-us/windows/wsl/install-win10

Harik
Sep 9, 2001

From the hard streets of Moscow
First dog to touch the stars


Plaster Town Cop
I once removed a dirty bcache when it was wedged instead of hitting the reset switch and letting it do recovery, so of course there went all the latest metadata of my btrfs.

Being that kind of stubborn idiot, I proceeded to write and contribute a fairly significant set of changes to the btrfs recovery tools to reconstruct a best-available tree and dump it.

What I'm saying is these posts are giving me PTSD.

Greatest Living Man
Jul 22, 2005

ask President Obama

thebigcow posted:

You can also just install Windows Subsystem for Linux and your choice of a handful of distros right off the Windows Store.

https://docs.microsoft.com/en-us/windows/wsl/install-win10

WSL has its own weird poo poo going on from my experience.

BlankSystemDaemon
Mar 13, 2009



Greatest Living Man posted:

WSL has its own weird poo poo going on from my experience.
Weird in what way? It works the same way as LX Zones on Solaris and the Linuxulator on FreeBSD (the latter was inspired by the first), by having a layer that does syscall, libc, and other translations from ones that Linux applications use to whichever native functions are supported by the OS.

Greatest Living Man
Jul 22, 2005

ask President Obama

D. Ebdrup posted:

Weird in what way? It works the same way as LX Zones on Solaris and the Linuxulator on FreeBSD (the latter was inspired by the first), by having a layer that does syscall, libc, and other translations from ones that Linux applications use to whichever native functions are supported by the OS.

There was one error I encountered specific to WSL Ubuntu with installing a python package that seemed to not have a solution. How does the linuxulator work? Is it possible to install in a FreeBSD jail?

mystes
May 31, 2006

Greatest Living Man posted:

There was one error I encountered specific to WSL Ubuntu with installing a python package that seemed to not have a solution. How does the linuxulator work? Is it possible to install in a FreeBSD jail?
There were initially various compatibility problems in WSL due to not implementing certain system calls, etc., but I think these have mostly gotten fixed in updates.

frh
Dec 6, 2014

Hire Kenny G to play for me in the elevator.

thebigcow posted:

You can also just install Windows Subsystem for Linux and your choice of a handful of distros right off the Windows Store.

https://docs.microsoft.com/en-us/windows/wsl/install-win10

Thank you! I've installed Ubuntu this way

mystes posted:

It should be something like this:

1) Reboot into the ubuntu usb stick and select "try ubuntu" or something like that from the menu to get a normal ubuntu desktop
2) Open a terminal
3) Run "sudo apt-get update" and then "sudo apt-get install python3-libesedb" and type "Y" when prompted
4) Save the python code to somewhere, say /home/ubuntu/dumpedb.py
5) Open the hard drive from the file explorer and find the full path of the edb file (you can right click on it and view the properties to get the path of the folder it's in and then add the filename to it; be aware that linux is case sensitive)
6) Edit the paths in dumpedb.py based on where the edb file is and where you want the output to go (if the edb file is on your boot drive and you are using fast boot on windows the ntfs volume may get mounted read only so you might have to save the edb file to the usb drive or you can can just temporarily write it to /home/ubuntu/whatever.txt and then copy it wherever)
7) Run python3 dumpedb.py

I apologize, and please bear with me as I am new to Linux. I am stuck on number 4. How do I save the python code there? I took your code, and saved it into a text document that I renamed dumpedb.py all in Windows 10 but I have no idea how to save it to that folder, as I appear to just be looking at commands:



I'm pretty stupid and thought this was gonna be some sort of virtual mode that had a GUI and a desktop and stuff, but I guess Ubuntu isn't that. No big deal, I just have absolutely no idea how to "access" /home/ubuntu/

Again sorry for these stupid questions

Adbot
ADBOT LOVES YOU

Eletriarnation
Apr 6, 2005

People don't appreciate the substance of things...
objects in space.


Oven Wrangler
If I recall correctly, it is actually possible to install a desktop environment in WSL and connect to it using VNC - it's just not there by default.

For transferring files into WSL, check /mnt. You should be able to see your host system's drives there listed by letter and accessible for copying files to/from your Linux filesystem.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply