Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
pseudorandom name
May 6, 2007

bind mounting doesn't do what Time Machine wants, either

potentially every directory on the filesystem has multiple hard links

Adbot
ADBOT LOVES YOU

pram
Jun 10, 2001
you can just do cp -al for local poo poo

Tankakern
Jul 25, 2007

hard-linking is messy and hard to keep tabs on though. just use btrfs subvolume snapshots.

pram
Jun 10, 2001
time machine uses hard links you dunce

pram
Jun 10, 2001
durr hard links are hard just use a beta file system it's easy Linux #1

Sapozhnik
Jan 2, 2005

Nap Ghost
lol yea lemme trust apple's godawful filesystems

Last Chance
Dec 31, 2004

Mr Dog posted:

lol yea lemme trust apple's godawful filesystems

lmao, just lmao

There Will Be Penalty
May 18, 2002

Makes a great pet!
the next question is does rsync support directory hard links on filesystems that support them

VAGENDA OF MANOCIDE
Aug 1, 2004

whoa, what just happened here?







College Slice

Mr Dog posted:

lol yea lemme trust apple's godawful filesystems

cowboy beepboop
Feb 24, 2001

Tankakern posted:

btrfs supports reflinking

cp --reflink=always

lol btrfs is still under heavy development

pram
Jun 10, 2001

Mr Dog posted:

lol yea lemme trust apple's godawful filesystems

I'm talking about Linux you imbecile R > C > P

Apocadall
Mar 25, 2010

Aren't you the guitarist for the feed dogs?

you should only use gods own file system, fat16

Notorious b.s.d.
Jan 25, 2003

by Reene

pram posted:

time machine uses hard links you dunce

i had always assumed time machine was smarter than that, because holy lol hardlinks are hosed up on hfs+

quote:

To keep track of hard links, HFS+ creates a separate file for each hard link inside a hidden directory at the root level of the volume. Hidden directories are kind of creepy to begin with, but the real scare comes when you remember that Time Machine is implemented using hard links to avoid unnecessary data duplication.

Listing the contents of this hidden directory (named "HFS+ Private Data", but with a bunch of non-printing characters preceding the "H") on my Time Machine backup volume reveals that it contains 573,127 files. B-trees or no b-trees, over half a million files in a single directory makes me nervous.

quote:



In HFS+, all hard linked files are really pointers to "actual" files in a special directory:

/^^^^HFS+ Private Data

Those four leading carets represent null characters (ASCII 0). We call this the metadata directory. The files all have names like

iNode<nnnn>

where <nnnn> is a link number. In practice, the link number is equal to the inode number (or CNID). However, this is not required in the specification, and TSK does not assume that this is so.

In TSK (in the standard build) those null characters, and all other nulls appearing in file names, are mapped to the caret character. Thus, in printed form, you will see carets, and you may enter carets when specifying such a path name.

The HFS+ hard link is a file in the file system Catalog which is marked as a "regular" file, but has some special characteristics that indicate that it is a hard link. One of its metadata fields is a "link number" which can be used to assemble the path name to the actual file which we refer to as the target of the link. The HFS+ file system is supposed to transparently direct all references to the hard link to the target file instead. Such target files are, themselves, never hard links.

Notorious b.s.d.
Jan 25, 2003

by Reene
linux has a working filesystem and i would say that is a pretty cool feature for a desktop

pseudorandom name
May 6, 2007

how do hard linked directories work, anyway?

if I unlink an entry in that directory, does the directory COW?

presumably you can unlink non-empty directories with a link count greater than one, otherwise you'd never be able to remove them.

how does .. work?

Phoenixan
Jan 16, 2010

Just Keep Cool-idge
it just works until one day it just breaks and you have no idea what the gently caress is going wrong with your computer and you're desperately trying to get your backups to work

but keep the most important files on another backup

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull

Notorious b.s.d. posted:

i had always assumed time machine was smarter than that, because holy lol hardlinks are hosed up on hfs+

hfs+ was originally designed for classic macos which does not natively have hardlinks and therefore hfs+ doesnt either

classic mac had a different thing called an alias which was mostly like a unix soft link (i.e. userspace can tell it's actually a pointer, not the original) but with the most important feature of a hardlink, which is not breaking when you move/rename the original. afaik os x still supports aliases

its silly to get all pointy-fingery about the weird implementation of hardlinks in hfs+ because the only reason unix-semantics hardlinking is still a thing today is backcompat with this one weird trick made possible by a quirky and terrible 45 year old fs. "oh hey we can have multiple dirents pointing to the same inode" *proceeds to write a shitload of hacks that will break horribly if this property ever goes away*

(i mean srsly idg why people hate on hfs+ so much, at least it has a loving b*tree directory. has linux even settled on a post-extN fs which has non caveman data structures? i noticed that el7 is defaulting to xfs now, a ringing endorsement of filesystems designed by the linux community)

Wild EEPROM
Jul 29, 2011


oh, my, god. Becky, look at her bitrate.
please never stop john siracusa-ing

the biggest thing i miss from his megareviews was his rambling about hfs+

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull

Wild EEPROM posted:

please never stop john siracusa-ing

the biggest thing i miss from his megareviews was his rambling about hfs+

well im a bad siracusa because i think hfs+ is kinda okay, rather than a disaster which needs to be cleansed with fire

speaking of siracusa i heard him on a podcast and he talks like he writes, only its way more insufferable because hes constantly interrupting and talking over everyone else with a rapidfire jumble of :words: and he reiterates every point like a million times

Tankakern
Jul 25, 2007

Notorious b.s.d. posted:

quote:

quote:

To keep track of hard links, HFS+ creates a separate file for each hard link inside a hidden directory at the root level of the volume. Hidden directories are kind of creepy to begin with, but the real scare comes when you remember that Time Machine is implemented using hard links to avoid unnecessary data duplication.

Listing the contents of this hidden directory (named "HFS+ Private Data", but with a bunch of non-printing characters preceding the "H") on my Time Machine backup volume reveals that it contains 573,127 files. B-trees or no b-trees, over half a million files in a single directory makes me nervous.


quote:

quote:



In HFS+, all hard linked files are really pointers to "actual" files in a special directory:

/^^^^HFS+ Private Data

Those four leading carets represent null characters (ASCII 0). We call this the metadata directory. The files all have names like

iNode<nnnn>

where <nnnn> is a link number. In practice, the link number is equal to the inode number (or CNID). However, this is not required in the specification, and TSK does not assume that this is so.

In TSK (in the standard build) those null characters, and all other nulls appearing in file names, are mapped to the caret character. Thus, in printed form, you will see carets, and you may enter carets when specifying such a path name.

The HFS+ hard link is a file in the file system Catalog which is marked as a "regular" file, but has some special characteristics that indicate that it is a hard link. One of its metadata fields is a "link number" which can be used to assemble the path name to the actual file which we refer to as the target of the link. The HFS+ file system is supposed to transparently direct all references to the hard link to the target file instead. Such target files are, themselves, never hard links.

holy poo poo :eyepop:

are you serious

this is the most hosed up impl of a backup system i've ever witnessed

pram
Jun 10, 2001
gluster does something similar with a mysterious hidden directory containing mapping to the brick files

jre
Sep 2, 2011

To the cloud ?



pram posted:

gluster

:smithicide:

Clever project just don't upgrade ever

Cybernetic Vermin
Apr 18, 2005

posting in the relative merit of technical details of backup system implementation approaches discussion thread

Sapozhnik
Jan 2, 2005

Nap Ghost

BobHoward posted:

hfs+ was originally designed for classic macos which does not natively have hardlinks and therefore hfs+ doesnt either

classic mac had a different thing called an alias which was mostly like a unix soft link (i.e. userspace can tell it's actually a pointer, not the original) but with the most important feature of a hardlink, which is not breaking when you move/rename the original. afaik os x still supports aliases

its silly to get all pointy-fingery about the weird implementation of hardlinks in hfs+ because the only reason unix-semantics hardlinking is still a thing today is backcompat with this one weird trick made possible by a quirky and terrible 45 year old fs. "oh hey we can have multiple dirents pointing to the same inode" *proceeds to write a shitload of hacks that will break horribly if this property ever goes away*

(i mean srsly idg why people hate on hfs+ so much, at least it has a loving b*tree directory. has linux even settled on a post-extN fs which has non caveman data structures? i noticed that el7 is defaulting to xfs now, a ringing endorsement of filesystems designed by the linux community)

ext4 has only had btrees for about the last seven years or so

Post extfs all of them do. I'm not super keen on btrfs sorry.

Soricidus
Oct 21, 2010
freedom-hating statist shill

BobHoward posted:

its silly to get all pointy-fingery about the weird implementation of hardlinks in hfs+ because the only reason unix-semantics hardlinking is still a thing today is backcompat with this one weird trick made possible by a quirky and terrible 45 year old fs. "oh hey we can have multiple dirents pointing to the same inode" *proceeds to write a shitload of hacks that will break horribly if this property ever goes away*

hacks such as time machine, the worlds most advanced incremental backup system?

like if hard links are inherently incompatible with apples fs design, why didn't they invent a fs feature that could be implemented without terrible hacks and then base their backup system on that instead

Phobeste
Apr 9, 2006

never, like, count out Touchdown Tom, man

Lysidas posted:

as loving terrible as it is that ubuntu breaks kernel ABI compatibility,


eh?

Ericadia
Oct 31, 2007

Not A Unicorn
y'all are missing the point that time machine is the best backup solution for home users, despite all the hacky poo poo. it just works (somehow)

Notorious b.s.d.
Jan 25, 2003

by Reene

Ericadia posted:

y'all are missing the point that time machine is the best backup solution for home users, despite all the hacky poo poo. it just works (somehow)

time machine is really cool

hfs+ is really terrible

Notorious b.s.d.
Jan 25, 2003

by Reene

ubuntu doesn't guarantee abi compatibility when they patch their kernels

a driver that worked fine in one ubuntu 12.04 might be broken in a different ubuntu 12.04 installation, depending on which patches and packages they have installed.

Soricidus
Oct 21, 2010
freedom-hating statist shill

Ericadia posted:

y'all are missing the point that time machine is the best backup solution for home users, despite all the hacky poo poo. it just works (somehow)

this is the linux thread, it's kind of taken for granted that minor filesystem implementation details are more important than usability

Notorious b.s.d.
Jan 25, 2003

by Reene

Soricidus posted:

this is the linux thread, it's kind of taken for granted that minor filesystem implementation details are more important than usability

does it really matter how "usable" the gui was after the lovely filesystem implementation corrupts all your backups

Optimus_Rhyme
Apr 15, 2007

are you that mainframe hacker guy?

http://gfycat.com/SlimyDistinctAntlion

Notorious b.s.d.
Jan 25, 2003

by Reene

lol ubuntu

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull

Soricidus posted:

hacks such as time machine, the worlds most advanced incremental backup system?

like if hard links are inherently incompatible with apples fs design, why didn't they invent a fs feature that could be implemented without terrible hacks and then base their backup system on that instead

because mid 2000s apple was not yet floating on iphone money and had a lot of very smart dudes who could see "oh hey we can actually pull this off with this hack and users will mostly not notice the hack and it will be a great feature and we wont have to pay 3x the cost and schedule to design a brand new fs"

(brand new because doing time machine poo poo cleanly is probably further than you can stretch hfs+, its only kinda okay not great remember)

Notorious b.s.d.
Jan 25, 2003

by Reene

BobHoward posted:

because mid 2000s apple was not yet floating on iphone money and had a lot of very smart dudes who could see "oh hey we can actually pull this off with this hack and users will mostly not notice the hack and it will be a great feature and we wont have to pay 3x the cost and schedule to design a brand new fs"

(brand new because doing time machine poo poo cleanly is probably further than you can stretch hfs+, its only kinda okay not great remember)

at the time, apple already had two other filesystems in osx: ufs and zfs. both are unambiguously less broken than hfs+. they continued to ship hfs+ instead.

apple doesn't care about how lovely their unix is. even when it could hurt users. being a good unix isn't even on the radar.

BobHoward
Feb 13, 2012

The only thing white people deserve is a bullet to their empty skull

Notorious b.s.d. posted:

at the time, apple already had two other filesystems in osx: ufs and zfs. both are unambiguously less broken than hfs+. they continued to ship hfs+ instead.

apple doesn't care about how lovely their unix is. even when it could hurt users. being a good unix isn't even on the radar.

lol that you think this

ufs: caveman fs

zfs: was unambiguously more broken than hfs+ on mac os x because the port wasn't done. they were investing in it up till the time when the project had to be killed because sun got oracled. larry and stebe may well have been best buds but they couldnt figure out how to do business together

(also it was never quite clear how apple planned to deploy zfs on mac without inflicting sysadminning on mac users, unless the plan was to just have it as an option for propeller heads and not as the default boot drive os)

Notorious b.s.d.
Jan 25, 2003

by Reene
if ufs is cavemen banging rocks together, hfs+ is botulism. a prokaryote that kills yo files

pram
Jun 10, 2001

Notorious b.s.d. posted:

apple doesn't care about how lovely their unix is. even when it could hurt users. being a good unix isn't even on the radar.

being 'good' isnt a concern when youre already the best/most advanced

Soricidus
Oct 21, 2010
freedom-hating statist shill

pram posted:

being 'good' isnt a concern when youre already the best/most advanced

really? you're saying it'sok if apple just rests on their laurels and doesn#'t try to improve until someone else catches up?

Adbot
ADBOT LOVES YOU

DONT THREAD ON ME
Oct 1, 2002

by Nyc_Tattoo
Floss Finder

Soricidus posted:

really? you're saying it'sok if apple just rests on their laurels and doesn#'t try to improve until someone else catches up?

i don't think pram was being entirely serious

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply