|
What is the best way of making a backup that sd card size doesn't matter? The first sd card that I used was a 32gb that I had around. I'm about to set it up now on a 8gb since the disk space used is only 187MB. But when I make an image it takes the 32gb partition. Would I just resize the partition down to say, 300MB for expansion and make an image of that partition?
|
#
?
Jun 9, 2015 23:07
|
|
|
|
| # ? May 27, 2024 16:01 |
|
|
Yes. Shrink the partition down to whatever you want, and then dd (or whatever you're going to use) and immediately be back up and running. You can then expand the file size accordingly. Unless you just shrink the 32GB down to 8, and then image from there.
|
|
#
?
Jun 11, 2015 19:13
|
|
|
Also depending on whether you want to gently caress around a bit, you can use multiple distros one 1 card using berryboot which might be fun and you have a ton of free space. 300mb + a 300mb backup leaves a lot of free space still.
|
|
#
?
Jun 11, 2015 20:04
|
|
|
Apparently someone has Android 5.1 running on the Raspberry Pi 2, no idea on how stable or what the performance is like. Also he is charging  per download. Something presume to be in a legal gray zone for sure.http://linux.softpedia.com/blog/Run-Android-5-1-Lollipop-on-Your-Raspberry-Pi-2-with-RaspAnd-477655.shtml
|
|
#
?
Jun 13, 2015 18:38
|
|
|
YouTuber posted:Apparently someone has Android 5.1 running on the Raspberry Pi 2, no idea on how stable or what the performance is like. Also he is charging His repos are here and here. He's just charging for the binaries, which is perfectly fine.
|
|
#
?
Jun 15, 2015 20:59
|
|
|
dpbjinc posted:His repos are here and here. He's just charging for the binaries, which is perfectly fine. Yeah, having to build yourself really isn't that big a deal. What's the license? You could be a shitlord and set up a Jenkins/Bamboo/etc server to ping GitHub every hour and auto-build+upload when he pushes. SourceForge isn't the only one who can play that game  What's GitHub's perspective on binaries anyway? Obviously that's not what you SHOULD be doing with Git, but how big an rear end in a top hat do you have to be before they take notice? Would they get mad if you uploaded 10mb of library binaries to ease your build, or do you have to start logging hundreds+ mb before they take notice? Is there an equivalent service for distributing open source binaries? It would be rad if more places supported bup - it's an adaptation of Git (mashed up with concepts from rsync) that can actually do efficient versioning of binaries. So if you're storing something that's mostly redundant content, like versioned binaries from builds, system images, backups, etc it handles it really well. Paul MaudDib fucked around with this message at 04:17 on Jun 16, 2015 |
|
#
?
Jun 16, 2015 04:07
|
|
|
I just got my RP2 B, and think I ordered the wrong case and heat sinks. Basically the space cut out for the CPU heat sink is a bit off, and there's no slot for the RAM heatsink (that I didn't get, just got two big ones). So for now I've assembled it, and put the heatsink on the CPU. It's not really covering it completely. Is this a problem if its primary purpose is media server/streaming stuff?
|
|
#
?
Jun 17, 2015 10:23
|
|
|
midnightclimax posted:I just got my RP2 B, and think I ordered the wrong case and heat sinks. Basically the space cut out for the CPU heat sink is a bit off, and there's no slot for the RAM heatsink (that I didn't get, just got two big ones). So for now I've assembled it, and put the heatsink on the CPU. It's not really covering it completely. Is this a problem if its primary purpose is media server/streaming stuff? Generally speaking they don't need heatsinks unless you overclock them or they're in an enclosed case in a hot environment. I did have a Raspberry Pi B overheat when it was sitting on top of a warm network switch in a case without much ventilation but I'm not sure a heatsink would have really helped in that situation anyway.
|
|
#
?
Jun 17, 2015 10:51
|
|
|
Rexxed posted:Generally speaking they don't need heatsinks unless you overclock them or they're in an enclosed case in a hot environment. I did have a Raspberry Pi B overheat when it was sitting on top of a warm network switch in a case without much ventilation but I'm not sure a heatsink would have really helped in that situation anyway. Ok, got it. So basically the only way to power it on/off is by pulling out the power cord? I read somewhere that you can connect the cable to USB TV in, and then power-on/off will happen when switching TV on/off. I'll try that next.
|
|
#
?
Jun 17, 2015 11:24
|
|
|
midnightclimax posted:Ok, got it. So basically the only way to power it on/off is by pulling out the power cord? I read somewhere that you can connect the cable to USB TV in, and then power-on/off will happen when switching TV on/off. I'll try that next. No you need to pull out the power cord to turn off the Pi. However make sure you always log in and run the shutdown command before you remove power. If you don't do this you will very easily corrupt the entire SD card and have to reinstall the operating system. SD cards are not designed to suddenly lose power and the file system will get destroyed if power is pulled when it's moving around blocks and changing other card state. I have a feeling what you read were probably scripts that people had setup with an IR receiver so when a certain remote control was sent it triggered the Pi to shutdown (by running a command like 'sudo shutdown -h now').
|
|
#
?
Jun 17, 2015 11:49
|
|
|
mod sassinator posted:No you need to pull out the power cord to turn off the Pi. However make sure you always log in and run the shutdown command before you remove power. If you don't do this you will very easily corrupt the entire SD card and have to reinstall the operating system. SD cards are not designed to suddenly lose power and the file system will get destroyed if power is pulled when it's moving around blocks and changing other card state. Will do. Yeah it's possbile there's some script involved that I haven't looked at yet. But everything works perfect so far, quite impressed (also the openelec boot speed - woah).
|
|
#
?
Jun 17, 2015 12:21
|
|
|
mod sassinator posted:No you need to pull out the power cord to turn off the Pi. However make sure you always log in and run the shutdown command before you remove power. If you don't do this you will very easily corrupt the entire SD card and have to reinstall the operating system. SD cards are not designed to suddenly lose power and the file system will get destroyed if power is pulled when it's moving around blocks and changing other card state. How likely is this really? I'm going to have one doing some very basic work 24/7 soon and while having a backup image is very simple in this case, having to restore from it every time there's a power outage (rare, thankfully) or the device has to be unplugged (maybe less rare) could get frustrating. While I was writing the software and testing with external devices, I managed to kill the pi a few times from trying to get too much power out of it, and it never corrupted the SD card. So I'm hoping this corruption thing is not really that likely on a per-incident basis.
|
|
#
?
Jun 17, 2015 13:46
|
|
|
Maybe it's my SD card or something about the first Pi or openelec, but I have yanked the power out from my pi (crashes, laziness, power outages) hundreds of times over about a year and have never actually had any issue with corruption. I'm aware that it's possible, and I keep a backup of everything so if the worst happens I'm ready for it, but it just hasn't happened. Knock on wood. It really might be something with openelec though, maybe it's not reading/writing to the card much? I don't know, but in my case at least it hasn't been an issue.
|
|
#
?
Jun 17, 2015 14:10
|
|
|
mod sassinator posted:SD cards are not designed to suddenly lose power No they absolutely are. It's the file system you might be using that might not be. SD cards were developed to handle use in digital cameras on batteries that might go out at any time, including when the camera was writing new photos. You shouldn't expect severe data loss unless you know the Pi's writing something important at that very minute.
|
|
#
?
Jun 17, 2015 15:07
|
|
|
All the forum posts I've seen on the topic of data loss seem to predate the B+, I'm not sure if this was a hardware improvement, or probably a software one, but I've been treating my A+ with Arduino levels of misuse/miscare (I was powering mine off of a Qi Charger last night for grins, I've been running it directly off of a 5.5v solar panel in the recent past) and haven't had any issues so far. I don't do any datalogging to the SD though. No issues with corrupting the SD card. My guess is that they did some unadvertised improvements to help minimize this issue, as I see lots of warnings about it, but nobody actually reporting the issue any more.
|
|
#
?
Jun 17, 2015 15:09
|
|
|
BTW what file system is the SD supposed to use? I chose FAT32 out of habit, but maybe ext4 would have been better?
|
|
#
?
Jun 17, 2015 15:15
|
|
|
BrainDance posted:Maybe it's my SD card or something about the first Pi or openelec, but I have yanked the power out from my pi (crashes, laziness, power outages) hundreds of times over about a year and have never actually had any issue with corruption. I'm aware that it's possible, and I keep a backup of everything so if the worst happens I'm ready for it, but it just hasn't happened. Knock on wood. Same for me. I never worry about it. Hell, I have an external drive connected and I don't even bother to unmount it when I pull it to do things.
|
|
#
?
Jun 17, 2015 15:24
|
|
|
midnightclimax posted:BTW what file system is the SD supposed to use? I chose FAT32 out of habit, but maybe ext4 would have been better? If you're using NOOBS, you format as FAT32, and NOOBS will shrink the partition as low as possible and make a new partition for you with the correct file system. The second partition will not be FAT32.
|
|
#
?
Jun 17, 2015 15:35
|
|
|
I've been flashing Raspbian to my SD cards direct. The thing about running off of solar is that it will presistently boot until it powers up the wifi dongle, which typically will cause a low power condition and cause the machine to reboot over and over until it gets in direct sunlight, at which point the panel supplies enough power to get over the initial "hump". The A+ on any given day might reboot hundreds of times if it never gets 3 minutes of full sunlight during initial boot.
Hadlock fucked around with this message at 15:43 on Jun 17, 2015 |
|
#
?
Jun 17, 2015 15:41
|
|
|
dpbjinc posted:If you're using NOOBS, you format as FAT32, and NOOBS will shrink the partition as low as possible and make a new partition for you with the correct file system. The second partition will not be FAT32. Hmmno I used gparted to format with FAT32, then I ran the openelec install-script.
|
|
#
?
Jun 17, 2015 15:41
|
|
|
BrainDance posted:Maybe it's my SD card or something about the first Pi or openelec, but I have yanked the power out from my pi (crashes, laziness, power outages) hundreds of times over about a year and have never actually had any issue with corruption. Same, I work with Linux servers and would never do this to them but my Pi is constantly rsyncing stuff from the Internet and I never had any problems shutting it down by yanking the power accidentally or because it locked up. E: Beaten like a corrupted file system. Grim Up North fucked around with this message at 15:53 on Jun 17, 2015 |
|
#
?
Jun 17, 2015 15:49
|
|
|
If you're going to pull the power a lot you should consider using ext3 instead of ext4, as ext4 is much more susceptible to data loss. ext3 guarantees that all file metadata changes will be written out within 5 seconds, and if you set data=ordered then all file data blocks will be written out before the metadata is written (i.e. within 5 seconds). With default settings ext4 will sometimes delay those writes for up to a minute if it needs to - so you're looking at significantly longer windows of data loss. https://lwn.net/Articles/322823/ This is actually the subject of a holy war in Unix because the guy who wrote ext4 thinks that software which doesn't constantly call fsync() is broken, and has explicitly stated that this includes the majority of the software in the world such as crap software like GNU fileutils (mv). Doing so will trash your performance of course, which is why nobody does it. You really should just pony up and buy a soft power-switch for your Pi. http://mausberry-circuits.myshopify.com/ Paul MaudDib fucked around with this message at 17:04 on Jun 17, 2015 |
|
#
?
Jun 17, 2015 17:01
|
|
|
I might build a supercap UPS that triggers shutdown. That would be pretty rad. My anecdata is that my Pi 1 B+ corrupts something on the filesystem maybe every second bad shutdown. Not badly enough to require reflashing, but it wants to boot into recovery to fix itself. After I repair it (using another computer because you can't run it on the mounted drive), it's good for another couple shutdowns. It's pretty annoying.
|
|
#
?
Jun 17, 2015 17:13
|
|
|
Nintendo Kid posted:No they absolutely are. It's the file system you might be using that might not be. SD cards were developed to handle use in digital cameras on batteries that might go out at any time, including when the camera was writing new photos. Watch this great talk on how flash memory work: https://www.youtube.com/watch?v=K3zb6p0thQU When flash is erased it can't just zero out a single byte at a time, it has to zero out an entire cell or chunk of the memory. This is a problem when you only want to delete a few bytes, a file, etc. so they end up pulling the entire cell into a RAM cache on the flash controller, deleting the entire cell, and then writing the data in RAM that wasn't deleted back out to new cells. The problem is when you remove power there might be data in the controller RAM that hadn't yet been written back to cells and is now gone forever. This is magnified when using a filesystem like ext4 which really isn't designed to handle this kind of deletion scenario & can easily corrupt file lookup tables. Trust me, you really don't want to make a habit out of pulling the power from the Pi without cleanly shutting it down. You might be fine 50% of the time, but that other 50% can easily corrupt the entire filesystem.
|
|
#
?
Jun 17, 2015 19:15
|
|
|
mod sassinator posted:Watch this great talk on how flash memory work: You're describing a race condition, power is lost at the point the system is flushing IO, before it can write the new/replacement blocks. I find it very unlikely that power loss would happen at precisely the right time for this to happen. Even if it is possible the likelihood of that event seems pretty narrow, it's the same problem normal disks have with large write caches, a theoretical concern but not usually an issue (which is easily solved with a battery backed cache). It's not a huge problem in most cases because write speeds are quick and sequential with flash memory anyway. Worst case is you have a few blocks of data that were lost at very most seconds before the power outage, which you'd probably have trouble pinpointing as happening before or after the power outage anyway. The chance of losing a whole system to a power outage is incredibly slim, especially on an rPi which isn't usually doing lots of disk IO.
|
|
#
?
Jun 17, 2015 19:49
|
|
|
ElCondemn posted:You're describing a race condition, power is lost at the point the system is flushing IO, before it can write the new/replacement blocks. I find it very unlikely that power loss would happen at precisely the right time for this to happen. I don't know why you think this has to be a "precise" timing, at any point after the data is read until the last byte is written back is the window. Depending on the block size and fragmentation, that might be a very large window that's frequently open. Flash controllers can be finicky on top of that, it generally takes significantly more current to write than read and a dodgy power supply might extend the writing even further. It's nothing so clever as duplicating the block and using an atomic bit write to signal completion that would require a very narrow window. That being said, I yanked the power on my rpi-as-WebTV often enough and never corrupted it. More often when I wasn't actively using it and just wanted to grab a 5v supply, but no active attempt to quiesce it.
|
|
#
?
Jun 17, 2015 20:02
|
|
|
ElCondemn posted:You're describing a race condition, power is lost at the point the system is flushing IO, before it can write the new/replacement blocks. I find it very unlikely that power loss would happen at precisely the right time for this to happen. Even if it is possible the likelihood of that event seems pretty narrow, it's the same problem normal disks have with large write caches, a theoretical concern but not usually an issue (which is easily solved with a battery backed cache). It's not a huge problem in most cases because write speeds are quick and sequential with flash memory anyway. SD card controllers cache other data in their RAM, like frequently accessed blocks. The kind of frequently accessed blocks that are the root of the filesystem's lookup tables. If those are lost then the entire filesytem is destroyed. If you look at the Raspberry Pi forums, comments, etc. you'll see one of the most frequent complaints is that the Pi is unreliable or SD cards are corrupted. In this thread there are lots of folks who have hit similar issues. Ultimately for people new to the Pi they need to know that you can't just pull out the power to turn it off, you need to cleanly shut down the OS and filesystem. You might pull out the power yourself and get lucky by not running into issues, but that experience is not typical and not something that people new to the Pi should follow (especially if they just yank out the power while X windows is running, etc.).
|
|
#
?
Jun 17, 2015 20:03
|
|
|
JawnV6 posted:I don't know why you think this has to be a "precise" timing, at any point after the data is read until the last byte is written back is the window. Depending on the block size and fragmentation, that might be a very large window that's frequently open. Flash controllers can be finicky on top of that, it generally takes significantly more current to write than read and a dodgy power supply might extend the writing even further. It's nothing so clever as duplicating the block and using an atomic bit write to signal completion that would require a very narrow window. You're telling me when you perform a read on flash memory it removes it from disk while it's in memory? Why would they do that? That's counter to how every disk controler I know of currently works, all the disk controllers I'm aware of do not remove any data until it's time to write. They don't even perform a "delete" really, they just overwrite previous data. The only kind of corruption that happens in modern disks (especially flash memory) are partial writes, which of course can be caused by things like power outages. mod sassinator posted:SD card controllers cache other data in their RAM, like frequently accessed blocks. The kind of frequently accessed blocks that are the root of the filesystem's lookup tables. If those are lost then the entire filesytem is destroyed. Controllers might have caching, but I'm not aware of any "most frequently accessed block" functionality in the rpi. These are usually file system level functions, at the block level it's not common unless you're getting into enterprise grade raid controllers (and even then it's just more advanced caching). Either way keeping the most frequently accessed sectors (what's it even called in flash memory?) in memory has nothing to do with writing or deleting, you don't lose anything and you certainly don't have a block level "lookup table" that can be lost. It seems to me you might be confusing block level features and functions with file system level features and functions.
|
|
#
?
Jun 17, 2015 20:13
|
|
|
mod sassinator posted:
I can confirm that. I had lots of trouble with SD cards until I moved the root partition to a spinning disk. That was with a Rpi B, which also had a really lovely SD slot; I don't know if the B+ and 2 have the same problems.
|
|
#
?
Jun 17, 2015 20:19
|
|
|
ElCondemn posted:You're telling me when you perform a read on flash memory it removes it from disk while it's in memory? Why would they do that? ElCondemn posted:That's counter to how every disk controler I know of currently works, all the disk controllers I'm aware of do not remove any data until it's time to write. They don't even perform a "delete" really, they just overwrite previous data. The only kind of corruption that happens in modern disks (especially flash memory) are partial writes, which of course can be caused by things like power outages.
|
|
#
?
Jun 17, 2015 20:21
|
|
|
JawnV6 posted:"Until the last byte is written" sorta sets up that I'm continuing the conversation about writing and not launching into a wholly unrelated conversation about reading unprovoked or contextualized in any way. My understanding of SSDs, linux and hard drive controllers is counter to what's being described, powering off an rpi suddenly should not cause system corruption ever, at worst you'd lose the last few seconds of data being written due to caching (does the rpi even have a disk cache?) JawnV6 posted:You don't get to "overwrite previous data" in any meaningful way. The only verbs available are "erase sector" and "write partial". Any smaller-than-sector erasure means slurping the entire block into RAM and writing out the entirety sector minus the erased bit. I was mostly just pointing out that writes don't happen how it's been described in this thread. In fact all operating systems and kernels have been aware of the limitations of flash memory for a while. For instance, the linux kernel knows to write to a new block instead of overwriting a previous block (look up wear leveling) so you're more likely to have your original data intact than is likely to lose any data at all. Of course if you power down during a write it will fail to write, but it would not cause system wide corruption or even affect any data that was previously on disk.
|
|
#
?
Jun 17, 2015 20:47
|
|
|
ElCondemn posted:Controllers might have caching, but I'm not aware of any "most frequently accessed block" functionality in the rpi. These are usually file system level functions, at the block level it's not common unless you're getting into enterprise grade raid controllers (and even then it's just more advanced caching). Either way keeping the most frequently accessed sectors (what's it even called in flash memory?) in memory has nothing to do with writing or deleting, you don't lose anything and you certainly don't have a block level "lookup table" that can be lost. It's not caching in the OS, it's caching in the SD card controller itself. An SD card is actually a little CPU and RAM attached to cells of raw flash storage. There's code in the controller to deal with wear leveling, caching, etc. and make the device appear to just be block storage. Look at the back of a microSD card and you can see a couple of the contacts are a millimeter or so longer than the other contacts. These are the power and ground to the chip and they're longer so the controller can detect when the card slides out and flush any state it has in memory back to flash in a few microseconds. When you yank out the power on the Pi you're subverting that kind of protection and can lose entire critical blocks. Check out the video I linked above, it's a quite good explanation of SD card and other flash memory storage issues.
|
|
#
?
Jun 17, 2015 21:50
|
|
|
mod sassinator posted:It's not caching in the OS, it's caching in the SD card controller itself. An SD card is actually a little CPU and RAM attached to cells of raw flash storage. There's code in the controller to deal with wear leveling, caching, etc. and make the device appear to just be block storage. I did watch the video, it was interesting but nothing that was said in that video supports the idea that blocks will just go missing if you remove power suddenly. There are definitely features in every flash controller that handle caching and performance etc., but I'm not so sure the rpi has any that would cause the kind of problems people are worried about in this thread. ElCondemn fucked around with this message at 22:04 on Jun 17, 2015 |
|
#
?
Jun 17, 2015 22:00
|
|
|
ElCondemn posted:I did watch the video, it was interesting but nothing that was said in that video supports the idea that blocks will just go missing if you remove power suddenly. There are definitely features in every flash controller that handle caching and performance etc., but I'm not so sure the rpi has any that would cause the kind of problems people are worried about in this thread. You need to rewatch the video: https://youtu.be/K3zb6p0thQU?t=2089
|
|
#
?
Jun 17, 2015 22:10
|
|
|
mod sassinator posted:You need to rewatch the video: https://youtu.be/K3zb6p0thQU?t=2089 Earilier in the video he explains that he's come up with some conjecture about how this specific feature functions: https://youtu.be/K3zb6p0thQU?t=944 I don't think it's reasonable or true that this feature exists in most if any SD cards and their controllers, if what he said were true we'd see lost partition tables left and right, and that's just not what happens when you unplug an SD card with no warning. He seems like a smart dude and I learned a bit from his video, but he's skipped lots of steps and features that would impact his work, so I would say he's not as accurate as he might seem. ElCondemn fucked around with this message at 22:22 on Jun 17, 2015 |
|
#
?
Jun 17, 2015 22:20
|
|
|
I've never noticed issues with sd card corruption and I have been power cycling ~40 devices nightly for the better part of a year My applications write barely anything to disk, but I haven't observed a single instance of full corruption.
|
|
#
?
Jun 17, 2015 22:23
|
|
|
 I'm not sure what else I can say--see over 37k Google hits for SD card corruption on the Pi forums: https://www.google.com/search?q=raspberry+pi+sd+card+corrupt+site:www.raspberrypi.org%2Fforums&ei=xuqBVZuWCJGqogSDlIEw Will it always happen? No. Can it happen? Absolutely, yes. SD card firmware varies greatly--cheap cards might have very little caching and be less susceptible than more expensive cards that keep more data in RAM to be faster. There can be bugs or issues in the SD card firmware that make the issue more or less of a problem. It doesn't change the fact that it's extremely bad advice to tell a Pi user that they can safely pull the power without shutting down the Pi's OS. Anecdotes about it not happening with your card under your conditions are unfortunately just anecdotes.
|
|
#
?
Jun 17, 2015 22:50
|
|
|
mod sassinator posted:
I'm not saying it can't happen, I'm just saying SD cards are just like any other disk device. You risk losing/corrupting data you're currently writing if you remove power suddenly. Total disk corruption is not more likely with flash memory and if it happens to you it's almost certainly not because you're using flash memory. If anything it's less likely with flash memory since writes happen much more quickly and caches are smaller (not to mention features like wear leveling which improve data integrity). If your SD card failed it's probably because it's cheap and lovely and not made well, it's not because you unplugged it suddenly.
|
|
#
?
Jun 17, 2015 22:58
|
|
|
ElCondemn posted:I'm not saying it can't happen, I'm just saying SD cards are just like any other disk device. You risk losing/corrupting data you're currently writing if you remove power suddenly. Total disk corruption is not more likely with flash memory and if it happens to you it's almost certainly not because you're using flash memory. If anything it's less likely with flash memory since writes happen much more quickly and caches are smaller (not to mention features like wear leveling which improve data integrity). No, SD cards are not like other disk drives ('other' being spinning hard disks) at all. They might look like a hard drive as far as the operating system is concerned but the physical characteristics are completely different. Entire cells of flash have to erased at a time vs. a hard disk that can zero out specific locations. Erasing data is order of magnitudes slower for flash vs. hard drives. Random access is orders of magnitude faster for flash vs. hard drive. The video I linked to explains these differences in great detail and how SD card controllers try to deal with the impedance mismatch that filesystems and OS' have been built and optimized for years to work with that characteristics of spinning disks vs. flash storage. quote:If your SD card failed it's probably because it's cheap and lovely and not made well, it's not because you unplugged it suddenly. This is very bad advice and you are doing more harm continuing to say it. People new to the Pi absolutely should not pull the power from the Pi without shutting down the operating system. Please stop saying otherwise.
|
|
#
?
Jun 17, 2015 23:09
|
|
|
|
| # ? May 27, 2024 16:01 |
|
|
mod sassinator posted:It's not caching in the OS, it's caching in the SD card controller itself. An SD card is actually a little CPU and RAM attached to cells of raw flash storage. There's code in the controller to deal with wear leveling, caching, etc. and make the device appear to just be block storage. This is good postin'. Thanks for sharing sir, that pretty much clears up the issue right there.
|
|
#
?
Jun 17, 2015 23:20
|
|