The Linux Questions Thread: a bunch of pitfalls, but technically it's possible

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Linux Questions Thread: a bunch of pitfalls, but technically it's possible

«‹›1038 »

ToxicFrog: Apr 26, 2008

Hadlock posted:

Has anyone ever been forced to use MKS Toolkit before?

Yep. (Disclaimer: everyone I knew at MKS is now working elsewhere and I haven't used MKSTk since 2003, so this info may be quite out of date.)

quote:

Is it as big a piece of poo poo as I feel it is?

Back in 2000 it was pretty slick. These days Cygwin and MSYS fill the same role and are only slightly less polished. Depending on what you're after you're probably better off using one of those, using NX, or just installing linux in a VM.

# ? Mar 23, 2013 00:34

Adbot: ADBOT LOVES YOU

# ? Jun 3, 2024 03:55

ExcessBLarg!: Sep 1, 2001

Longinus00 posted:

When you use dd you will be copying every bit from the first file/device onto the other and when you use it on things in /dev you're bypassing the filesystem. As a consequence, all that "free space" is going to appear to the ssd controller as real data that should be written down and not tossed. After you mount the newly copied filesystem it won't know that it's just been copied so won't trim all that free space. The only option in this case is to force a trim manually.

The "conv=sparse" option to dd does sparse writes, so any zero blocks on the original block device won't be written.

That only works if both the original device and the new one present trimmed blocks as zero blocks. I can't say for certain that, that behavior is spec (I've seen at least one flash device where trimmed blocks were 0xffs). Most devices should behave that way though, as e2fsck goes nuts on devices were trimmed inode table blocks appear differently.

# ? Mar 23, 2013 02:21

pliable: Sep 26, 2003; this is what u get for "180 x 180 avatars"

this is what u fucking get u bithc; Fun Shoe

I just wanna say how much I loving love Linux.

aptitude is the most brilliant package manager, ever. It's so god drat easy, it's retarded. Honestly, I think it's even easier than the Mac App Store.

# ? Mar 24, 2013 01:01

Gounads: Mar 13, 2013; Where am I?
How did I get here?

pliable posted:

aptitude is the most brilliant package manager, ever. It's so god drat easy, it's retarded. Honestly, I think it's even easier than the Mac App Store.

You just made me imagine star-ratings and reviews in aptitude.

# ? Mar 24, 2013 01:09

Tad Naff: Jul 8, 2004; _{I told you you'd be sorry buying an emoticon, but no, you were hung over. Well look at you now. It's not catching on at all!}

Gounads posted:

You just made me imagine star-ratings and reviews in aptitude.

Doesn't synaptic do this?

# ? Mar 24, 2013 02:10

Longinus00: Dec 29, 2005; Ur-Quan

FeloniousDrunk posted:

Doesn't synaptic do this?

No, but ubuntu's software center does.

# ? Mar 24, 2013 02:45

Tad Naff: Jul 8, 2004; _{I told you you'd be sorry buying an emoticon, but no, you were hung over. Well look at you now. It's not catching on at all!}

Longinus00 posted:

No, but ubuntu's software center does.

Forgive me, I thought they were the same thing.

# ? Mar 24, 2013 03:29

Lukano: Apr 28, 2003

Ugh, I think I may be setting myself up for sleeping on the couch for the rest of my life, if I can't fix this before my wife realizes I've lost tons of family photos and stuff. Here's the situation, in as condensed a version as I can manage.

- Had a RAID1 MDADM array consisting of 2x 500gig partitions.
- Wanted to grow two more 500gig partitions into the array, and convert to RAID5. Threw a mdadm --grow /dev/md0 --level=5 --devices=4 blah blah at it. No biggy there.
- My server was restarted due to other issues, and the resync on the RAID1->RAID5 was still pending, so I started work on assembling a second 4x 500gig array, this one RAID6.
- Due to the server restart, drive letters had changed, and due to me being in a rush, I wasn't paying enough attention. I threw it a mdadm --create, don't recall seeing any warnings about those drives already being in an array (likely due to being pending resync to RAID5 already), and didn't notice until this morning that I've effectively trashed the first array, containing nearly a hundred gigs of family photos (nevermind other documents and stuff on there).

As for the filesystem on the array, it was LVM with a cryptsetup luks container ontop of it, and an EXT4 filesystem within the encrypted storage.

Am I all sorts of poo poo out of luck here, or do I have a snowballs chance at recovery here?

# ? Mar 27, 2013 22:42

powderific: May 13, 2004; Grimey Drawer

There's stuff you can try, it's beyond my experience so I can't offer specific advice, but it sounds like there may be ways to tell MDADM to consider those disks part of a RAID1 array again. I'm sure you're fully capable of googling but here you go anyway: http://askubuntu.com/questions/69086/mdadm-superblock-recovery https://raid.wiki.kernel.org/index.php/RAID_Recovery

Do you really have no other copies of those photos? It seems crazy to spend the time making an encrypted redundant photo storage server thing but skip having an actual backup.

# ? Mar 27, 2013 23:08

Lukano: Apr 28, 2003

Yeah I have cloud backups of the photos, but apparently Jungledisk for linux stopped updating those backups sometime last June. I had been meaning to drop jungledisk/S3 and move to hosting those backups in an offshore VPS, but hadn't gotten to it yet.

Thanks for the links though, I'll poke at it and see what results I get. I'll check back for other more specific suggestions if anyone has them.

edit - and while I am perfectly capable of googling, sheer panic and adrenaline had me overlooking the obvious things to try in those sites you linked (which I had already visited) so thanks again! I'm going to start by forcing them back to the original 2 disk RAID1 instance and see how that goes.

edit2 - and yes, that seems to have done it already. a vgchange -ay has the volume group for that array showing up again, luks container opened fine, and contents appear generally instact. I'll leave it resyncing and then fsck it later, but thank you very much

Lukano fucked around with this message at 23:33 on Mar 27, 2013

# ? Mar 27, 2013 23:25

Varkk: Apr 17, 2004

If nothing has been written on the disks yet try yanking them and imaging the disks on another PC I use something like ddrescue for this. Then either run photorec or try to rebuild the original array using the disk images instead of the actual disks.

# ? Mar 27, 2013 23:33

evol262: Nov 30, 2010; #!/usr/bin/perl

Lukano posted:

Ugh, I think I may be setting myself up for sleeping on the couch for the rest of my life, if I can't fix this before my wife realizes I've lost tons of family photos and stuff. Here's the situation, in as condensed a version as I can manage.

- Had a RAID1 MDADM array consisting of 2x 500gig partitions.
- Wanted to grow two more 500gig partitions into the array, and convert to RAID5. Threw a mdadm --grow /dev/md0 --level=5 --devices=4 blah blah at it. No biggy there.
- My server was restarted due to other issues, and the resync on the RAID1->RAID5 was still pending, so I started work on assembling a second 4x 500gig array, this one RAID6.
- Due to the server restart, drive letters had changed, and due to me being in a rush, I wasn't paying enough attention. I threw it a mdadm --create, don't recall seeing any warnings about those drives already being in an array (likely due to being pending resync to RAID5 already), and didn't notice until this morning that I've effectively trashed the first array, containing nearly a hundred gigs of family photos (nevermind other documents and stuff on there).

As for the filesystem on the array, it was LVM with a cryptsetup luks container ontop of it, and an EXT4 filesystem within the encrypted storage.

Am I all sorts of poo poo out of luck here, or do I have a snowballs chance at recovery here?

mdraid --force or --assume-clean should get you going here, depending on how far you got on assembling the second array. If all you did was "mdadm --create", you overwrote superblocks, but your ext4 inside LUKS inside LVM is still safe unless you did any mkfs/pvcreate/whatever crap. You can recreate/reassemble the arrays as many times as you want (in theory, I wouldn't abuse this if I were you) until you get the right combination. Just don't mkfs, vgcreate, pvcreate, lvcreate, parted, fdisk, or anything else.

Firstly, mdadm should have warned you. It doesn't matter whether it's a pending resync or not (did you add the new partitions/drives as hot spares before the --grow? You should have), since mdadm should have looked at the superblocks and screamed at you. If it didn't, it all went wrong before you ever touched the second array.

In the future, check /proc/mdstat to make sure you know what disks are in arrays. mdadm -E/--examine every disk you're adding. And have backups. It's 2013. There's little reason you shouldn't have a 100GB of family photos backed up to a FreeNAS/OpenFiler/BSD/Solaris/whatever VM, LVM snapshots (if you'd used LVM mirroring, this would not be an issue), ZFS snapshots, a USB hard drive, or something.

Edit:

I'm glad to see you recovered it. Now back it up to another system.

evol262 fucked around with this message at 23:36 on Mar 27, 2013

# ? Mar 27, 2013 23:34

Lukano: Apr 28, 2003

Yeah, I'm moving the backups to the offshore VPS as soon as the resync and fsck are complete. Nothing like a moment of sheer unadulterated panic to remind oneself to have proper backups. I'm just exceptionally happy this story ended well for me.

# ? Mar 27, 2013 23:42

powderific: May 13, 2004; Grimey Drawer

Ha, that must have been a relief. Glad you managed to dodge a bullet on that one.

# ? Mar 28, 2013 00:34

gabensraum: Sep 16, 2003; LOAD "NICE!",8,1

Lukano posted:

Yeah, I'm moving the backups to the offshore VPS as soon as the resync and fsck are complete. Nothing like a moment of sheer unadulterated panic to remind oneself to have proper backups. I'm just exceptionally happy this story ended well for me.

You're not the first. I am very happy with my local/remote backup setup now, but it took me performing a ridiculous operation on a HDD to finally make me sort it out.

# ? Mar 28, 2013 01:22

Adult Sword Owner: Jun 19, 2011; u deserve diploma for sublime comedy expertise

So one of our servers is 10 drives, SATA 1 tb and other specs, and one of the drives has died and needs to be replaced before another drive dies and causes a problem. I put in a request and apparently nobody carries the exact drive anymore.

I found drives with the exact same brand, exact same specs, but different model number. How bad will it be to use these? I know there can be unlisted changes that can be unpredictable but I don't think I really have a choice at this point.

# ? Mar 28, 2013 01:27

Longinus00: Dec 29, 2005; Ur-Quan

Lukano posted:

Ugh, I think I may be setting myself up for sleeping on the couch for the rest of my life, if I can't fix this before my wife realizes I've lost tons of family photos and stuff. Here's the situation, in as condensed a version as I can manage.

- Had a RAID1 MDADM array consisting of 2x 500gig partitions.
- Wanted to grow two more 500gig partitions into the array, and convert to RAID5. Threw a mdadm --grow /dev/md0 --level=5 --devices=4 blah blah at it. No biggy there.
- My server was restarted due to other issues, and the resync on the RAID1->RAID5 was still pending, so I started work on assembling a second 4x 500gig array, this one RAID6.
- Due to the server restart, drive letters had changed, and due to me being in a rush, I wasn't paying enough attention. I threw it a mdadm --create, don't recall seeing any warnings about those drives already being in an array (likely due to being pending resync to RAID5 already), and didn't notice until this morning that I've effectively trashed the first array, containing nearly a hundred gigs of family photos (nevermind other documents and stuff on there).

As for the filesystem on the array, it was LVM with a cryptsetup luks container ontop of it, and an EXT4 filesystem within the encrypted storage.

Am I all sorts of poo poo out of luck here, or do I have a snowballs chance at recovery here?

I know you fixed this already but consider using uuids/device ids in the future to prevent issues like this.

# ? Mar 28, 2013 01:35

evol262: Nov 30, 2010; #!/usr/bin/perl

Saint Darwin posted:

So one of our servers is 10 drives, SATA 1 tb and other specs, and one of the drives has died and needs to be replaced before another drive dies and causes a problem. I put in a request and apparently nobody carries the exact drive anymore.

I found drives with the exact same brand, exact same specs, but different model number. How bad will it be to use these? I know there can be unlisted changes that can be unpredictable but I don't think I really have a choice at this point.

I wouldn't even worry about it.

# ? Mar 28, 2013 02:12

JHVH-1: Jun 28, 2002

Saint Darwin posted:

So one of our servers is 10 drives, SATA 1 tb and other specs, and one of the drives has died and needs to be replaced before another drive dies and causes a problem. I put in a request and apparently nobody carries the exact drive anymore.

I found drives with the exact same brand, exact same specs, but different model number. How bad will it be to use these? I know there can be unlisted changes that can be unpredictable but I don't think I really have a choice at this point.

As long as the LBA size is the same or larger it usually doesn't matter.

# ? Mar 28, 2013 02:35

spankmeister: Jun 15, 2008

Eh, if you buy drives straight from HP you get different models and even different brands. At my previous job there's a Seagate and a Fujitsu running in raid 1 somewhere.

Also, order 2 so you always have a spare on hand.

# ? Mar 28, 2013 07:40

Gucci Loafers: May 20, 2006; Ask yourself, do you really want to talk to pair of really nice gaudy shoes?

Is there a way I could make a short telnet script that would automatically do a password reset for me? This is what I have...

code:

$oldpass = passold
$newpass = passnew
$username = user

(open 192.168.1.2
sleep 2
echo $username
sleep 2
echo $oldpass
sleep 2
echo passwd
echo $oldpass
sleep 2
echo $newpass
sleep 2 
echo $newpass
sleep 2
echo exit)|telnet

(open 192.168.1.3
sleep 2
echo $username
sleep 2
echo $oldpass
sleep 2
echo passwd
echo $oldpass
sleep 2
echo $newpass
sleep 2 
echo $newpass
sleep 2
echo exit)|telnet

This will be repeated on about about a dozen IP Addresses and telnet (or rlogin?) is the only protocol I'm able to use. It fail when it switches to the next switch, which I can't figure out. Thoughts?

# ? Mar 29, 2013 13:24

hifi: Jul 25, 2012

Tab8715 posted:

Is there a way I could make a short telnet script that would automatically do a password reset for me? This is what I have...
code:
$oldpass = passold
$newpass = passnew
$username = user

(open 192.168.1.2
sleep 2
echo $username
sleep 2
echo $oldpass
sleep 2
echo passwd
echo $oldpass
sleep 2
echo $newpass
sleep 2 
echo $newpass
sleep 2
echo exit)|telnet

(open 192.168.1.3
sleep 2
echo $username
sleep 2
echo $oldpass
sleep 2
echo passwd
echo $oldpass
sleep 2
echo $newpass
sleep 2 
echo $newpass
sleep 2
echo exit)|telnet
This will be repeated on about about a dozen IP Addresses and telnet (or rlogin?) is the only protocol I'm able to use. It fail when it switches to the next switch, which I can't figure out. Thoughts?

I don't know why it's failing but this is something that should be done in expect.

# ? Mar 29, 2013 13:56

evol262: Nov 30, 2010; #!/usr/bin/perl

Tab8715 posted:

Is there a way I could make a short telnet script that would automatically do a password reset for me? This is what I have...

This will be repeated on about about a dozen IP Addresses and telnet (or rlogin?) is the only protocol I'm able to use. It fail when it switches to the next switch, which I can't figure out. Thoughts?

Pick your poison (language) and use something written for it. That basically means Expect in shell, Net::Telnet in Perl and Ruby, telnetnetlib or pexpect in Python, etc. You could sent it into netcat as well, if you have it installed (in basically the same way you're piping it into telnet right now).

# ? Mar 29, 2013 15:06

SYSV Fanfic: Sep 9, 2003; by Pragmatica

I am having a weird issue with my 7870 and the open source and binary drivers on kubunutu 12.10. With desktop effects on, resizing, minimizing and restoring konsole causes me to crash back to the session manager. Lxterm crashes it on launch. Turning off desktop effects fixes konsole. I have never had an X issue like this before and I do not know how to debug it. Any tips?

# ? Mar 29, 2013 16:26

evol262: Nov 30, 2010; #!/usr/bin/perl

keyvin posted:

I am having a weird issue with my 7870 and the open source and binary drivers on kubunutu 12.10. With desktop effects on, resizing, minimizing and restoring konsole causes me to crash back to the session manager. Lxterm crashes it on launch. Turning off desktop effects fixes konsole. I have never had an X issue like this before and I do not know how to debug it. Any tips?

Xorg logs?

# ? Mar 29, 2013 16:32

SYSV Fanfic: Sep 9, 2003; by Pragmatica

Yeah, the Xorg log just had a generic segmentation fault. It didn't even have a backtrace. I guess I could manually start the x server and enable verbose logging... I guess I didn't think of that because I be dumb. Thanks!

# ? Mar 29, 2013 16:40

SYSV Fanfic: Sep 9, 2003; by Pragmatica

Nope, got a backtrace, but it isn't terribly helpful... I guess this is what I get for buying an AMD card. I want my eyecandy :negative:

# ? Mar 29, 2013 17:19

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Are you using the binary drivers (fglrx/Catalyst) or the open-source ones (radeon)?

# ? Mar 29, 2013 17:22

SYSV Fanfic: Sep 9, 2003; by Pragmatica

Oddly enough it has done it with both. I switched to catalyst from the open source driver because of this issue. Then I installed the beta version of the driver. I will try it with the open source driver and see if I get more helpful output. I don't have windows, so for all I know, it could just be an issue with the card.

Edit:
I don't have windows installed. I guess I can do that and see if there are any issues...

SYSV Fanfic fucked around with this message at 18:05 on Mar 29, 2013

# ? Mar 29, 2013 17:52

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

It's extremely unlikely that there's a segfault with both the open-source drivers and the binary ones. I'd double-check with verbose logging to make sure it's happening there.

It's also possible that there's a crash in X completely unrelated to graphics. Can you copy the log file and then pastebin it or something?

# ? Mar 29, 2013 18:10

SYSV Fanfic: Sep 9, 2003; by Pragmatica

I can't duplicate the issue with the open source driver anymore. Maybe it was patched? Kubuntu wouldn't default to using a non free driver would it? Edit: I guess I will just chalk this up to me being dumb.

Now I have to try a couple of things to reduce tearing with the open source driver.

SYSV Fanfic fucked around with this message at 20:19 on Mar 29, 2013

# ? Mar 29, 2013 20:01

Goon Matchmaker: Oct 23, 2003; I play too much EVE-Online

I've just been informed I need to make postfix able to handle upwards of 300,000 emails a day. What exactly needs to be done? I've set up postifx before but I've never had to scale it so high and I don't know where to start.

# ? Mar 29, 2013 20:06

SYSV Fanfic: Sep 9, 2003; by Pragmatica

I don't know a lot about postfix, but it didn't used to be multi-threaded. If your hardware is good enough and you aren't going to run out of disk space, you might be OK. 300,000 emails a day is only 3.4 emails a second or so. I have used smtp-source to stress test mail servers before, so you might try that to see if your current config is good enough.

Edit: Is that 300,000 messages a day going to be incoming or outgoing?

SYSV Fanfic fucked around with this message at 20:23 on Mar 29, 2013

# ? Mar 29, 2013 20:17

Bob Morales: Aug 18, 2006; ~~Just wear the fucking mask, Bob~~

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Goon Matchmaker posted:

I've just been informed I need to make postfix able to handle upwards of 300,000 emails a day. What exactly needs to be done? I've set up postifx before but I've never had to scale it so high and I don't know where to start.

That amount is no problem at all. The hard part is keeping your IP's off the spam lists.

All of our problems are with the programs that we use to generate and queue up the mail.

# ? Mar 29, 2013 20:23

Adult Sword Owner: Jun 19, 2011; u deserve diploma for sublime comedy expertise

Bob Morales posted:

That amount is no problem at all. The hard part is keeping your IP's off the spam lists.

All of our problems are with the programs that we use to generate and queue up the mail.

That's why high performance mail systems cost $$$, because they can.

And yeh, mail reputation is a thing that people often don't think about until oops all your mailings are getting dumpstered.

# ? Mar 29, 2013 20:48

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Goon Matchmaker posted:

I've just been informed I need to make postfix able to handle upwards of 300,000 emails a day. What exactly needs to be done? I've set up postifx before but I've never had to scale it so high and I don't know where to start.

You're working for a spam company, or?

# ? Mar 29, 2013 21:20

d3c1: Feb 3, 2013; by Y Kant Ozma Post

I dont have platinum so I cant search through all 300+ replies in this thread but does anyone know of a good ebook/book or tutorial on setting up, configuring, and or maintaining a SELinux installation?

edit: what i said before plus any awesome goon tips

# ? Mar 29, 2013 21:28

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

http://people.redhat.com/tcameron/Summit2012/SELinux/cameron_w_120_selinux_for_mere_mortals.pdf

Try something like this?

# ? Mar 29, 2013 21:29

d3c1: Feb 3, 2013; by Y Kant Ozma Post

Suspicious Dish posted:

http://people.redhat.com/tcameron/Summit2012/SELinux/cameron_w_120_selinux_for_mere_mortals.pdf

Try something like this?

Thanks. Any other links? Awesome tips also welcome.

# ? Mar 29, 2013 21:32

Adbot: ADBOT LOVES YOU

# ? Jun 3, 2024 03:55

Goon Matchmaker: Oct 23, 2003; I play too much EVE-Online

Suspicious Dish posted:

You're working for a spam company, or?

No. I'm not really sure what the purpose is. My reaction was pretty much the same, "So we're going to start spamming people now?" The best I got was a vague answer about informing people of news and events related to a government program we have a contract for.

I fully expect us to get blacklisted after the first month or two. But hey, I can't stop management from being stupid, and I sure as hell can't stop developers from being stupid too.

# ? Mar 29, 2013 22:53

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Linux Questions Thread: a bunch of pitfalls, but technically it's possible

«‹›1038 »