|
Oh weird, the problem fixed its self after rebooting the last time. I guess I just need to stay the hell away from Linux until I figure out what I'm doing? I imagine this happened because of something I did.
|
#
?
Apr 22, 2015 19:53
|
|
|
|
| # ? Jun 11, 2024 10:08 |
|
|
Hollow Talk posted:Just because it is currently #1 on HackerNews and seems kind of topical: http://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html
|
|
#
?
Apr 22, 2015 20:12
|
|
|
Prince John posted:Thanks, if you're interested, I've included a paste of the results here: http://pastebin.com/ChjYHwzF Seems pretty clear to me that running off of a usb drive, even a spiffy one, is slow as balls.
|
|
#
?
Apr 23, 2015 00:46
|
|
|
Prince John posted:Thanks, if you're interested, I've included a paste of the results here: http://pastebin.com/ChjYHwzF To explain those iostat numbers, you're getting low double digit writes per second, your average queue size (think of each read/write command requiring its own slot in the queue) is in the hundreds, and on average it takes up to 5 seconds for a write to go through. Basically this is on the low end of the charts I posted. There are alternatives though, you'll notice a few of the usb stick drives on that chart had IOs that were really high, in the hundreds or thousands. Those usb sticks are actually internally just the same chips you'll find in consumer 2.5 SSDs just in a smaller formfactor and with less parallelism because of fewer flash dies. Also many (all?) of them you can't trim. Also expect to wear out your USB stick really quickly because it's going to have to wear level all these small writes when it was really tuned for storing large files a normal user would put on it (think photos, powerpoint docs, movies, etc). USB sticks (and consumer portable flash drives in general) tend to use really cheap flash dies with low lifetime guarantees. Longinus00 fucked around with this message at 04:44 on Apr 23, 2015 |
|
#
?
Apr 23, 2015 04:41
|
|
|
Longinus00 posted:Also expect to wear out your USB stick really quickly because it's going to have to wear level all these small writes when it was really tuned for storing large files a normal user would put on it (think photos, powerpoint docs, movies, etc). USB sticks (and consumer portable flash drives in general) tend to use really cheap flash dies with low lifetime guarantees. Thanks, I feel very educated! It'll probably still be cheaper for me to flog a £10 flash drive to death, even over a reduced lifespan, than buy a small SSD - there would be no free slots inside the case for one either. All of the programs I run on it have working directories on the disk array, so the only time I ever notice slowdown is due to OS updates. In practice, 99% of the time, it's just sat there serving files or streaming video quite happily so it suits my limited needs. Prince John fucked around with this message at 10:03 on Apr 23, 2015 |
|
#
?
Apr 23, 2015 10:01
|
|
|
Robo Reagan posted:Oh weird, the problem fixed its self after rebooting the last time. I guess I just need to stay the hell away from Linux until I figure out what I'm doing? I imagine this happened because of something I did. No, it means you need to stay away from computers.  Honestly it is unlikely that Linux did anything to your video card, it's more likely that your power supply could be having issues providing enough power to the GPU or the GPU is on its way out. If you really want to tee this, find a spare PC worh of pars, or use a Linux live CD and cross your fingers.
|
|
#
?
Apr 23, 2015 14:10
|
|
|
What do some of you goons with bigger shops use for IPAM? In a previous life we moved from a hodgepodge of bind + dhcpd + ipplan to infoblox, and I loved it. I'm trying to get budget to get infoblox in my new shop since we use powerdns + dhcpd with no IPAM, but I'm getting pushback from management for cost reasons, so I've been tasked with exploring alternatives. Aside from solarwinds, I don't really know what else is out there, and at first glance it seems like it has a pretty heavy Microsoft slant, which we don't have much of. Does anyone have suggestions on where else I should be looking? Has anyone come up with a FOSS solution that's easy to manage and scales to thousands of nodes without major pain points?
|
|
#
?
Apr 23, 2015 22:18
|
|
|
RFC2324 posted:This almost always follows it. And given my experience as a sysadmin in both the *NIX and Windows worlds, I can't see how devops would really work well in anything but a small environment, since I clash with devs on a weekly, if not daily, basis because devs have terrible practices because 'I write the thing so I know all!!!! RFC2324 posted:My understanding of devops was that it was where you combine the 2 roles(dev and sysadmin) into one job, so your sysadmins are in tune with what the devs need and your devs understand why sysadmins do the things we do that make their lives harder. Its a wonderful idea, it just seems that its fairly incompatible to shoehorn the 2 roles into one person, since the goals of both are never going to be entirely harmonious. Disclaimer: I'm currently attending DevOps Days Rockies, so I've clearly drunk the kool aid on this DevOps has been coopted to mean anything and everything at this point, kinda like cloud. But when the term was first coined, it was just about everyone in the company being on the same page. Displaying trust and understanding. Cooperating cross-functionally to make the business more agile instead of doing the awful traditional Waterfall SDLC, operating in silos and passing the buck/pointing fingers as much as possible. Aligning the incentives of both groups is a huge deal, as you said. If Ops is rewarded 100% for stability and dev is rewarded 100% for delivering new features, that's gonna be a problem. Both teams have to be measured and rewarded on BOTH of those or this is a total non starter. Over time, it's come to mean that the way developers and operations work are converging. Yes, this means you as a sysadmin will have to learn like 5% of what it take to work as a software dev. But it's really not that bad and the productivity gains pay off 100 fold. Because we've been able to see how effective practices like source control, testing, and continuous integration/delivery can be. With tools like config management and VM's/containers, we can leverage decades of learning on how to effectively develop software and apply it to the sysadmin side of the house. And the flip side is not "devs don't change anything and just tell ops what to do cause they know best". The flip side should be that devs take operational concerns into account from day 1. They write software that is easy to deploy, monitor, and debug. They participate in on-call for issues stemming from the code. They develop in an environment representative of production and at least superficially understand how production looks. Instead of writing code on Windows against MSSQL that then runs on Linux against Postgres in production or whatever. Taken all together, this enables some pretty cool stuff. Ops can build developer environments from the same configs they use to build production. This eliminates the classic "lol it worked on my machine, your problem now brah" bullshit. Ops can set up their monitoring system to accept metrics via a well-known API, and dev can then output stats from their apps to that API and track whatever they care about. Dev can build a /health/ API endpoint into every service for easy monitoring. And so on. Massive companies like Target, Disney, Raytheon and Wal-Mart are "doing DevOps", it's not just the province of 3 person startups. Nor is it hiring one person to write all your code and run all of your servers. That's called abuse or negligence. Your example in the first quote is like the polar opposite of DevOps. That person is just being a dick. If you are clashing with devs on a daily basis, you are by its very definition not doing DevOps. Until you're cooperating and collaborating instead of butting heads and bitching about how awful the other team is, all the coder-sysadmins and config management tools and cloud infrastructure are just window dressing.
|
|
#
?
Apr 24, 2015 04:23
|
|
'
|
The company I am working for we are working to have a standardized environment managed by a set of automated tools so there is less manual work, and less room for human error. Part of that is going to be getting dev environments replaced with the same stack that runs in all the other environments. It helps us because there won't be surprises when it comes time to push the code out, and it helps them because they don't have to do sys admin tasks configuring their own servers. They tell us what the application needs to run, if we think its viable use of resources we put it in place and then they can focus more on working on the application. Its kinda cool to be in an organization where that can happen, because it might not always be possible with politics and group divisions. It means breaking a lot of band habits though. My boss told me at one point he told someone from one of the application development teams that if you want to do infrastructure/ops work then he can switch to the IT group instead of being a developer. That kinda got the guy to re-evaluate what he was doing. Anyway, which "devops tools" you use to get the job done is highly dependent on the type of application you are running. Even though a container may be a cool technology it might not be the best fit. It may require sys admins to do more scripting to automate things and be more closely familiar with the build process and what the application does. That just means companies higher smaller groups of people with a different level of skill instead of having more manual labor and spend money on more employees to just throw grunts at it.
|
|
#
?
Apr 24, 2015 14:55
|
|
|
Docjowles posted:Your example in the first quote is like the polar opposite of DevOps. That person is just being a dick. If you are clashing with devs on a daily basis, you are by its very definition not doing DevOps. Until you're cooperating and collaborating instead of butting heads and bitching about how awful the other team is, all the coder-sysadmins and config management tools and cloud infrastructure are just window dressing. My example in the first post wasn't supposed to be an example of DevOps, we do not do that here. We do tend to work with devs to try and get them to their end result without violating any of our policies, but we have the special snowflakes who have to have it their way, and those we tend to clash with, frequently. We have multiple special environments set up for Dev and R&D work, separate from our various operational environments(Last I checked, over 70 LDAP domains, plus multiple windows domains) so when they want to do something that goes against policy, its policy in the special isolated environment set up for them, not even the more general and tighter run environments that run everything else. And in an org as big as mine, even 1% special snowflakes is hundreds of them.
|
|
#
?
Apr 24, 2015 15:29
|
|
|
Ok, I have a weirdly unwell Linux server here. Symptoms: - incoming connections to sshd are closed instantly - connections to smtpd succeed but it never says anything and responds to no messages - 'systemctl' thinks for a while and then says "connection timed out" - existing ssh sessions still work - tmux lets me create new shells, but attempts to su in those shells also hang (fortunately I already had a root shell open for other work) - yast2 works, but trying to load a module hangs - dmesg contains nothing but audit messages from selinux in permissive mode, none of them relevant - hundreds of zombie sshd processes are parented to systemd, not sshd - systemd is listed as status D (blocked on disk) in htop and shows no signs of unblocking - attempting to attach to it with gdb prints "attaching to process 1" and then hangs. At the moment my best guess is that the new (0.6.4) ZFS driver is loving up somehow and systemd is getting stuck in an IO call forever, but I'm not sure how to verify if this is actually the case, how to prevent it from happening again, or how to fix it short of rebooting (if this is even possible, I kind of suspect it isn't).
|
|
#
?
Apr 25, 2015 01:55
|
|
|
Sounds like things are so fubared that dbus and pam aren't working.
|
|
#
?
Apr 25, 2015 02:02
|
|
|
Edit: quote is not edit.
|
|
#
?
Apr 25, 2015 02:02
|
|
|
Further information: - echo t > /proc/sysrq-status dumps stacks for all 300-odd zombie sshd processes but not, weirdly, for systemd, i.e. the process I'm most interested in a stack for - E: echo w works but the stack matches the one below - cat /proc/1/stack shows me this: code:I want to just reboot and call it a day but if there's some way I can get more information about this which will help me prevent it in the future, I figure that has to be done before I reboot. E: I think this is the 3.11 cgroup mutex deadlock bug, actually! I'm running the right version for it and sysrq w shows stacks consistent with it. Updating past 3.11 should fix it. ToxicFrog fucked around with this message at 02:27 on Apr 25, 2015 |
|
#
?
Apr 25, 2015 02:09
|
|
|
Check to make sure that your disks are looking OK in terms of free space, free inodes, and health.
|
|
#
?
Apr 25, 2015 03:07
|
|
|
No doubt a blindingly obvious systemd question here, but my googling isn't turning up much. I'm having audio issues with kodi (formerly xbmc) on my fedora 21 server, but only when I run it as a systemd service. The problems occur with, and without, pulseaudio installed. Loading X and i3wm as user 'kodi' (using startx), then running kodi-standalone from an xterm, everything works perfectly. For both pulseaudio and ALSA, it 'just works' - I can see various audio adapters in the settings panel and sounds play as expected. However, when I use systemd to start kodi (no X running beforehand), there is a long hang of 10 seconds or so, then it loads, but there are no audio adapters present (the selection dialogue is greyed out). No audio plays. The kodi.service file is pretty simple, and uses the same user account 'kodi', taken from the wiki here. quote:[Unit] It seems to be running the same executable, with the same user and group. Does anyone have any insight into how systemd might initialise audio differently compared to a 'normal' user account? Prince John fucked around with this message at 00:09 on Apr 26, 2015 |
|
#
?
Apr 26, 2015 00:07
|
|
|
Maybe you need to start it after pulseaudio
|
|
#
?
Apr 26, 2015 00:22
|
|
|
pulseaudio runs in the user's X session. Normally, there is some autospawn logic so that pulseaudio will spawn on first use. I imagine some environment variable isn't being set properly from the xinit path.
|
|
#
?
Apr 26, 2015 01:07
|
|
|
spankmeister posted:Maybe you need to start it after pulseaudio From the reading I've done, I believe it's supposed to start pulseaudio automatically when it's needed. Nevertheless, I tried your suggestion and started it manually with /usr/bin/start-pulseaudio-x11, and then launched the kodi service after this. I no longer get the 10 second hang, and I can see my pulseaudio device in the settings now, however I get no sound output. When I run pacmd, I can see everything there as it should be. I'm reluctant to mess too much with my pulseaudio configs, because I know it works when I'm running an x session, but I don't know if that offers some insight into what's going on...
|
|
#
?
Apr 26, 2015 01:09
|
|
|
On Windows, EasyBCD has an option to install its own bootloader that can boot off USB drives even on computers that don't normally support it. Is there anything like that for Ubuntu? Ironically, I used EasyBCD to install Ubuntu on this machine, deleted the Windows partition and then found that the only program that could help me was not available for Linux. Alternatively, is there any way to install Lubuntu directly from within Ubuntu without booting off external media first?
|
|
#
?
Apr 26, 2015 03:54
|
|
|
Ur Getting Fatter posted:On Windows, EasyBCD has an option to install its own bootloader that can boot off USB drives even on computers that don't normally support it. I've hade luck with the Plop Boot Manager.
|
|
#
?
Apr 26, 2015 09:20
|
|
|
Prince John posted:I no longer get the 10 second hang, and I can see my pulseaudio device in the settings now, however I get no sound output. When I run pacmd, I can see everything there as it should be. Many distributions have some arrangement that makes audio devices accessible only to the user that is physically at the computer. This is to prevent pranks like another non-root user SSHing in, setting the volume to maximum and playing something highly annoying. I think it is commonly done with PAM: either by giving the user that is logging on at the console some extra group memberships, or by modifying the permissions (or even ACLs) of the audio devices in /dev/snd/ in a suitable way. After a bit of googling, it seems that Fedora uses the "audio" group. If it isn't already done, add the "kodi" user to the "audio" group, to make it have access to sound devices whether or not it is part of a console session: code:
|
|
#
?
Apr 26, 2015 11:50
|
|
|
Ur Getting Fatter posted:Alternatively, is there any way to install Lubuntu directly from within Ubuntu without booting off external media first? sudo apt-get install lubuntu-deskop
|
|
#
?
Apr 26, 2015 13:22
|
|
|
kujeger posted:I've hade luck with the Plop Boot Manager. This worked, thanks!
|
|
#
?
Apr 26, 2015 23:24
|
|
|
Thanks for the response telcoM. I can confirm that the kodi user was definitely in the audio group, which made it even more mysterious. I'm afraid I copped out in the end (I'd also wasted a lot of time fighting selinux without realising it on another issue) and just installed Debian as there's a new release out, I'm more familiar with it and never seem to run into the niggly issues I encounter with Fedora. Edit: vv Yeah, thanks evol. It was entirely me being dumb - I bashed my head against the wall for ages before the thought of selinux even occurred to me. Permissive mode worked fine, once I remembered. The curse of not normally using an selinux enabled distro I think. 
Prince John fucked around with this message at 17:01 on Apr 27, 2015 |
|
#
?
Apr 27, 2015 10:56
|
|
|
audit2allow is your friend (as is permissive mode). The combination of these two will basically write rules for you to allow what you're doing without actually blocking it. Regarding kodi, I was away for the weekend, but there's a very good mpd doc which will probably help
|
|
#
?
Apr 27, 2015 14:35
|
|
|
https://twitter.com/lattera/status/592746942121279488 wat
|
|
#
?
Apr 27, 2015 18:56
|
|
|
Finally got Lubuntu installed and Wine is failing to run an EXE on it that worked perfectly on Ubuntu. The program's installer runs all the way to the end and the tells me the installation failed. I get that they're different distros so I might just be hosed, but maybe I'm just missing some dependencies or something?
|
|
#
?
Apr 27, 2015 19:07
|
|
|
This is years old. Also, people need to stop conflating "systemd" (the binary which handles init) with systemd, the project. This particular bit is for journald. Because they (rightly) decided that inventing yet-another log shipping format is completely irrelevant when http+json does what they need and makes endpoints really easy to do.
|
|
#
?
Apr 27, 2015 19:07
|
|
|
evol262 posted:
do you think this would be solved if one or the other got a slightly different name? e- just to be clear, i am pro-systemd
|
|
#
?
Apr 27, 2015 19:13
|
|
|
Captain Foo posted:do you think this would be solved if one or the other got a slightly different name? Yes. Especially if they renamed systemd to systemd-init (in the same way as they're doing systemd-nspawn, etc). But in general, people should stop treating systemd as a monolithic project and start treating it like coreutils. systemd is a little more modular and a little less piecemeal, but nobody would flip their poo poo if some component in coreutils added some feature. networkd, journald, timed, and the other daemons with obvious "non-systemd" names are a good start (even if they're part of the systemd repo). This particular change was in journald, not systemd. But meh.
|
|
#
?
Apr 27, 2015 19:27
|
|
|
When your kernel core dumps and doesn't boot, it prints out a QR code on the console so that you don't have to take an awkward camera pic of an error message, you just scan it with your phone. It's super useful. HTTP server stuff is journald replication, and it's completely optional. It also uses microhttpd, it doesn't have its own HTTP server codebase.
|
|
#
?
Apr 27, 2015 19:44
|
|
|
so as i expected, the people tweeting about this should not really be taken seriously about this in any way
|
|
#
?
Apr 27, 2015 19:55
|
|
|
But WHY DOES MY INIT HAVE HTTP U GUISE
|
|
#
?
Apr 27, 2015 20:12
|
|
|
It's more people going "look at all this code!!!! that's a lot of code!!!" and not realizing that they're running 100K lines on boot without systemd regardless and we make it work.
|
|
#
?
Apr 27, 2015 20:17
|
|
|
Ur Getting Fatter posted:Finally got Lubuntu installed and Wine is failing to run an EXE on it that worked perfectly on Ubuntu. The program's installer runs all the way to the end and the tells me the installation failed. Ubuntu and Lubuntu are really not different distros, just different desktops. It is true there will be a lot of different software choices but that is usually only things like a GTK based app instsad of a QT app. Under the hood, they should be the same. As proof, you can take your old Ubuntu install and install the "Lubuntu-desktop" package, which would install LXDE and a few other LXDE apps into a stock Unity based system. The end result would be you could choose to run Unity or LXDE whenever you want. In other words, if the exe worked on Ubuntu it should work on Lubuntu as well. What is the exe and what is the error message?
|
|
#
?
Apr 27, 2015 22:09
|
|
|
Crotch Fruit posted:Ubuntu and Lubuntu are really not different distros, just different desktops. It is true there will be a lot of different software choices but that is usually only things like a GTK based app instsad of a QT app. Under the hood, they should be the same. As proof, you can take your old Ubuntu install and install the "Lubuntu-desktop" package, which would install LXDE and a few other LXDE apps into a stock Unity based system. The end result would be you could choose to run Unity or LXDE whenever you want. EXE is 701Server.exe, a server program for Soyal electronic key systems. It's old-rear end software (although still supported, somehow, since they put out patches every now and then), I think the original version is from 2001. I'll post a screenshot of the error message when I get home, but it's something like "Installation failed" or something like that. The error message is thrown out by the installer itself (ie: not Wine or Lubuntu but the Microsoft Installer). I'm going to try installing Lubuntu from within Ubuntu itself, like you mention, and see if that helps.
|
|
#
?
Apr 27, 2015 22:18
|
|
|
Suspicious Dish posted:When your kernel core dumps and doesn't boot, it prints out a QR code on the console so that you don't have to take an awkward camera pic of an error message, you just scan it with your phone. It's super useful. The QR code is for the journal FSS; Linux still doesn't support any kind of useful kernel dumps or boot error reporting.
|
|
#
?
Apr 28, 2015 05:48
|
|
|
kmsg goes into the journal
|
|
#
?
Apr 28, 2015 05:58
|
|
|
|
| # ? Jun 11, 2024 10:08 |
|
|
If the kernel has crashed, the journal is no longer running. Anyway, the QR code is purely to output the forward sealing key, it doesn't do anything else, and its entirely optional.
|
|
#
?
Apr 28, 2015 07:02
|
|