The Ultimate BSD Thread™ - The Devil Wears a Neckbeard

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Ultimate BSD Thread™ - The Devil Wears a Neckbeard

«‹›34 »

shrughes: Oct 11, 2008; (call/cc call/cc)

Ninja Rope posted:

Yep, it's pretty stupid. You should be able to install 9.0 and have everything work as normal.

Yeah. I did that. It worked. emacs runs.

# ? Jan 26, 2013 09:13

Adbot: ADBOT LOVES YOU

# ? May 23, 2024 23:05

cowboy beepboop: Feb 24, 2001

icantfindaname posted:

You can use 'make config-recursive'

How is this not the default

# ? Feb 2, 2013 01:06

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Anyone here has practical experience in running a virtual ZFS fileserver with FreeBSD? How stable is it and how are the transfer speeds over the virtual network adapter (using SMB I guess)?

I'm interested in doing something like that. The platform of choice will be likely Hyper-V, since it's built into my host OS (Windows 8). Interestingly, there's a patchset to add Hyper-V support to FreeBSD.

# ? Feb 3, 2013 01:25

EvilMoFo: Jan 1, 2006

Combat Pretzel posted:

The platform of choice will be likely Hyper-V, since it's built into my host OS (Windows 8). Interestingly, there's a patchset to add Hyper-V support to FreeBSD.

I would not say that hyper-v support is quite that ready.

# ? Feb 3, 2013 02:40

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Yeah, I guess that figures. I stumbled into freebsd-virtualization, and the guys responsible for the drivers wanted clarification just last week about some things before moving their stuff into -CURRENT. I guess I'll be holding that project off a while longer (VMware would be an option, but I'd like the virtual server to start at boot before the login screen).

# ? Feb 3, 2013 03:30

EvilMoFo: Jan 1, 2006

In Workstation 9 there are shared virtual machines that can run at boot.

Good to hear the code is about to hit current though.

# ? Feb 3, 2013 06:06

JamesOff: Dec 12, 2002; What a frightening beast!

my stepdads beer posted:

How is this not the default

If you use portmaster (/usr/ports/ports-mgmt/portmaster) to install/upgrade things, it does all the configs first and also downloads source archives in the background while other things are building.

# ? Feb 3, 2013 13:45

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Well, gently caress me. A quick test under Hyper-V got me a sustained network peak of around 30MB/s over the emulated network adapter using samba36. I was expecting way worse. I wonder how it'll perform once the Hyper-V drivers are in the kernel.

# ? Feb 3, 2013 21:08

cowboy beepboop: Feb 24, 2001

JamesOff posted:

If you use portmaster (/usr/ports/ports-mgmt/portmaster) to install/upgrade things, it does all the configs first and also downloads source archives in the background while other things are building.

Thanks, will give that a go next time.

I have a bunch of old (FreeBSD 4 - 6) servers I need to upgrade, anyone had any experience skipping versions? Ideally I'd like to head to 9.0 for now, and 9.1 as soon as it's stabilised a bit.

# ? Feb 3, 2013 22:09

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

When I suspend my computer, e.g. over night, the FreeBSD VM of mine gets frozen. When I wake it up, FreeBSD continues counting time from when it was frozen. ntpd then gets a poo poo fit because the clock usually drifted more than 1000 seconds. I've put ntpd_flags="-g" in rc.conf, but the -g flag does really only seem to work when I run ntpd manually. Any idea how to make this work the proper way (i.e. not a cron job, ironically)?

# ? Feb 5, 2013 14:36

falz: Jan 29, 2005; 01100110 01100001 01101100 01111010

Put something in /etc/apmd.conf that does:

code:

/etc/ntpd stop
/etc/ntpdate start
/etc/ntpd start

maybe something like this? the first exec was already in there

code:

apm_event STANDBYRESUME {
        exec "/etc/rc.resume apm standby";
        exec "/etc/ntpd stop";
        exec "/etc/ntpdate start";
        exec "/etc/ntpd start";
}

falz fucked around with this message at 18:17 on Feb 5, 2013

# ? Feb 5, 2013 18:13

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

When suspending the host, HyperV stops scheduling the VMs. They aren't even aware about it.

--edit: Something else... Realizing a FreeBSD router, is it possible to configure ipfw/natd to forward all unknown inbound traffic to a specific host? Similar to a home router DMZ setting? --edit: If I use redirect_address in natd, it would still map outgoing connections correctly, right?

natd works fine!

Combat Pretzel fucked around with this message at 22:16 on Feb 7, 2013

# ? Feb 7, 2013 21:49

Ninja Rope: Oct 22, 2005; Wee.

Could you set tinker panic 0 in ntp.conf and then run ntp with -x?

# ? Feb 7, 2013 21:58

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Thanks, that was what I was looking for!

# ? Feb 7, 2013 22:15

Xenomorph: Jun 13, 2001

FreeBSD: Is this new? Samba 4 final is now in ports.

portsnap just pulled in Samba 4, patches from Feb 18th, version 4.0.3.

I now have ports/net/samba4 in addition to ports/net/samba4-devel. Is it safe to delete samba4-devel?

Link:
http://svnweb.freebsd.org/ports/head/net/samba4/

Xenomorph fucked around with this message at 21:05 on Feb 22, 2013

# ? Feb 22, 2013 20:49

JamesOff: Dec 12, 2002; What a frightening beast!

There's a note about it in /usr/ports/UPDATING

# ? Feb 23, 2013 06:52

feld: Feb 11, 2008; Out of nowhere its.....

Feldman

shrughes posted:

Yesterday, I tried installing FreeBSD. I tried using the ports collection. It ended up creating an interminable series of configuration menus, spaced 10 minutes apart.

I discovered a command pkg_add and tried using that. It was trying to access a URL from a FreeBSD ftp server that did not exist. It was trying to access ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.1-release.

How can they release a system where installing packages doesn't work? The last time I had an installation experience as bad as this was when trying to install Arch Linux.

I wanted/needed to port some software to FreeBSD, a port that some people were asking for, but apparently they can't have that because the operating system doesn't really work.

I'm sorry FreeBSD isn't a Linux and doesn't have your apt-get or yum, but this is an obvious troll. Why would you try an entirely new OS and never take 5 minutes to read any of the documentation about how it works? FreeBSD is less frustrating than any Linux distro I've ever used, so I'm not sure why this is so difficult for you.

If you could tell me which piece of software you're trying to port I'll gladly do it myself.

# ? Feb 23, 2013 16:45

Xenomorph: Jun 13, 2001

feld posted:

I'm sorry FreeBSD isn't a Linux and doesn't have your apt-get or yum, but this is an obvious troll. Why would you try an entirely new OS and never take 5 minutes to read any of the documentation about how it works? FreeBSD is less frustrating than any Linux distro I've ever used, so I'm not sure why this is so difficult for you.

If you could tell me which piece of software you're trying to port I'll gladly do it myself.

How is it a troll? FreeBSD actually released software (9.1) that half didn't work. pkg_add works fine for 9.0, but not 9.1 (unless you change an environmental variable manually). The pkgng beta repositories were emptied and the pkg_add repositories never existed.

It's not entirely FreeBSD's fault (as they had the security breach), but they did release their own tools/commands that pull up 404 errors and completely fail to function "out of the box" due to missing paths and binaries. Anyone coming to the platform from any other functioning platform (Mac OS X, Windows, Ubuntu Linux, Red Hat/CentOS Linux, etc) would see some serious loving red flags if after a new install the OS was throwing 404 errors every time you tried to install something - especially with a command that worked with their last release.

Some people don't want a software install to sit for 12 hours downloading sources & compiling every single file when they can just type a command to load a pre-compiled binary in 5 minutes. While the end result is most likely the same, I don't have all day to wait for poo poo to compile.

# ? Feb 23, 2013 20:12

SamDabbers: May 26, 2003

If you have multiple FreeBSD boxes, or just one box with several jails, you can set up one machine to build your own package repo with just the packages you'll use and their dependencies.

The poudriere* tool can build packages for multiple OS versions (e.g. 8.3-RELEASE, 9.1-RELEASE, and 10-CURRENT) regardless of the host version, automatically parallelizes builds based on the number of CPU cores in your build machine, and can be run unattended via script/cron. It's pretty slick, and you won't have to rely on the official repos anymore.

You don't necessarily have to serve the packages out via HTTP/FTP if you're just going to use them with jails on the local box. The packagesite variable in pkg.conf supports file:/// URLs too, so you can use nullfs mounts of your repo dir in each jail.

*requires a ZFS filesystem

# ? Feb 23, 2013 20:50

Xenomorph: Jun 13, 2001

It just seems like that is making things overly complex.

Instead of setting up a local repository, I just set PACKAGESITE to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable/Latest/ and was done with it. The FreeBSD people already compiled the stuff. I just want to install it.

# ? Feb 23, 2013 21:39

shrughes: Oct 11, 2008; (call/cc call/cc)

feld posted:

Why would you try an entirely new OS and never take 5 minutes to read any of the documentation about how it works?

Because people were asking me to port some software to FreeBSD.

And I did read the documentation.

# ? Feb 23, 2013 22:10

SamDabbers: May 26, 2003

Xenomorph posted:

It just seems like that is making things overly complex.

Instead of setting up a local repository, I just set PACKAGESITE to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable/Latest/ and was done with it. The FreeBSD people already compiled the stuff. I just want to install it.

I agree that it is more complex to set up your own repo, but there are some advantages if you do. First, the packages at that packagesite are over 4 months old, and poudriere uses portsnap so you always have the latest version of the ports tree, which is updated constantly. For example, the Samba 4.0 release hit the ports tree on Feb 18, and the version in the packagesite is alpha11, which was very old even when it was built. Also, if you are managing several boxes and/or jails, you can ensure all of your hosts have not only the latest bugfixes/security updates, but the SAME version packages, which can reduce certain types of sysadmin pain.

# ? Feb 23, 2013 23:32

feld: Feb 11, 2008; Out of nowhere its.....

Feldman

Xenomorph posted:

How is it a troll? FreeBSD actually released software (9.1) that half didn't work. pkg_add works fine for 9.0, but not 9.1 (unless you change an environmental variable manually). The pkgng beta repositories were emptied and the pkg_add repositories never existed.

The public package repository has never been meant for production use. Those packages are built from a snapshot of the ports tree at the time the OS was released. There was never a tool to *upgrade* your installed packages in the base system, and the package repository for your release rarely (if ever) gets updates. It exists to quickly get your system up and running and that's about it. The ports tree has been the proper solution for standalone systems, and if you run many servers the traditional method was running your own private package repo built from Tinderbox.

If you're going to hate on FreeBSD it's probably about time you start hating on OpenBSD too as they never release package updates and (worse) they expect you to run -CURRENT builds (updating the OS to the latest snapshot every month or so) and use the -CURRENT package repository if you want binary packages. Otherwise you're expected to build your own from their ports tree.

NetBSD currently has the best setup for binaries in my opinion, but it's not available out of the box. If you install their new package management tool "pkgin" you will get a nice interface to install/search/update/upgrade/etc. They appear to do a better job of keeping their packages up to date, but they also do not have a modular package system. Just like on FreeBSD, if you install the PHP package you will *not* get the Apache mod_php module. You have to build it yourself from ports or run your own package repository where PHP is built with the MOD_PHP option enabled.

FreeBSD's pkgng is currently the best package management tool on the *BSDs and it was heavily influenced by the best features of every package management tool out there. I honestly feel it's the best out there period. Writing RPMSpecs or creating a deb is a nightmare compared to building a FreeBSD port.

Anyway, the next step for the FreeBSD world is the migration to "subpackages" which is what the Linux world is used to. Every build option for a program will be available as their own subpackages so you can cherry pick any feature you want from the public repository without having to compile anything. This requires a major modification to the current repository build tools and some heavy lifting of the ports tree, but it's certainly achievable and I hope to see this near completion before 2014.

feld fucked around with this message at 16:46 on Feb 24, 2013

# ? Feb 24, 2013 00:26

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

Hrm, UFS2 sure seems to be some sensible snowflake. I've had a test VM running here for two weeks in a row, essentially just serving files on a ZFS pool. The system itself is on an UFS2 partition, and apart from writes to logs, there probably shouldn't have been anything much in IO these two weeks. I had to reset my box earlier because a session of 3dsmax went haywire and sent my system swapping like gently caress. The UFS2 partition is pretty much FUBAR. What could have caused this? Files that have been created and last accessed these same two weeks ago are broken.

--edit: After forcing fsck over it, it'S completely hosed. I hope ZFS root will prove more stable to unexpected reboots/power outages.

Combat Pretzel fucked around with this message at 14:44 on Feb 24, 2013

# ? Feb 24, 2013 04:00

feld: Feb 11, 2008; Out of nowhere its.....

Feldman

Combat Pretzel posted:

Hrm, UFS2 sure seems to be some sensitive snowflake.

Uhh, that's not good. UFS2 is probably the most resilient filesystem I've ever seen. I have tons of VMs with UFS2 that I just hard reboot right in the middle of I/O and they always come up clean after a fsck. Usually when this happens you have hardware dying somewhere.

# ? Feb 24, 2013 16:45

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

The hypervisor is Hyper-V. The disk devices are all emulated. At this point, I believe the dynamically expanding VHDX file took a poo poo on all of it.

# ? Feb 24, 2013 16:53

EvilMoFo: Jan 1, 2006

ew, dynamically expanding .... I seem to remember reading that is best avoided with freebsd

# ? Feb 24, 2013 17:09

Goon Matchmaker: Oct 23, 2003; I play too much EVE-Online

FreeBSD's Hyper-V support is still pretty new too.

# ? Feb 24, 2013 18:16

Longinus00: Dec 29, 2005; Ur-Quan

Combat Pretzel posted:

The hypervisor is Hyper-V. The disk devices are all emulated. At this point, I believe the dynamically expanding VHDX file took a poo poo on all of it.

That's an extra level of caching the data has to go through before it reaches the disk. Not to mention write barriers are going to be emulated and you have a whole other filesystem to corrupt your data with. Basically don't use your desktop computer to serve files you care about with a vm, especially if swapping forces you to powercycle.

# ? Feb 24, 2013 20:47

Combat Pretzel: Jun 23, 2004; No, seriously... what kurds?!

The "important" files are on a ZFS pool, and that one survived everything just peachy, despite magnitudes more IO than on the UFS2 partition. It was only the OS that got fubared. As far as flushing and write barriers go, files that were last touched a fortnight ago were affected, too (like the ports I've compiled, pretty much most files corrupted). I'd assume that they and their metadata would have hit the disk by the time the failure happened. Now that it is mentioned that FreeBSD doesn't like expanding VHDXs, I guess it makes sense to assume that failure happened there either way, the metadata about block layout getting corrupted. I'm rerunning the setup with a fixed size VHDX and a ZFS rool pool. We'll see how that goes. It's only a test setup right now, to find out about issues like these.

(As far as the swapping goes, 3dsmax going crazy and allocating a lot more than the 10-12 gigabytes of free remaining memory was kind of completely unexpected. Actually in the case of buggy piece of crap 3dsmax, this should be expected.)

# ? Feb 25, 2013 00:02

Xenomorph: Jun 13, 2001

So is it standard practice for FreeBSD to let SEVERE DATA-CORRUPTING BUGS just go unresolved for months?

Issue: the "mfi" driver (used in Dell PowerEdge systems with a "PERC" H310 RAID and JBOD card, aka the LSI SAS2008) only "sees" up to 2TB. On drives larger than 2TB, it just rolls over and starts overwriting data at the beginning of a drive.

FreeBSD 9.1 went "final" on December 4th, 2012. The fixed driver (part of the kernel) wasn't patched until December 10th, 2012, which made it too late to include in FreeBSD 9.1 final.

FreeBSD posted this:
http://www.freebsd.org/releases/9.1R/errata.html

quote:

[November 2, 2012] The current mfi(4) driver has an overflow bug when handling disks larger than 2^32 sectors in SYSPD volumes, also known as JBODs, which will cause data corruption. This bug has been fixed on this FreeBSD-CURRENT but was too late for inclusion in this release. An Errata Notice for 9.1-RELEASE is planned.

What does that mean, "an Errata Notice for 9.1-RELEASE is planned"? Is that link ("errata.html") not an errata notice? When will this errata notice be released? Will "freebsd-update" ever have an update/fix for the kernel?

If they knew that data corruption-causing kernel was released, why has three months passed without them releasing a fix?

I added the updated mfi driver to my 9.1-RELEASE kernel source, recompiled it, and rebooted my server using it. So I think I should be OK. But isn't there any automated way to update this? My Ubuntu & CentOS boxes download kernel updates as part of their standard "update" command. Does FreeBSD do that?

I avoided 9.0-release due to bugs in mfi. I went with 9.1-release because it was supposed to have fixes in mfi. It turns out it has even more serious mfi bugs. Is the solution to just NEVER use a "-RELEASE" version, and only stick with "-STABLE" versions?

# ? Mar 6, 2013 16:31

SamDabbers: May 26, 2003

Xenomorph posted:

I added the updated mfi driver to my 9.1-RELEASE kernel source, recompiled it, and rebooted my server using it. So I think I should be OK. But isn't there any automated way to update this? My Ubuntu & CentOS boxes download kernel updates as part of their standard "update" command. Does FreeBSD do that?

I avoided 9.0-release due to bugs in mfi. I went with 9.1-release because it was supposed to have fixes in mfi. It turns out it has even more serious mfi bugs. Is the solution to just NEVER use a "-RELEASE" version, and only stick with "-STABLE" versions?

They will likely release a new 9.1-RELEASEp1 kernel through freebsd-update eventually. Just like with ports, the main way of maintaining and updating a FreeBSD system is updating/compiling the source. Even if you follow the -STABLE tree, you'd still need to update/compile, which you can automate with some shell scripts if you were so inclined. If you have multiple machines that need the fix, it's not terribly difficult to copy the new kernel to them. Maybe look into setting up puppet (sysutils/puppet in ports) to automate some of this?

SamDabbers fucked around with this message at 19:06 on Mar 6, 2013

# ? Mar 6, 2013 18:31

Ninja Rope: Oct 22, 2005; Wee.

Is there any ETA on getting binary packages built for 9.1? Or is the plan to skip it for 9.1 and wait for 9.2? Is this communicated officially somewhere?

# ? Mar 14, 2013 18:38

hifi: Jul 25, 2012

Ninja Rope posted:

Is there any ETA on getting binary packages built for 9.1? Or is the plan to skip it for 9.1 and wait for 9.2? Is this communicated officially somewhere?

http://www.freebsd.org/news/2012-compromise.html

quote:

The binary package building infrastructure has undergone a major security review, and as a result many changes have been made to the code. The review completed on the 16th February and we are now in the process of bringing it up on new hardware. At this point, we expect new binary packages to be available in 2-4 weeks.

Unless there are any other major changes, this is planned to be the last status update to this page. An email will be sent to the FreeBSD announcements mailing list when the package build infrastructure is online and packages are once again available.

# ? Mar 15, 2013 01:58

Xenomorph: Jun 13, 2001

Is there some place I can read about Kernel changes?

quote:

# uname -a
FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64

freebsd-update apparently has downloaded the first 9.1 kernel binary update since December 4th, 2012 (the one that contains that data corrupting mfi driver).

They let that issue sit for a few months. I want to know if they acknowledged the fix in any release notes.

# ? May 6, 2013 19:22

hifi: Jul 25, 2012

Xenomorph posted:

Is there some place I can read about Kernel changes?

freebsd-update apparently has downloaded the first 9.1 kernel binary update since December 4th, 2012 (the one that contains that data corrupting mfi driver).

They let that issue sit for a few months. I want to know if they acknowledged the fix in any release notes.

There's supposed to be a errata notice "sometime" but there should be a PR for it. If you want to track commits yourself there's http://freshbsd.org/ (this is more generic bsd oriented) and http://svnweb.freebsd.org/ . This will give you the commit of each kernel patch level for 9, and then every commit earlier than that will get included in that update.

# ? May 6, 2013 20:24

Xenomorph: Jun 13, 2001

Is there a way to see what version of mfi module the kernel has loaded?

# ? May 6, 2013 21:04

EvilMoFo: Jan 1, 2006

iirc, freebsd-update updates src files so you should be able to determine if the fix is included by looking at it

# ? May 7, 2013 16:08

Xenomorph: Jun 13, 2001

The "release" and "releng" src does not contain the fix, just "stable".

So... I just said "gently caress it" and updated the system from 9.1-release to 9.1-stable. I already had compiled my kernel and just about all the installed software (from ports), so I figured I may as well compile the whole system, just to make sure I had all the latest fixes and a working MFI driver.

Maybe I'll let it sit at "9.2-RELEASE" (and stick with binary updates) when that comes out. I'm guessing the MFI issues will be fixed by then (unless a new one pops up, as the issue was with 9.0 and 9.1).

It took about 40 minutes to buildworld, 10 minutes to buildkernel, then about 10 minutes to update my configs after running mergemaster.

# ? May 7, 2013 17:34

Adbot: ADBOT LOVES YOU

# ? May 23, 2024 23:05

feld: Feb 11, 2008; Out of nowhere its.....

Feldman

Xenomorph posted:

The "release" and "releng" src does not contain the fix, just "stable".

So... I just said "gently caress it" and updated the system from 9.1-release to 9.1-stable. I already had compiled my kernel and just about all the installed software (from ports), so I figured I may as well compile the whole system, just to make sure I had all the latest fixes and a working MFI driver.

Maybe I'll let it sit at "9.2-RELEASE" (and stick with binary updates) when that comes out. I'm guessing the MFI issues will be fixed by then (unless a new one pops up, as the issue was with 9.0 and 9.1).

It took about 40 minutes to buildworld, 10 minutes to buildkernel, then about 10 minutes to update my configs after running mergemaster.

1) If you're running bleeding edge hardware you should be running -STABLE or whatever HEAD is. Yes, that new weird Dell PERC controller is plenty bleeding edge. And I'd avoid HEAD until all the crazy cool new stuff for FreeBSD 10 is worked out.

2) Don't plan to run even numbered releases (9.0, 9.2) for long. They get very short (probably 1 year) of security updates before they're abandoned. Odd numbered releases get ~3 years.

3) Your mergemaster should take about 1 minute max. Use this mergemaster.rc file -- put it in /etc/

Xenomorph posted:

Is there some place I can read about Kernel changes?

svn co http://svn.freebsd.org/base/stable/9 /usr/src && cd /usr/src && svn log | more

I also recommend you subscribe to the freebsd-stable and freebsd-current mailing lists. You should really voice your opinion about getting that backported. It likely won't make it into 9.1 though because it's probably a feature change / major update and they only do security fixes in a RELEASE.

edit: it looks like your fixes were probably MFC'd December 3rd. This commit looks way too intrusive to allow into 9.1. FreeBSD will only backport security fixes. Anything that may change behavior must wait until the next -RELEASE or you can get it yourself by tracking -STABLE.

feld fucked around with this message at 20:16 on May 7, 2013

# ? May 7, 2013 20:02

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Ultimate BSD Thread™ - The Devil Wears a Neckbeard

«‹›34 »