Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Malcolm XML
Aug 8, 2009

I always knew it would end like this.

Dante80 posted:

How do you mean? The scheduling is the same that the consoles have, so it was pretty easy for AMD to do the same (it also makes logical sense due to the way the Uarch is arranged on both dies).

Microsoft and game engines on PC have different scheduling, and one of the jobs needed to port the game into PC from the newest consoles is to change said scheduling from what Jaguar does to what Intel does (AMD never had SMT before, so there was no need to do otherwise in the past).

Thus, without proper patches to Windows and the games themselves, we see Ryzen performance getting worse when SMT is enabled. Which also speaks a lot about how loving clown-ish and rushed the launch was...AMD stock has lost more than $1.5bn in two days due to their PR department lol.

TBH this is the game dev's fault for core locking without checking the layout of the processor, or core locking in general on a general purpose OS which is dumb.

I suspect AMD reused the core numbering system from bulldozer modules and since no game dev apparently tests on AMD...


The stock is still up 500% or so in a year; a lot of the selling is profit taking and idiots who thought that Ryzen would be competitive with the 7700K in gaming. Ryzen 7 was designed to steal Intel's high margin HEDT market and likely their low end server market, not the lower margin desktop stuff, which makes sense since AMD needs revenue.

There's also still the APUs, which apparently every single HPC client is salivating over, and Vega (lol) to shore up revenue. Lisa Su basically has AMD as a going concern, and that's good enough.

Adbot
ADBOT LOVES YOU

Maxwell Adams
Oct 21, 2000

T E E F S

Truga posted:

Are we reading the same thread?

"mse" is mean square error, i.e. "how much of this video is different from the lossless one"

x264 is noticeably better in general, and copes far better with high motion games like 1st/3rd person games, which I stream.

Any info on AMD's hardware encoding? OBS Studio just had an update that expanded support for it.

NewFatMike
Jun 11, 2015

Malcolm XML posted:

There's also still the APUs, which apparently a person interested in game performance on mobile platforms is salivating over.

Gotta keep that 1080p30 pipe dream for laptops. Fingers crossed a brave OEM will license out for TB3 ports. Lisa Su pls.

I think a clip from the level one show summed it up nicely, framing the R7s as server chips in a desktop platform. I'm interested in doing some home server experimentation with a rig based on that, although the I/O limitations on AM4 may be a limiting factor there.

Truga
May 4, 2014
Lipstick Apathy

Maxwell Adams posted:

Any info on AMD's hardware encoding? OBS Studio just had an update that expanded support for it.

AFAIK the new thing on 480 is at least in theory better than NVENC, which is already p. great. I haven't seen any benchmarks proper yet though, and either the OBS implementation or the drivers currently appear to be trash. It's the VCE rows in that table, down at the bottom.

Platystemon posted:

This is a tangentially related question, but I don't know of a thread it fits in better and we're on the subject anyway:

What hardware do commercial video streaming services use for encoding?

I'm guessing most will buy the specialized cards with h264/h265 chips (you can get anywhere from 1 to 16 of these cores on such a card). They cost a lot, but they'll transcode your video at near lossless in real time while also not eat 2000W doing it. They definitely have their quirks though, which can often be identified by feeding them certain videos (for example, twitch will gently caress your video up hard if you're playing doom 1 or 2 and have those shiny animated textures in your map, despite playing at a low low resolution of 320x200).

For google/youtube it's widely rumoured they just split your video into 5-10 second chunks and encode it on many servers though, because hey, they can. :v:

FuturePastNow
May 19, 2014


NewFatMike posted:

Gotta keep that 1080p30 pipe dream for laptops. Fingers crossed a brave OEM will license out for TB3 ports. Lisa Su pls.

I think a clip from the level one show summed it up nicely, framing the R7s as server chips in a desktop platform. I'm interested in doing some home server experimentation with a rig based on that, although the I/O limitations on AM4 may be a limiting factor there.

laptop makers segregate AMD APUs to a ghetto of flimsy plastic laptops with bad screens and poo poo batteries, and Zen probably won't change that

JnnyThndrs
May 29, 2001

HERE ARE THE FUCKING TOWELS

FuturePastNow posted:

laptop makers segregate AMD APUs to a ghetto of flimsy plastic laptops with bad screens and poo poo batteries, and Zen probably won't change that

My (constantly poor)buddy just bought one and that's absolutely true and it's too bad, because his graphics performance is fairly good compared to low end mobile Intel chips.

champagne posting
Apr 5, 2006

YOU ARE A BRAIN
IN A BUNKER

FuturePastNow posted:

laptop makers segregate AMD APUs to a ghetto of flimsy plastic laptops with bad screens and poo poo batteries, and Zen probably won't change that

Much like the FX-8800p, which on paper could've been a neat platform, was shoved into shite. I don't think I ever saw a laptop with the chip in it's high-tdp configuration which is a shame.

PC LOAD LETTER
May 23, 2005
WTF?!

ConanTheLibrarian posted:

AMD is still a loss making company. Cash flow problems could have dictated that they release Ryzen as soon as they could.
drat I know they've got cash flow problems for years but do you believe it was that bad? I mean if the company was gonna fail without a few months of stock boosting/CPU sales RIGHT now bad? Because to me that is the only way that would make sense but I've never been in that sort of position anywhere much less at AMD.

I vaugely remember some stock guy saying AMD would fail by 2018 or so if they didn't turn things around but that was almost 2yr ago now I believe.

edit: yeah, about a 3 month delay, another month or might've been OK. Certainly would've helped their PR.\/\/\/\/

PC LOAD LETTER fucked around with this message at 17:57 on Mar 4, 2017

Woden
May 6, 2006

PC LOAD LETTER posted:

drat I know they've got cash flow problems for years but do you believe it was that bad? I mean if the company was gonna fail without a few months of stock boosting/CPU sales RIGHT now bad? Because to me that is the only way that would make sense but I've never been in that sort of position anywhere much less at AMD.

I vaugely remember some stock guy saying AMD would fail by 2018 or so if they didn't turn things around but that was almost 2yr ago now I believe.

Ryzen has already been delayed once, if they delayed the launch too far it'd push back other SKU's.

fishmech
Jul 16, 2006

by VideoGames
Salad Prong

FuturePastNow posted:

laptop makers segregate AMD APUs to a ghetto of flimsy plastic laptops with bad screens and poo poo batteries, and Zen probably won't change that

Because systems where you're willing to start paying extra for graphics, you also want higher CPU performance which AMD just hasn't been able to give in the laptop space for like, gently caress, 15 years now? Unless you were willing to go full desktop replacement chunky laptop, because the AMD laptop stuff was simply more power hungry for performance.

Now that Zen has ok performance and ok power consumption, it might actually become worthwhile for anything besides low end garbage.

EmpyreanFlux
Mar 1, 2013

The AUDACITY! The IMPUDENCE! The unabated NERVE!
Again, there are a ton of half decent reason to release now, and one of them is to get any kind of market going for the AM4 platfrom. If these patches and fixes are needed without what would be huge reveals in Ryzen performance, then releasing to market your highest end processors make the most sense - capitalize on hype, capitalize on preorders, capitalize on higher margins. The people buying now either need what Ryzen offers (while still having not abysmal performance regardless of scenario because Ivy is still good enough) or they are die hards who will buy your product anyway. Use the now much larger market to push in fixes for the platfrom, launch your more gamer oriented SKUs with these fixes with your new consumer GPUs to maximize effect and prevent cannibalization of sales for your higher margin poo poo, and then follow up with mobile and budget release later in the year.

If AMD is really doing that, then a March release makes sense, because the budget/mobile release needs to happen in say, June/July, which means the Q2 release has to be mid April/very early May.

I may also be attributing too much strategic planning to AMD.

Dante80
Mar 23, 2015

I don't think there was any strategy involved. The CPU was already postponed from Q4 2016, and AMD had to show something, anything for Q1 to their investors. This would also explain the pre-order debacle (although it is pretty much standard practice lately).

They simply were late.

eames
May 9, 2009

I think preorder before NDA shifted a lot of 1800X/1700X sales that would have been 1700 sales had the reviews been public.

A local european price aggregator site shows sale charts. The order was 1800X - 1700X - 1700 - 7700k when preorders opened, now shifted to 1800X - 1700 - 7700k - 1700X and I'm pretty sure it'll be 1700 - 7700k -1800X - 1700X next week.

Oh and 6900K sales are trending upwards :v:

Dante80
Mar 23, 2015

I think this is another outlier.

https://www.youtube.com/watch?v=zRAuZUJF68I

He also posted this.

quote:

For any questions regarding discrepancies with benchmarking, some progressing evidence: https://goo.gl/Kqkfvm

That leads to this.

https://twitter.com/theoverclocker/status/837760151570403328

Which probably means that the guy is simply running a GPU bottlenecked bench and quoting someone else that does not provide details..just wants a clickbaity title to push his next magazine..

Dante80 fucked around with this message at 20:18 on Mar 4, 2017

ConanTheLibrarian
Aug 13, 2004


dis buch is late
Fallen Rib

PC LOAD LETTER posted:

drat I know they've got cash flow problems for years but do you believe it was that bad? I mean if the company was gonna fail without a few months of stock boosting/CPU sales RIGHT now bad? Because to me that is the only way that would make sense but I've never been in that sort of position anywhere much less at AMD.

A cash flow issue doesn't have to mean "we can't pay our staff next month unless we release today". It has been mentioned a lot that AMD hasn't been spending much on marketing for this launch. If they held back Ryzen until Vega was ready, they'd have even less to spend and they'd have to spread it between two different products.

Combat Pretzel
Jun 23, 2004

No, seriously... what kurds?!
Numerical evaluation by a computer program means poo poo, just like looking at a FFT doesn't tell you whether a MP3/AAC file sounds good or not. How it looks when played back is what matters.

Watermelon Daiquiri
Jul 10, 2010
I TRIED TO BAIT THE TXPOL THREAD WITH THE WORLD'S WORST POSSIBLE TAKE AND ALL I GOT WAS THIS STUPID AVATAR.
how would a qualitative measure of how close a compressed file compares to the lossless original not imply how good it is? I mean, i'm sure you could cheat it if theres like a repeating pixel pattern or something as the only difference, but for the most part it'd be accurate especially if you keep file size into account.

Platystemon
Feb 13, 2012

BREADS

Watermelon Daiquiri posted:

how would a qualitative measure of how close a compressed file compares to the lossless original not imply how good it is? I mean, i'm sure you could cheat it if theres like a repeating pixel pattern or something as the only difference, but for the most part it'd be accurate especially if you keep file size into account.

It turns out that people are not computers.

I mean, psychoacoustic/psychovisual weirdness is the basis of modern audio/video impression.

If the human brain primarily cared about strict numerical deviation, we wouldn’t be using the H.264 codec at all.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Combat Pretzel posted:

Numerical evaluation by a computer program means poo poo, just like looking at a FFT doesn't tell you whether a MP3/AAC file sounds good or not. How it looks when played back is what matters.

Yeah, sure, if you have a waveform and nothing to compare it against how would you even know? Let alone a waveform that's been run through a FFT?

Mean squared error against a lossless source file would tell you whether the MP3 sounds good or not though.

Yes, psychoacoustic models are a thing but that's still a numerical model that a computer runs which tells you whether it thinks a human would think it sounds good or not.

So yes, a computer actually can tell you whether it sounds good or not. At least, if your psychoacoustic model is a good one. No, they aren't perfect, yes, they usually are better than subjective reviewers. Ever read head-fi or some of the other "audiophile" forums to see what happens when you let subjective reviewers jerk themselves off?

Paul MaudDib fucked around with this message at 06:05 on Mar 5, 2017

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
At least with video encoder errors it's not going to blow your equipment up. But y'all should read NwAvGuy's blog because it has a lot of relevance to the Ryzen launch too.

quote:

BAD TESTING IS WORSE THAN NO TESTING: The great thing about the web is you can post something and share it with a very large number of people. If it's something that's obviously subjective such as a favorite restaurant, those reading your review know their tastes might be different that yours. But if something appears factual, and the author seems to know what they're talking about, that's very different. But what if it's really wrong or misleading? Because so few people are in a position to verify the results, and the few who can likely won't take the time, the bad data is very likely to go unchallenged. So it just hangs out there on the web to potentially mislead anyone who happens to find it. This happens more than you might think!

VIRAL TEST RESULTS: There are many examples where someone posts their test results for some piece of new gear and the numbers look great, they rave about it, and soon others run out and buy the same gear and post their own glowing subjective reviews, and before you know it the product has an almost cult-like following of fans on the web. But if you trace some of these back to their source, there's sometimes only a single set of sketchy results that helped start it all. And it can be a long time before anyone tries to verify the results—if ever. All the rest of the glowing reviews are often purely subjective--i.e. people's personal opinions. And those are biased from all the other positive comments and other influences.
...
TEST CONDITIONS MAKE A HUGE DIFFERENCE: If I publish a Ford Mustang can go from 0-60 MPH in 5.5 seconds that sounds fairly impressive. But is that on perfectly flat ground or was it down hill? Was there a head or tail wind? How much extra weight was in the car? How accurately was it timed? Was it some guy in the passenger seat with a wristwatch or with professional timing equipment? Was the road wet with lots of wheelspin or dry with good traction? All of these things will significantly change a car's 0-60 time--sometimes dramatically.

MISLEADING INFORMATION:
If I run a 0-60 test of a Mustang on dry pavement, and someone else does a 0-60 test on a fairly similar Chevy Camaro but does it in the rain, which one do you think will have the better result? Obviously the Mustang will get a lot more traction at the start and be the clear winner. But if you just saw the two numbers published on the web, with little explanation of how they were obtained, you might easily think the Mustang is a much faster car than the Camaro. This is exactly analogous to what happens when people test audio gear and post their results. While testing a car's acceleration in the rain is hopefully a bad idea to most people, many are not even aware they’re making similar mistakes during audio testing.

THE BEST TEST RESULTS CAN BE READILY COMPARED: What good are test results if you can't make valid comparisons to other results? That's why car magazines try to test cars under as controlled of conditions as possible. They correct for weight, wind, and even things like temperature which affects an engine's horsepower. So when they test a Mustang in January in Detroit and a Camaro in August in California, you can safely compare the results to each other. But most of the audio results being published on the web cannot be compared in similar ways because they're often measured under different or unknown conditions.

THE BEST TEST RESULTS ARE VERIFIED BY OTHERS: In the scientific and medical communities results that can't be verified are completely dismissed as invalid. But in audio many tend to take them as fact. When you conduct testing in a controlled way, it's much easier for others to verify your results. But if they don't know under what conditions the tests were made--what signal levels, loads, settings, with what equipment, etc.--they're almost impossible to verify. And without being able to verify the results, there's no way to know if they're reasonably accurate.

THE FINEST TESTING ADVERTISING WON'T BUY: There are certainly organizations out there with the equipment and knowledge to run proper tests. But, sadly, most of the consumer oriented magazines and sites tend to leave out, or gloss over, anything very negative for fear of losing their advertisers (which very often make the very gear they're testing). A classic example is even some relatively expensive (i.e. $1000) A/V receivers come nowhere close to their advertised power ratings under real world conditions. But tests on these receivers are often done in such a way to avoid revealing just how bad they really are). Why? Most likely because those same manufactures advertise with the same organization publishing the review.
http://nwavguy.blogspot.com/2011/02/testing-methods.html

So honest question here... how many people here had even heard of Joker Productions before he became /r/AMD's last great hope for Ryzen? Cause I sure hadn't.

TheJeffers
Jan 31, 2007

Paul MaudDib posted:

At least with video encoder errors it's not going to blow your equipment up. But y'all should read NwAvGuy's blog because it has a lot of relevance to the Ryzen launch too.

http://nwavguy.blogspot.com/2011/02/testing-methods.html

So honest question here... how many people here had even heard of Joker Productions before he became /r/AMD's last great hope for Ryzen? Cause I sure hadn't.

I looked into his methods and he's minimized CPU differences by testing with a lot of D3D12 titles. There's a point to be made there but it's hardly the best way to produce a CPU review.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Dante80 posted:

I think this is another outlier.

https://www.youtube.com/watch?v=zRAuZUJF68I

He also posted this.


That leads to this.

https://twitter.com/theoverclocker/status/837760151570403328

Which probably means that the guy is simply running a GPU bottlenecked bench and quoting someone else that does not provide details..just wants a clickbaity title to push his next magazine..

I mean there's no question that you need to enable "performance" mode to get the most out of any computer, I can literally feel the difference on my laptop because "power save" locks the processor to like 900 MHz and makes everything run like poo poo, and "balanced" runs it a little more freely and will make the bottom scorching hot because I haven't opened it up and vacuumed it out in like 4 years.

The SMT thing is probably legit too, wouldn't be surprised if AMD hosed it up on their new architecture, especially given the haste of the release.

The memory speed is also a thing, apparently the northbridge clock is tied to the memory clocks so if true the behavior here would be unexpected, increasing clocks and latency in step (ala 3000-CAS15 vs 3200-CAS16) should increase performance while you would usually expect it to remain linear. At least up until everything destabilizes.

But overall is all of this going to save Zen? Nah probably not, that'll happen when Zen 2.0 comes out and fixes all the errata.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

TheJeffers posted:

I looked into his methods and he's minimized CPU differences by testing with a lot of D3D12 titles. There's a point to be made there but it's hardly the best way to produce a CPU review.

Exactly, he tested at 1440p/4K and it hid any difference between the processors. You could probably have gotten the same results on a Bulldozer too.

And then Steve had to explain what a GPU bottleneck was to him live on his stream. While Joker was wearing a "make AMD great again" t-shirt.

I take my hardware advice from: GN, Anandtech, TechReport, TechPowerUp, in no particular order. And also David Kanter and Scott Waisson, although they don't do reviews anymore.

The rest are nice confirmation that someone didn't go senile, or miss some errata. More eyes is better. But when someone else comes up with a weird results... my initial assumption isn't that the other 6 guys screwed the pooch. Those guys are professional enough that if you prove they're wrong they'll say so.

Paul MaudDib fucked around with this message at 06:39 on Mar 5, 2017

AVeryLargeRadish
Aug 19, 2011

I LITERALLY DON'T KNOW HOW TO NOT BE A WEIRD SEXUAL CREEP ABOUT PREPUBESCENT ANIME GIRLS, READ ALL ABOUT IT HERE!!!

Paul MaudDib posted:

At least with video encoder errors it's not going to blow your equipment up. But y'all should read NwAvGuy's blog because it has a lot of relevance to the Ryzen launch too.

http://nwavguy.blogspot.com/2011/02/testing-methods.html

So honest question here... how many people here had even heard of Joker Productions before he became /r/AMD's last great hope for Ryzen? Cause I sure hadn't.

I had heard of him well before all the Ryzen stuff, he's been doing the tech youtube thing for a while. He has a history as a huge AMD fan boy and his results are well out of line with other reviewers, it's all circumstantial but I think everyone should take his results with a pretty huge grain of salt. :shrug:

Klyith
Aug 3, 2007

GBS Pledge Week
Here's a good article about the limits of standard algorithms for judging quality of lossy video. (cached version, I can't get to the original right now.)

a) it's somewhat more valid in the case of images than audio. Our ears suck, eyes are slightly less easy to fool. Unlike with psycho-acoustic audio compression, you don't get totally inverted relationships between perceived quality and PSNR or whatever. The perceptual models of our hearing are more mature than our vision, and it was an easier problem in the first place. The next generation of video encoding will maybe start to incorporate models of eye-tracking, face recognition, and stuff like that.

b) for cartoons and other artificial images, PSNR and other "dumb" algorithms track much better to perceived quality than for video or real things. The CH animation results in the second set of charts show that. Where a video game falls on that spectrum I don't know -- I suspect something like DOTA is more animation-like, and a 1st/3rd person action game more "real-world".

Paul MaudDib posted:

Mean squared error against a lossless source file would tell you whether the MP3 sounds good or not though.

No, in fact it does not. Optimizing MP3s for PSNR against the original is actively detrimental to sound quality. It's impossible to overstate how much our ears are "listening" to different things from the audio waveform.

And using the psychoacoustic model of a lossy algorithm to judge the results is dumb -- it's just going to grade itself as perfect. What you're trying to determine is the quality of the psychoacoustic model itself, which means scoring it by humans (in a proper experiment of course).

AzraelNewtype
Nov 9, 2004

「ブレストバーン!!」

Watermelon Daiquiri posted:

how would a qualitative measure of how close a compressed file compares to the lossless original not imply how good it is? I mean, i'm sure you could cheat it if theres like a repeating pixel pattern or something as the only difference, but for the most part it'd be accurate especially if you keep file size into account.

Nobody's used MSE in the context of video for ages, because it is not a reliable metric of human visual transparency. There are two much more commonly used metrics for this (PSNR: peak signal to noise ratio, SSIM: structural similarity), and neither of which is actually the same as a true human eye examination. x264 doesn't even optimize for these by default for exactly that reason. Psychovisual optimizations can yield improved visual transparency with decreased PSNR/SSIM values, and as such they're on by default.

Granted, I don't think the hardware encoders are doing particularly better psychovisual optimization here, but if the software encoder being used as the gold standard doesn't even think worrying about algorithmic quality standards is a great idea, I'm not sure why we should be.

Platystemon
Feb 13, 2012

BREADS

Paul MaudDib posted:

So yes, a computer actually can tell you whether it sounds good or not. At least, if your psychoacoustic model is a good one. No, they aren't perfect, yes, they usually are better than subjective reviewers. Ever read head-fi or some of the other "audiophile" forums to see what happens when you let subjective reviewers jerk themselves off?

A numerical comparison will never be better than a good, blind comparison made by actual human beings because you’re attempting to discern is human opinion.

As good as a computer model of a building might be, it will never more accurate than constructing the building and testing the real thing.

More convenient, definitely, but not more accurate.

Platystemon fucked around with this message at 06:59 on Mar 5, 2017

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Klyith posted:

Here's a good article about the limits of standard algorithms for judging quality of lossy video. (cached version, I can't get to the original right now.)

a) it's somewhat more valid in the case of images than audio. Our ears suck, eyes are slightly less easy to fool. Unlike with psycho-acoustic audio compression, you don't get totally inverted relationships between perceived quality and PSNR or whatever. The perceptual models of our hearing are more mature than our vision, and it was an easier problem in the first place. The next generation of video encoding will maybe start to incorporate models of eye-tracking, face recognition, and stuff like that.

b) for cartoons and other artificial images, PSNR and other "dumb" algorithms track much better to perceived quality than for video or real things. The CH animation results in the second set of charts show that. Where a video game falls on that spectrum I don't know -- I suspect something like DOTA is more animation-like, and a 1st/3rd person action game more "real-world".

I wasn't implying that MSE was the gold standard by any means. Does it tell you more than just looking at an encoded waveform with no reference waveform to compare against? Most definitely yes.

quote:

No, in fact it does not. Optimizing MP3s for PSNR against the original is actively detrimental to sound quality. It's impossible to overstate how much our ears are "listening" to different things from the audio waveform.

And using the psychoacoustic model of a lossy algorithm to judge the results is dumb -- it's just going to grade itself as perfect. What you're trying to determine is the quality of the psychoacoustic model itself, which means scoring it by humans (in a proper experiment of course).

That's why you have multiple psychoacoustic models. Run the MP3 through the OGG Vorbis psychoacoustic model and see what it says. If all the (high-quality) models say it's good, it's more likely to be good than something they say is lovely.

Secondly, it's not like you encode something against a psychoacoustic/visual model and that means it has zero error. Let's say I encode a 16 kbps MP3, do you think that would sound perfect?

Finally - what we have here are two totally separate decoders. We can most definitely look at the psychovisual error between the two of them and say which one the model says is better.

I firmly reject the idea that psychoacoustic/visual models have absolutely nothing to tell us. As does the entire field of audiovisual compression, in fact. That's why we have the psychoacoustic/visual models.

Do the psychoacoustic/visual models have to be validated by actual people? Yes, of course, you win, :fishmech:

NewFatMike
Jun 11, 2015

One day we'll be as good as the machines and I look forward to my Ray Kurzweil future.

Also hopefully the R7s get retested when the R5s and R3s come out because this driver/BIOS/errata situation, while comical, isn't that fun for regular consumers who would like to build a home server/workstation.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Platystemon posted:

A numerical comparison will never be better than a good, blind comparison made by actual human beings because you’re attempting to discern is human opinion.

As good as a computer model of a building might be, it will never more accurate than constructing the building and testing the real thing.

More convenient, definitely, but not more accurate.

Again, the problem with this is clearly evident if you've ever been around audiophiles.

Blind testing is a lot more effort than most people will go to and/or can go to (depending on the field).

If you know which sample you're looking at, your opinion is meaningless as far as empirical results are concerned. This has been long-since established.

Also, you're explaining this to a guy who was blind-testing his family on DIVX and XVID codec settings back in ~2002, age 13 :lol:

Paul MaudDib fucked around with this message at 07:24 on Mar 5, 2017

Klyith
Aug 3, 2007

GBS Pledge Week

Paul MaudDib posted:

I wasn't implying that MSE was the gold standard by any means. Does it tell you more than just looking at an encoded waveform with no reference waveform to compare against? Most definitely yes.


That's why you have multiple psychoacoustic models. Run the MP3 through the OGG Vorbis psychoacoustic model and see what it says. If all the (high-quality) models say it's good, it's more likely to be good than something they say is lovely.

Secondly, it's not like you encode something against a psychoacoustic/visual model and that means it has zero error. Let's say I encode a 16 kbps MP3, do you think that would sound perfect?

Finally - what we have here are two totally separate decoders. We can most definitely look at the psychovisual error between the two of them and say which one the model says is better.

I firmly reject the idea that psychoacoustic/visual models have absolutely nothing to tell us. As does the entire field of audiovisual compression, in fact. That's why we have the psychoacoustic/visual models.

Do the psychoacoustic/visual models have to be validated by actual people? Yes, of course, you win,

I am not sure what your original point was in that case, other than "I like to argue". The sentence I quoted was definitively wrong. If the way you want to make it correct is to say that MSE/PSNR is better than no data at all, then sure.

"Reversing" a perceptual lossy product through other codecs is not something that anyone would do -- at least for audio, the models that each codec uses are extremely similar and the places that they are different are tuned for the particular idiosyncrasies of that codec. You would still be going down the wrong path if your method for finding the best 16kbps MP3 is by MSE/PSNR similarity to 128kbps Ogg Vorbis.

Instead, you'd do what Netflix has done in that article I linked: build a second, unrelated, algorithm to judge quality based on a data set evaluated by humans (VMAF in this case). This is a perceptual model in itself, but it doesn't do the same thing as the ones in the codecs. They're not interchangeable. For a while the VMAF will give good results, but if the codecs start to incorporate VMAF's methods or tune themselves to it, the divergence between VMAF's results and real humans will increase.

AFAIK nobody built a "judgement" program for audio because the problem was easier and doing tests with humans is quick for audio compared to video.

:ironicat:

edit:

Paul MaudDib posted:

Again, the problem with this is clearly evident if you've ever been around audiophiles.

Blind testing is a lot more effort than most people will go to and/or can go to (depending on the field).

If you know which sample you're looking at, your opinion is meaningless as far as empirical results are concerned. This has been long-since established.

Also, you're explaining this to a guy who was blind-testing his family on DIVX and XVID codec settings back in ~2002, age 13 :lol:
Soooooo... because some people out there are wrong, the posters you are replying to must also be wrong even thought they're not saying the wrong thing, because you're Paul MaudDib and you were blind testing codecs at age 13? WTF?

Klyith fucked around with this message at 07:49 on Mar 5, 2017

Klyith
Aug 3, 2007

GBS Pledge Week
I mean if you want to argue with golden-ears audiophiles go to headfi or wherever they are. They're not hard to find! Don't argue with us by constructing imaginary positions we didn't say.


edit either that or you just have a really confrontational way of agreeing with people

Klyith fucked around with this message at 07:57 on Mar 5, 2017

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Klyith posted:

I am not sure what your original point was in that case, other than "I like to argue".

Project much?

quote:

The sentence I quoted was definitively wrong. If the way you want to make it correct is to say that MSE/PSNR is better than no data at all, then sure.

That was the point you missed, yes.

Klyith posted:

Instead, you'd do what Netflix has done in that article I linked: build a second, unrelated, algorithm to judge quality based on a data set evaluated by humans (VMAF in this case). This is a perceptual model in itself, but it doesn't do the same thing as the ones in the codecs. They're not interchangeable. For a while the VMAF will give good results, but if the codecs start to incorporate VMAF's methods or tune themselves to it, the divergence between VMAF's results and real humans will increase.

No poo poo sherlock, that's exactly what I proposed. Find yourself a second model to evaluate your first one..

You seem to be extra confrontational about that concept for some reason though. OK, Vorbis isn't a good candidate because it uses a model that is too similar, who gives a poo poo. Way to miss the point.

quote:

"Reversing" a perceptual lossy product through other codecs is not something that anyone would do -- at least for audio, the models that each codec uses are extremely similar and the places that they are different are tuned for the particular idiosyncrasies of that codec. You would still be going down the wrong path if your method for finding the best 16kbps MP3 is by MSE/PSNR similarity to 128kbps Ogg Vorbis.

See you understood perfectly well what I was referring to, that's why you brought up what Netflix did. Using a second model to validate your first one. But you want to argue because _________? :fishmech:

Paul MaudDib fucked around with this message at 08:41 on Mar 5, 2017

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Klyith posted:

I mean if you want to argue with golden-ears audiophiles go to headfi or wherever they are. They're not hard to find! Don't argue with us by constructing imaginary positions we didn't say.


edit either that or you just have a really confrontational way of agreeing with people

OK now that you've gotten the point, please don't ever again suggest that going to a forum thread with labelled samples is a good way to evaluate any sort of lossy compression codec.

I would pretty much take any sort of error in a reasonable attempt at an empirical measurement over resorting to head-fi bullshit. You do it or you don't, you don't do it halfway and bias your audience.

Paul MaudDib fucked around with this message at 08:49 on Mar 5, 2017

Klyith
Aug 3, 2007

GBS Pledge Week

Paul MaudDib posted:

OK now that you've gotten the point, please don't suggest that going to a forum thread with labelled samples is a good way to evaluate any sort of lossy compression codec.

I suggest maybe going back a page or two and re-reading, maybe keeping notes on who said what, because that's important.

edit: I'd even suggest reading the OBS thread in question, because the guy has a chart of MSE results and then talks about how the nvidia encoder is worse by visual examination, and says that MSE is not good enough. He then encourages people to download his samples to look at them.

Paul MaudDib posted:

You seem to be extra confrontational about that concept for some reason though.

See you understood perfectly well what I was referring to, that's why you brought up what Netflix did.

Dude, I made post about netflix and their evaluations of different ways of judging perceptual quality, and you fishmeched me. Of the people in this thread, you're the one interpreting anyone who quotes you as looking for a fight. People can say things about a topic without it being the opening salvo of a flamewar.

Klyith fucked around with this message at 08:54 on Mar 5, 2017

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE

Klyith posted:

I suggest maybe going back a page or two and re-reading, maybe keeping notes on who said what, because that's important.

Dude, I made post about netflix and their evaluations of different ways of judging perceptual quality, and you fishmeched me. Of the people in this thread, you're the one interpreting anyone who quotes you as looking for a fight. People can say things about a topic without it being the opening salvo of a flamewar.

Perhaps you would care to actually indicate what posts I missed? Because this sort of "make my argument for me" is actually a key element of fishmeching.

Also, to be technically correct (the best kind of correct) your post is inaccurate because I am not actually fishmech. :fishmech:

quote:

edit: I'd even suggest reading the OBS thread in question, because the guy has a chart of MSE results and then talks about how the nvidia encoder is worse by visual examination, and says that MSE is not good enough. He then encourages people to download his samples to look at them.

The actual meat of your post: I agree but that's why you post a couple samples and let people rate them themselves, then tell them what they voted on (i.e. a blind test). Ideally you wouldn't even let them see what other people are rating the samples.

Once you've posted labels to the samples their opinions are invalid and I'll move on to the empirical measurements thanks. Biased samples are useless. Actually worse because people will think they're legitimate.

My personal opinion: by my personal observations the NVENC encoder is poo poo unless you throw an immense amount of bitrate at it. My (1440p/60fps) FPS game recordings are done at 25 Mbps since Maxwell, and were 35 Mbps on Kepler. Quality gets lovely at no less than 10 Mbps and I felt bad when I was streaming at 5 Mbps. 1440p/60fps is a lot but it's less than you would think since faster framerate means smaller deltas per frame and higher resolution means more pixels for the compression to work on.

Same for the QuickSync encoder (worse actually).

I don't pretend those are scientific measurements though. That's the difference.

Paul MaudDib fucked around with this message at 09:08 on Mar 5, 2017

Platystemon
Feb 13, 2012

BREADS
I would like to suggest that it is not actually that hard to label some clips “A”/“B” and ask some people which looks better.

Computational testing is useful when you are developing an encoder, but when you are comparing a handful of existing options, that’s not an insurmountable volume to test on real live people.

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
Yeah. Blind testing is not even a big deal.

TBH I don't even know why there's not a website that you can post a link on a forum to that send them a "better: A or B" but it's randomly selected from 1 of N samples (served from your server) and bumps/drops the ratings correspondingly, that should be a thing. Where are the app creators when you need them?

eames
May 9, 2009

https://twitter.com/GamersNexus/status/838221363991166981

Adbot
ADBOT LOVES YOU

Paul MaudDib
May 3, 2006

TEAM NVIDIA:
FORUM POLICE
I wouldn't have had to do it if he weren't paid off by (((Intel)))!

--/r/amd

Paul MaudDib fucked around with this message at 10:12 on Mar 5, 2017

  • Locked thread