animist posted:

okay, so, first thing you gotta understand is what people mean by "deep neural networks". a neural network is made up of "neurons", which are functions of weighted sums. thats it. here's a "neuron":

code:

W = [10, 300, -.5]

def car_price(wheel_size, engine_horsepower, miles_driven):
  total = wheel_size * W[0] + engine_horsepower * W[1] + miles_driven * W[2]

  return (total if total > 0 else 0)

an ML person would say this is a good analogy for a biological neuron, and draw it like this:



inputs get multiplied by weights ([10, 300, -.5]), then summed, and passed through a nonlinear function. the function we're using is called ReLU, defined relu(x) = (x if x > 0 else 0). you might think thats a lovely function but it's super common in deep learning for some loving reason

of course, neural network people hate labeling things, so theyd actually draw it like this:



which looks more convincing.

now, what if you don't know what weights to use? that's easy. pick some random weights. then, steal a dataset of car specs + prices from somewhere. pick a random car and feed its specs to your function. your function will return a price, which will be wrong. so, tweak your weights to make them more correct. this is easy to do, because you know how much the output will change if you change your weights. if the car you're looking at has a wheel size of 10, changing W[0] by 1.5 will change the output by 15. capische? so just tweak all your weights a little so your function's output is a little closer to the actual price.

now do this a bazillion times. if you're lucky, your network will now give good estimates for car prices. if it doesn't, you can always add more neurons:



the later ones are sums of the earlier ones, see. what do the ones in the middle mean? idk, but now your network can express more functions. the training algorithm still works the same way, divide-and-conquer style.

but that's childs play. thats barely any neurons at all. that's the sort of poo poo you'd see in a neural network paper from the 1980s. weve got gpus now. you can throw as many neurons as you want around, in giant 3d blocks of numbers, each made up of the sums of other giant 3d blocks of numbers. just go hogwild:








these are all state-of-the-art networks. if you take a long time and learn a bazillion tricks to train them correctly, you can get these to give pretty good accuracies for benchmark problems. they were all discovered by, basically, loving around. there's barely any theoretical basis for any of this. machine learning!

this post got fuckoff long so im not gonna even post about interpretability. just think about picking some numbers from the middle of those giant networks and trying to decide if they're racist. now imagine your career hinging on getting good results from that. welcome to my grad program

eschaton posted:

if I were asked to make an “object recognizer,” I wouldn’t train one huge network on a million images of the object I want to recognize and allow steganography to break everything, I’d train a large number of smaller networks on different characteristics to recognize, and use additional separate networks to determine confidence according to recognitions, etc. finally arriving at the one confidence value

rchon posted:

don't ask me about deploying mask r-cnn models to production systems with no quantifiable validation metrics or model versioning

suffix posted:

should i be blindly copying code i don't understand from random repositories

animist posted:

lol

https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/

"To get a better handle on what the full development pipeline might look like in terms of carbon footprint, Strubell and her colleagues used a model they’d produced in a previous paper as a case study. They found that the process of building and testing a final paper-worthy model required training 4,789 models over a six-month period. Converted to CO2 equivalent, it emitted more than 78,000 pounds and is likely representative of typical work in the field."

that's the lifetime emissions of 5 cars btw

Winkle-Daddy posted:

oh look, the handy work of Summly, the "AI News Summarizing Product" built by some kids that Marissa gave like a billion dollars to. Working at yahoo under marissa was the fuckin worst. "guys, guys, don't you see how valuable tumblr is???"

quote:

Ray Phan, a senior computer-vision and deep-learning engineer at Hover, a 3D software startup based in San Francisco, told The Register that the lectures were confusing and contained mistakes. The online projects were too simplistic to be useful in the real world, and Raval was absent and unsupportive.

animist posted:

there's a bunch of "black box attacks" that work this way, only seeing inputs and outputs

often you don't even need those, though, because adversarial attacks usually transfer between networks. so you can just train your own model, and as long as it's even vaguely similar to whatever you're attacking, images that trick it will probably also trick your target

Pinterest Mom posted:

animist posted:

update: instead of learning, it is drifting about filter feeding in the abyssal zone. probably for the best

power botton posted:

if you want to do math in a dumb dead language use fortran

Feisty-Cadaver posted:

lol

https://twitter.com/bschulz5/status/1212198171436310533?s=21

quote:

Numbering System

The first digit of a CS course number indicates its general level of difficulty:

0-99 service course for non-technical majors
100-199 other service courses, basic undergraduate
200-299 advanced undergraduate/beginning graduate
300-399 advanced graduate
400-499 experimental
500-599 graduate seminars
The ten's digit indicates the area of Computer Science it addresses:

00-09 Introductory, miscellaneous
10-19 Hardware Systems
20-29 Artificial Language
30-39 Numerical Analysis
40-49 Software Systems
50-59 Mathematical Foundations of Computing
60-69 Analysis of Algorithms
70-79 Computational Biology and Interdisciplinary Topics
90-99 Independent Study and Practicum

Sagebrush posted:

ah well then it's probably exactly what it says on the box. "grad students, go make a thing, if you need more information here's a list of papers to read, i'll be writing grants. if you make anything cool i'm putting my name as first author"

DELETE CASCADE posted:

what kind of hosed up PI doesn't let the main grad student on the project be first author? how else will he ever graduate

animist posted:

also tbh if you're trying to understand data I'd reach for visualizations wayyy before ML because ML will just spit black boxes back at you. that's because I like to understand things tho

rchon posted:

this network has learned something but no animal ever has

ultrafilter posted:

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead