machine learning: this place is not a place of honor

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > machine learning: this place is not a place of honor

«‹›24 »

N-N-N-NINE BREAKER: Jul 12, 2014

flakeloaf posted:

i miss markov polov

wait what happened to it :ohdear:

did it become racist and get banned

# ? May 30, 2019 18:42

Adbot: ADBOT LOVES YOU

# ? Apr 27, 2024 22:53

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

thanks for ml learning advice

# ? May 30, 2019 19:21

huhwhat: Apr 22, 2010; by sebmojo

N-N-N-NINE BREAKER posted:

wait what happened to it
did it become racist and get banned

*makes jerking off motion* eventually!

# ? May 30, 2019 19:38

Sagebrush: Feb 26, 2012

DONT THREAD ON ME posted:

has anyone developed an ML algorithm that can tell you if the thing you're about to post is problematic?

i was thinking the other day that ML algorithms would actually be fantastic for identifying biases in a process, if anyone bothered to use them that way

# ? May 30, 2019 20:17

rotor: Jun 11, 2001; classic case of pineapple derangement syndrome

The Management posted:

it�s a loving filter.

all programs are filters.

Only registered members can see post attachments!

# ? May 30, 2019 20:28

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

rotor posted:

all programs are filters.

so i guess that makes AI the great filter?

# ? May 30, 2019 20:35

animist: Aug 28, 2018

ai won't kill us, but it should because we deserve it

# ? May 30, 2019 20:38

rotor: Jun 11, 2001; classic case of pineapple derangement syndrome

animist posted:

ai won't kill us

not all of us anyway, and certainly not all at once.

# ? May 30, 2019 20:42

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

honestly i think trumps brain is just a machine learning matrix for owning liberals

# ? May 30, 2019 20:44

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

reverse the direction so instead of reducing dimensionality, it increases, like https://science.sciencemag.org/content/358/6364/793

(like the second half of a autoencoder)

# ? May 30, 2019 20:47

I HAVE GOUT: Nov 23, 2017

Lysidas posted:

reverse the direction so instead of reducing dimensionality, it increases, like https://science.sciencemag.org/content/358/6364/793

(like the second half of a autoencoder)

i wanna make a robot nose that u put in ur house and uses ml to detect different smells like carbon monoxide or formaldehyde or various vocs or who just farted. and every once in a while, the model gets updated and sent to all noses so now it can know new smells.
two problems tho are:
1. if they dont match spec exactly (like down to the size of each hair sensor) then theyll each need to be retrained every time a new model gets pushed. u might be able to build that training data in factory, but itll still be super annoying
2. theyll probs get clogged, so youll need it to be able to sneeze once a week to clear itself out

# ? May 30, 2019 21:34

huhwhat: Apr 22, 2010; by sebmojo

this system discriminates against people who clean themselves regularly

# ? May 30, 2019 21:45

huhwhat: Apr 22, 2010; by sebmojo

oh yeah i just remembered that google was offering free gpu and tpu compute

https://cloud.google.com/tpu/docs/colabs
https://www.easterscience.com/google-is-offering-free-tpu-and-gpu-for-ai-using-colaboratory-colab/

now you can work on your (pattern matcher/filter/fancy stats/the other 96 names of machine learning) project without having it cut into your regular 4k hd 3d vr porn time because you had to use those sweet sweet twin gtx 2080ti's you bought that you were totes gonna use to make an ai chattebotte

# ? May 30, 2019 23:13

Cybernetic Vermin: Apr 18, 2005

google along with a lot of other actors have realized that a lot of successful ml simply involves ridiculous amounts of semi-skilled man-hours tweaking parameters and topologies randomly until the thing seems to do the thing, so it is worth it to try to capture more hands early on.

there's some glimmers of theoretical work happening on what works and what doesn't, notably lin-tegmark(-rolnick) https://arxiv.org/abs/1608.08225v4 , which does the very basic needful demonstrating that e.g. encoding multiplication requires exponentially wide hidden layers, which should reasonably be interpreted as "cannot be learned with current methods".

i never find the time, but i am personally quite sure that it is a pretty doable task to draw from minsky's perceptrons book (which famously killed off neural networks last time they were popular) the needed machinery, combined with reasonable assumptions on what is the limits of learning (e.g. finite precision) to demonstrate that some practical-sounding problems cannot be taught to deep neural networks without encoding as much information as a full table realization of the function requires. which should again be interpreted as "cannot be learned with current methods". i don't think such a paper would change the enthusiasm that much, but it is one of those directions that ought to be investigated sooner or later.

# ? May 31, 2019 10:29

distortion park: Apr 25, 2011

to do ml you need lots of data. there are a few types of data:
* open source standard image recognition etc data sets. only use this if you are confident your technique is better than the previous attempts.
* random free data you can find to download on the internet. this is normally boring stuff like poverty rates in different postal areas or how polluted a river is at different points along its course. skip this stuff unless you're doing a thesis for uni
* "free" data which isn't sold as such, but you can scrape from the web. this might how much a phone charger costs on amazon. more interesting but you will need to have some proxies and some dev time to get the data. the cheapest proxies are bot nets or these guys who totally have their endpoints consent https://luminati.io/
* financial data vendors will happily sell you lots of data for lots of money, but everything about the experience will suck. they will all handle edge cases in different ways. there will be bullshit restrictions on how you can use the data (only on one machine, can't make a graph and show it to a client etc) but you can ignore those
* the good stuff is the personal behaviour data. this is geolocation (from phones), internet searches and traffic (from free "antivirus"), emails (from email apps and spam removers), credit card transactions (idk exactly how they get this i think it's apps as well). this stuff is all expensive and the vendors are rubbish at delivery. occaisionally they'll gently caress up and send you some non-anonymised data (as though you couldn't work out who spent 8 hours at a school and then 3 at a strip club was if you really wanted to) at which point they'll email you and tell you to delete it and import a new version. might be tricky to get some of this past your ethics committee at uni but it's all legal, even in the EU as it is anonymised.

for supervised learning you'll need a training set. outsource this to india for best results.

distortion park fucked around with this message at 14:43 on May 31, 2019

# ? May 31, 2019 14:35

Schadenboner: Aug 15, 2011; by Shine

pointsofdata posted:

(as though you couldn't work out who spent 8 hours at a school and then 3 at a strip club was if you really wanted to)

Look, she�s working her way though Medical School, ok?

:mad:

# ? May 31, 2019 14:41

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

animist posted:

okay, so, first thing you gotta understand is what people mean by "deep neural networks". a neural network is made up of "neurons", which are functions of weighted sums. thats it. here's a "neuron":
code:
W = [10, 300, -.5]

def car_price(wheel_size, engine_horsepower, miles_driven):
  total = wheel_size * W[0] + engine_horsepower * W[1] + miles_driven * W[2]

  return (total if total > 0 else 0)
an ML person would say this is a good analogy for a biological neuron, and draw it like this:

inputs get multiplied by weights ([10, 300, -.5]), then summed, and passed through a nonlinear function. the function we're using is called ReLU, defined relu(x) = (x if x > 0 else 0). you might think thats a lovely function but it's super common in deep learning for some loving reason

of course, neural network people hate labeling things, so theyd actually draw it like this:

which looks more convincing.

now, what if you don't know what weights to use? that's easy. pick some random weights. then, steal a dataset of car specs + prices from somewhere. pick a random car and feed its specs to your function. your function will return a price, which will be wrong. so, tweak your weights to make them more correct. this is easy to do, because you know how much the output will change if you change your weights. if the car you're looking at has a wheel size of 10, changing W[0] by 1.5 will change the output by 15. capische? so just tweak all your weights a little so your function's output is a little closer to the actual price.

now do this a bazillion times. if you're lucky, your network will now give good estimates for car prices. if it doesn't, you can always add more neurons:

the later ones are sums of the earlier ones, see. what do the ones in the middle mean? idk, but now your network can express more functions. the training algorithm still works the same way, divide-and-conquer style.

but that's childs play. thats barely any neurons at all. that's the sort of poo poo you'd see in a neural network paper from the 1980s. weve got gpus now. you can throw as many neurons as you want around, in giant 3d blocks of numbers, each made up of the sums of other giant 3d blocks of numbers. just go hogwild:

these are all state-of-the-art networks. if you take a long time and learn a bazillion tricks to train them correctly, you can get these to give pretty good accuracies for benchmark problems. they were all discovered by, basically, loving around. there's barely any theoretical basis for any of this. machine learning!

this post got fuckoff long so im not gonna even post about interpretability. just think about picking some numbers from the middle of those giant networks and trying to decide if they're racist. now imagine your career hinging on getting good results from that. welcome to my grad program

hey now posting convnets is cheating

# ? May 31, 2019 19:07

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

also i feel bad for recognizing some of those from illustrations alone

# ? May 31, 2019 19:14

Fiedler: Jun 29, 2002; I, for one, welcome our new mouse overlords.

lancemantis posted:

also i feel bad for recognizing some of those from illustrations alone

seek help

# ? May 31, 2019 21:16

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

it's my world now fortunately/unfortunately

# ? May 31, 2019 21:17

rotor: Jun 11, 2001; classic case of pineapple derangement syndrome

For real though anyone doing research in ML or machine vision should stop. The downsides of these technologies significantly outweigh the benefits.

# ? Jun 1, 2019 01:06

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

tbh that can be said of pretty much any technology

# ? Jun 1, 2019 04:52

DONT THREAD ON ME: Oct 1, 2002; by Nyc_Tattoo; Floss Finder

it's been all downhill since the stick with a rock on the end imo

# ? Jun 1, 2019 05:02

The Management: Jan 2, 2010; sup, bitch?

Cybernetic Vermin posted:

google along with a lot of other actors have realized that a lot of successful ml simply involves ridiculous amounts of semi-skilled man-hours tweaking parameters and topologies randomly until the thing seems to do the thing, so it is worth it to try to capture more hands early on.

there's some glimmers of theoretical work happening on what works and what doesn't, notably lin-tegmark(-rolnick) https://arxiv.org/abs/1608.08225v4 , which does the very basic needful demonstrating that e.g. encoding multiplication requires exponentially wide hidden layers, which should reasonably be interpreted as "cannot be learned with current methods".

i never find the time, but i am personally quite sure that it is a pretty doable task to draw from minsky's perceptrons book (which famously killed off neural networks last time they were popular) the needed machinery, combined with reasonable assumptions on what is the limits of learning (e.g. finite precision) to demonstrate that some practical-sounding problems cannot be taught to deep neural networks without encoding as much information as a full table realization of the function requires. which should again be interpreted as "cannot be learned with current methods". i don't think such a paper would change the enthusiasm that much, but it is one of those directions that ought to be investigated sooner or later.

I�m not an academic type but if I�m reading this correctly you�re saying that ML is horseshit, and I can confirm this from practical experience.

# ? Jun 1, 2019 06:21

rotor: Jun 11, 2001; classic case of pineapple derangement syndrome

lancemantis posted:

tbh that can be said of pretty much any technology

you can say anything you can think of but for some things its true and for some things its not

# ? Jun 1, 2019 06:56

eschaton: Mar 7, 2007; Don't you just hate when you wind up in a store with people who are in a socioeconomic class that is pretty obviously about two levels lower than your own?

neural networks (ML) aren�t entirely horseshit, it�s just that what they actually let you do is far less than what a bunch of idiots eager for your cash let you do

animals do use neural nets for �everything,� sure, but real neurons are a bit more than simple integrators, so we�re not really very close to actually using neural nets the way animals do and getting the benefits of a faster substrate

instead we�re able to do basically one simple task in a way we don�t really understand, and we push even that far beyond reasonable use

if I were asked to make an �object recognizer,� I wouldn�t train one huge network on a million images of the object I want to recognize and allow steganography to break everything, I�d train a large number of smaller networks on different characteristics to recognize, and use additional separate networks to determine confidence according to recognitions, etc. finally arriving at the one confidence value

# ? Jun 1, 2019 07:15

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

eschaton posted:

if I were asked to make an �object recognizer,� I wouldn�t train one huge network on a million images of the object I want to recognize and allow steganography to break everything, I�d train a large number of smaller networks on different characteristics to recognize, and use additional separate networks to determine confidence according to recognitions, etc. finally arriving at the one confidence value

This is actually basically Fast R-CNN - except object classification was done by SVMs instead of NNs

really all the convnet did for that is feature encoding

it�s called fast rcnn but it�s pretty slow compared to the approaches that supplanted it

# ? Jun 1, 2019 13:53

Schadenboner: Aug 15, 2011; by Shine

eschaton posted:

if I were asked to make an �object recognizer,� I wouldn�t train one huge network on a million images of the object I want to recognize and allow steganography to break everything, I�d train a large number of smaller networks on different characteristics to recognize, and use additional separate networks to determine confidence according to recognitions, etc. finally arriving at the one confidence value

�What if the three blind men and the elephant only it were a good thing?�

# ? Jun 1, 2019 13:56

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

oh and also fast rcnn was doing that all given that you were already giving it proposals for where objects in the image were

# ? Jun 1, 2019 13:57

I HAVE GOUT: Nov 23, 2017

eschaton posted:

I�d train a large number of smaller networks on different characteristics to recognize, and use additional separate networks to determine confidence according to recognitions, etc. finally arriving at the one confidence value

Thats what resnet does, except it picks those attributes automatically and you can later verify what each output from each layer means.
And what u can also do is load resnet and train the last layers with new images to detect the things u want (like different breeds of animals). Then u dont need millions of images anymore. You can get by with like 30 and it will still be super accurate for ur new specific case.

# ? Jun 1, 2019 14:20

I HAVE GOUT: Nov 23, 2017

One of the most valuable parts of ml is u can take a existing model and then train it with ur data to make it do what u want. So with each new tech, u wait for the super nerds to do all the work for u, then u use their model to do ur stuff easy peasy.

# ? Jun 1, 2019 14:22

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

well resnet is more of a particular convnet architecture, like the densenet illustration at the end of that earlier post, than a particular machine vision objective

# ? Jun 1, 2019 14:53

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

like pre-convnet image classification and object detection (which are overlapping tasks really) was all about using something like sift vectors, which was pretty much taking an image and encoding it into this vector where each dimension represented some particular human chosen function output and you hoped that the vectors were distinct enough that images of different things would encode distinctly but that also the encoding was general/robust enough that changes in pose or scale/distance wouldn�t prevent things from matching up

so an objective like object detection was a bunch of sliding various windows of subsections of the image and searching your database of sift vectors to see if anything matched

and of course there was research into how to partition and search through that database of sift vectors faster etc

# ? Jun 1, 2019 15:05

rchon: Feb 19, 2015

don't ask me about deploying mask r-cnn models to production systems with no quantifiable validation metrics or model versioning

# ? Jun 1, 2019 15:26

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

rchon posted:

don't ask me about deploying mask r-cnn models to production systems with no quantifiable validation metrics or model versioning

bonus if it�s just the Facebook repo

# ? Jun 1, 2019 15:42

rchon: Feb 19, 2015

lancemantis posted:

bonus if it�s just the Facebook repo

matterport, actually

(detectron apparently doesn't run in a vm with cpu and 2gb of ram. still want to try maskrcnn-benchmark though because it's in pytorch.)

# ? Jun 1, 2019 16:08

animist: Aug 28, 2018

lancemantis posted:

hey now posting convnets is cheating

ill have you know that convnets are an extremely exciting area of research with wide-reaching implications in the fields of speech synthesis and niche pornography

# ? Jun 1, 2019 20:43

suffix: Jul 27, 2013; Wheeee!

is attention just more matrices multiplied together and trained the standard way

should i be blindly copying code i don't understand from random transformer repositories instead of from rnn tutorials

# ? Jun 1, 2019 21:02

Arcteryx Anarchist: Sep 15, 2007; Fun Shoe

suffix posted:

should i be blindly copying code i don't understand from random repositories

I mean you�re doing machine learning aren�t you?

# ? Jun 1, 2019 21:09

Adbot: ADBOT LOVES YOU

# ? Apr 27, 2024 22:53

animist: Aug 28, 2018

interpretabilityposting:

how tf do you understand what's going on in a deep convnet? how do you know what the network will do?

some people have tried using like, formal proofs and geometry to describe network behavior. sometimes they can get this to work for more than, like, 10 neurons. but it's mostly useless for larger networks. too many variables, too high-dimensional.

more interesting approaches for understanding neural networks are empirical, imo: do experiments, treat neural networks like lab animals and try to do biology to them.

the best recent paper is prolly https://distill.pub/2018/building-blocks/ which is fun to play around with in the browser if you've got 10 minutes. their code hallucinates images that activate different slices of a neural network; you can tell it to activate some neuron or group of neurons and it'll make a picture that activates them. sometimes the pictures even make sense.

it's hard to tell what extent that's just pareidolia and selection bias, though. if you reimplement that paper but don't tune your hallucination code right, you'll just get white noise instead of pictures of dogs for your visualizations; and there's no reason to assume that white noise is a worse description of the behavior of the network than the dog pictures. it's like... trying to understand how a squid thinks by FMRIing its brain. but then only using the FMRIs that you think look cool. idk.

the other super cool empirical paper to come out recently is the lottery ticket hypothesis, which has pretty far-reaching implications if it turns out to be true. it's suggesting that the big convnets we're training are actually tools that let us do searches over combinatorially-huge numbers of smaller meaningful subnetworks. basically if you throw a giant bucket of spaghetti at the wall a little bit of the spaghetti will stick. if it pans out it could explains why, like, deep learning works at all. on the other hand it might be bupkis

# ? Jun 1, 2019 21:13

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > YOSPOS > machine learning: this place is not a place of honor

«‹›24 »