|
google along with a lot of other actors have realized that a lot of successful ml simply involves ridiculous amounts of semi-skilled man-hours tweaking parameters and topologies randomly until the thing seems to do the thing, so it is worth it to try to capture more hands early on. there's some glimmers of theoretical work happening on what works and what doesn't, notably lin-tegmark(-rolnick) https://arxiv.org/abs/1608.08225v4 , which does the very basic needful demonstrating that e.g. encoding multiplication requires exponentially wide hidden layers, which should reasonably be interpreted as "cannot be learned with current methods". i never find the time, but i am personally quite sure that it is a pretty doable task to draw from minsky's perceptrons book (which famously killed off neural networks last time they were popular) the needed machinery, combined with reasonable assumptions on what is the limits of learning (e.g. finite precision) to demonstrate that some practical-sounding problems cannot be taught to deep neural networks without encoding as much information as a full table realization of the function requires. which should again be interpreted as "cannot be learned with current methods". i don't think such a paper would change the enthusiasm that much, but it is one of those directions that ought to be investigated sooner or later.
|
#
¿
May 31, 2019 10:29
|
|
|
|
| # ¿ May 11, 2024 19:47 |
|
|
ZZZorcerer posted:There’s lots of books of how to use ML to tackle loads of problems but is there a good one that talks about which kind of problems are adequate and which are not ? not sure about the word "adequate" here, but i think the answer is still: no, because we don't really know. there may be some literature on rules of thumb, but overall the most sophisticated current practice is throwing stuff at the wall and seeing what sticks.
|
|
#
¿
Jun 6, 2019 14:48
|
|
|
can't expect organized views and action from unknowing individuals though.
|
|
#
¿
Jun 6, 2019 17:57
|
|
|
Krankenstyle posted:yeah sure, a statistical model of the solar system is a pretty good predictor for eclipses. thats kindof what we've had since millennia (more or less successfully) it is incredibly dumb since it is literally a curve fitting task for which machine learning is both overkill and pretty bad.
|
|
#
¿
Sep 28, 2019 11:05
|
|
|
multiplying two inputs with *each other* without using some specialized blocks is the hard bit. multiplying inputs *by a constant* and adding them up with some clamping function applied is the basic building block.
|
|
#
¿
Oct 1, 2019 21:52
|
|
|
i.e. a state of the art for 1996 reimplementation of pagerank
|
|
#
¿
Oct 10, 2019 10:15
|
|
|
theadder posted:0.08 op we live in the unicode future these days op
|
|
#
¿
Nov 11, 2019 20:33
|
|
|
Fanged Lawn Wormy posted:*points at seagull* ai. pigeons are, surprisingly, entirely naturally occurring.
|
|
#
¿
Nov 28, 2019 13:10
|
|
|
cobol programmers get paid too well doing cobol to bother doing python for free. i am also not quite sure why you want it.
|
|
#
¿
Dec 4, 2019 10:12
|
|
|
uh, and, numpy/scipy are a bunch of fortran with some python coating, i am not sure where you were expecting fortran to go?
|
|
#
¿
Dec 4, 2019 14:58
|
|
|
redleader posted:is fortran still faster than c for some types of numeric calculations for deeply technical yet tedious reasons? this is a weird comparison since fortran is in many ways a much more alive language than c, having been modernized to add e.g. proper modules, operator overloading (pretty big deal for numerics), forall loops, and if you dip a far in as fortran'03, object orientation. also, yeah, cuda is another super-important leg to all this stuff, but tons of kernels are extremely hard to parallelize and require a lot of very careful work to split even across a small number of well-interconnected general cpus. e.g. the vast zoo of bespoke matrix decompositions for extremely specific subcases.
|
|
#
¿
Dec 5, 2019 10:55
|
|
|
it is easy to get in a mindset where one imagines that the code would be easy to work with if it was just structured just so (or written in ones preferred language), but some code just expresses sufficiently complicated concepts that it will always be difficult. godspeed etc. v v
|
|
#
¿
Dec 5, 2019 22:04
|
|
|
that's surely the horniest stack programming in particular has ever been
|
|
#
¿
Dec 13, 2019 23:01
|
|
|
ninepints posted:I know this one. the problem with linear activation functions is that they collapse the representative power of the whole network down to the representative power of a single layer. the final outputs become a linear combination of linear combinations...of linear combinations of the inputs, which can be simplified to a single linear combination of the inputs. one can for intuition frame this as not having "decisions"; if a layer wants to differentiate between constant values x and y, while being reasonably robust to small errors, it can have parameters which place x and y into a part of the activation function where such fuzziness is truncated out; e.g. scale x so values in its neighborhood lands in the flat 0 of ReLU, or x and y on the bottom and top of a sigmoid for a cleaned up {x -> 0, y -> 1} mapping. that is, the layer can take a value z=0.9*x+0.1*y, and simulate deciding that z is actually x by outputting z'=0.99999*x+0.00001*y (in the sigmoid case). with only linear activation functions you cannot achieve the same thing, since the output will just be some linear scaling of z, retaining both its x and y "parts". it just collapsing to one layer is the more powerful argument, but it might be useful to think about how non-linearity lets the network tweak what information is retained.
|
|
#
¿
Dec 15, 2019 11:06
|
|
|
redleader posted:it's a pity that nlp and cv have fallen to the ml gremlins a lot of ml research is dumb and bad (almost as dumb and bad as "tech visionaries" ideas of what ml can do), amounting to a phd student twiddling various parameters until a model seems to learn something, with no deeper analysis or insight. however, it is not right to view ml as having been bad for computer vision and natural language processing, not least ml is pretty simple, so it is not a hard tool to apply. i've historically been in nlp myself, and while its been a weird decade ever since statistical n-gram methods broke the backs of the chomsky'ites things are looking better and better now. it has gotten clear which ml bits are indispensable and possible to analyse (e.g. word embeddings) and the field is just getting a lot more high-level thanks to ml bits handling a lot of the nitty-gritty (e.g. making it a lot easier to make stuff robust against grammatical mistakes). it has chilled interest in formal grammars and automata solutions a lot, but there was so much theoretical navel-gazing there that i don't think that's bad (and that was my primary research area). i am also currently pretty excited about a new research program from the research group i primarily affiliate with, where they are embracing the bias-soaking nature of ml to study gender bias in written text. that is, as a very first step, looking at the word embeddings for ostensibly non-gendered words in a given publication and see how orthogonal those feature vectors are from gendered vectors. a ton of tricky issues there (e.g. it matters a lot how the dimensionality reduction, i.e. the ml, works), but a bright new phd student (affiliated both with us at cs and the dept. of gender studies) is working on it, and i think it'll be extremely interesting research no matter the exact outcome. well, that's a long post. tl,dr: ml *in* research often good.
|
|
#
¿
Jan 3, 2020 12:57
|
|
|
fart simpson posted:is nlp still basically only done in english the available good *structured* data is mostly english (e.g. the penn treebank and the amr semantics bank), and of course the papers are written in english. so to some extent it always remains the case. when doing formal grammar/automata work one habitually invokes random languages for having tricky grammatical structures. like swiss german for having cross-serial dependencies (they exist in english only in contrived cases, like "the coffee, cake, and biscuit cost $2, $3 and $4, respectively", but are a normal grammatical feature in swiss german). that is mostly a matter of motivating the navel-gazing however, keeping alive research in "mildly context-sensitive" grammatical formalisms for ages without them ever really demonstrating any practical usefulness. dumb statistical and ml models have luckily improved the situation a fair bit, since it doesn't matter nearly as much how much painstakingly cleaned and hand-annotated data you have. initially it was all n-gram work entirely devoid of grammar, but most research now mix things a bit, with a bit of grammatical structure both induced by statistical models and fed to other statistical models, in a way that generalizes pretty easily to most languages. it also now seems obvious that this is the only way to do it, the idea that humans have some inherent grammar which is not hopelessly intermingled with general intelligence seeming hopelessly naive. well, to me. this is plenty controversial.
|
|
#
¿
Jan 3, 2020 18:38
|
|
|
i don't know chinese, but i presume that word segmentation is a mostly artificial idea in it. that is, it does not exist explicitly on the page and is not central in the mind of the people communicating in chinese (who will compound concepts as they see useful), making the problem pretty ill-defined. a lot of nlp problems suffer badly from an (elitist) normative view of language, e.g. grammar checkers defined entirely by a certain kind of person going "well that's not *proper* english" over and over. statistical methods are unlikely to do a better job segmenting text, rather they are used to scan through the text, extracting the relevant concepts and components (an abstract paraphrase in a sense), hopefully leaping over a bunch of hurdles like "incorrect" segmenting, compounding, typos, unexpected typography, etc. (in effect by looking at a larger context with a bit more "understanding"). ye olde nlp systems would just get this wrong in step 1 of a cascade of transformations, and then never recover. in many ways this is precisely the kind of thing the thread (rightly) hates, in that it takes something that used to be about a strict syntactic understanding of something important, and then throws ml at it muddling parts into a pile of incomprehensible statistics. the difference imho is that here the strict understandable solution never existed, at least not in any human brain, and that the ml bits are well-defined in both extent and purpose.
|
|
#
¿
Jan 4, 2020 12:09
|
|
|
should note that i am neither a linguist, a chinese speaker, nor all that successful a nlp researcher, so everything with a grain of salt. it is pretty interesting though. from a quick google 是 is common enough in compounds, but extremely common in this copular verb form. despite cynicism about the problem in general i'd have expected one of these dictionary-driven things to manage this much, maybe still some flag/parameter set particularly poorly?
|
|
#
¿
Jan 4, 2020 13:24
|
|
|
that's a p. good tweet.
|
|
#
¿
Jan 11, 2020 17:18
|
|
|
it is somewhat unsurprising that the copy-paste-from-stackexchange mode of programming could be fully automated really.
|
|
#
¿
Jul 5, 2020 12:27
|
|
|
at least it was from 2008. not that it was more acceptable then, but it was easier to pretend that the data didn't matter as much (as ml was simply pretty bad), so as long as a proper apology is issued and the data removed we can get back to trying to get present-day projects to not be hilariously racist.
|
|
#
¿
Jul 6, 2020 21:02
|
|
|
hooooly gently caress, even for the abysmal standards or acceptability in machine learning startups that is a horrible idea
|
|
#
¿
Jul 28, 2020 20:01
|
|
|
ultrafilter posted:Might need a bigger training set, or a different algorithm, or a fundamental rethinking. this is one where no amount of technology, thinking, and ethical awareness fixes the fundamental problem: hearing someones name and having a guess at their gender is socially unacceptable, and recognizably so even without any kind of modern awareness of the issues.
|
|
#
¿
Jul 28, 2020 20:13
|
|
|
it is such an open-ended question, but i have in the past commented on how ml has opened up nlp research a lot.
|
|
#
¿
Aug 9, 2020 08:18
|
|
|
it hurt me to look at that, so i didn't follow up, but that is some horrible joke right?
|
|
#
¿
Aug 13, 2020 21:46
|
|
|
fack you posted:so like you know in star trek all the doors on the enterprise slide open, but it was just some person pulling on wires or something at the right time in the script, not a real automatic door, so it appears to work not on motion, but like a person's intent because sometimes there'd be a whole conversation right next to a door and it'll magically open when riker is done chatting up troi or whatever ugh. could you feed in all the ridiculous footage into all this machine learning poo poo and end up with software that'd actually open a door based on intent just like how it appears on the show I don't know i think this is one of those cases which some ml model would do pretty well yeah. not like fully star trek magical, but differentiating between someone going in, someone standing still, and someone walking past should be doable in a fairly accurate way. and is actually not a bad idea, the amount of energy lost to automatic doors opening unnecessarily in winters and such probably adds up considering the relative simplicity of implementation.
|
|
#
¿
Oct 30, 2020 17:31
|
|
|
|
| # ¿ May 11, 2024 19:47 |
|
|
how much is jeffrey paying them for that dreary review work?
|
|
#
¿
Nov 16, 2020 17:01
|
|