|
I have a project where I need to train a CNN into detecting different classifications of lets say images of cars. Ford, Ferrari, Mitsubishi, and Toyota; while I can get datasets with lots of cars and maybe a dataset that is only ford I cannot seem to find a convenient dataset that has all four neatly divided into categories. Is there a trick I can do on a dataset that I know has all four mixed together to neatly divide them (like I dunno, some sort of unsupervized clustering to try to pre-divide/sort the data to cut down on the work I have to do manually) or do I basically have to divide it all manually? Or do we have a more dedicated data science thread for this question?
|
#
?
Jun 7, 2022 23:39
|
|
|
|
| # ? Apr 30, 2024 03:10 |
|
|
cardinality of data? if its like 20k thats a nice afternoon for a few peeps w/ real eyeballs
|
|
#
?
Jun 7, 2022 23:41
|
|
|
The requirements are for a minimum of 1200 training images and 400 testing images balanced between the four classes. Is it normal to just use the Mark 1 Eyeball for this? The programmer in me definitely feels like if there was a way to conveniently automate this I should, but that also sounds suspiciously exactly like the point of the project 
|
|
#
?
Jun 7, 2022 23:46
|
|
|
bob dobbs is dead posted:if its like 20k thats a nice afternoon for a few peeps w/ real eyeballs No joke, measure how long it takes you to label say, 50 images. Then figure how many people you could assign to labeling the data and how many hours it would take for each of them. If you don't have anyone, like if this is a personal project (with non-sensitive data), then maybe outsource to mechanical turk?
|
|
#
?
Jun 7, 2022 23:49
|
|
|
Raenir Salazar posted:The requirements are for a minimum of 1200 training images and 400 testing images balanced between the four classes. Is it normal to just use the Mark 1 Eyeball for this? The programmer in me definitely feels like if there was a way to conveniently automate this I should, but that also sounds suspiciously exactly like the point of the project eyeball will be cheaper and faster hard to get a decent programmer for less than $100/hr. can get a manual peep to classify for min wage 1200, you dont even need to hire peeps. take two hours and whack it out
|
|
#
?
Jun 7, 2022 23:50
|
|
|
Just to be clear this is a university/school project, not something I'd have a budget for.  The class is an introduction to AI sort of course as part of completing my BCompSci.
|
|
#
?
Jun 7, 2022 23:52
|
|
|
And this is why to my understanding people working in AI like to use and rely on pre-labeled public datasets. If there were an algorithm to cluster the images why would you need to do classification in the first place?
|
|
#
?
Jun 7, 2022 23:54
|
|
|
if they're making new algorithms, sure if you're classifying per se in production there's a fuckton of images and you're getting a sample to train with
|
|
#
?
Jun 7, 2022 23:58
|
|
|
Yeah that's why I was checking to make sure I wasn't overcomplicating things or missing anything that I *should* be doing.
|
|
#
?
Jun 8, 2022 00:14
|
|
|
Raenir Salazar posted:I have a project where I need to train a CNN into detecting different classifications of lets say images of cars. Ford, Ferrari, Mitsubishi, and Toyota; while I can get datasets with lots of cars and maybe a dataset that is only ford I cannot seem to find a convenient dataset that has all four neatly divided into categories. If the dataset is not labeled in some way and you can't programmatically divide them then any unsupervised algorithm won't really help you; there's enough natural variance in an RGB image that you couldn't really do that without a trained classifier in the first place. Theoretically if a car manufacturer has a ubiquitous enough design language you wouldn't need that many samples but of course when you add wear and tear, different generations, etc into the mix that muddies the waters. But anyway this sentence is mostly rambling. Since this is a uni project a predone dataset is probably your best bet, this one seems to be from Stanford, has numerous samples, and has make/model in the features from what I can tell. I couldn't tell you what the distribution of makes is, but I hope a Stanford dataset would have tried to somewhat balance that. https://www.kaggle.com/datasets/jessicali9530/stanford-cars-dataset http://ai.stanford.edu/~jkrause/cars/car_dataset.html Kaggle is always a good resource for things like this in my experience.
|
|
#
?
Jun 8, 2022 00:14
|
|
|
I went and by hand separated my images into their folders, took me like 8 hours while watching Let's Plays in the background.  My next challenge was to how to use these datasets with the example code of the various Labs we did in class so far. I managed to stumble on Neural Network Console by Sony, which seems to not only let me produce a image classifier dataset, but works with subfolders and assigns labels based on the subfolders; yay. Took me a couple of tries as while it will output a training set and a testing set, the code example I had would auto divide it randomly for me so I just outputted the whole dataset as 1 csv file which I loaded using a custom loader using some code I found online in a youtube tutorial. This seems to work but then turns out my python environment is hosed because I get the error "Initializing libiomp5md.dll, but found mk2iomp5md.dll already initialized" with the problem being I must have some conflicted python library installed. I pushed my code and now I wait as I try to reinstall spyder in a new conda python environment and reinstall my modules and hope that works. Anyways here's my code so far: (my loader) code:code:Raenir Salazar fucked around with this message at 23:35 on Jun 8, 2022 |
|
#
?
Jun 8, 2022 23:32
|
|
|
Raenir Salazar posted:I have a project where I need to train a CNN into detecting different classifications of lets say images of cars. Ford, Ferrari, Mitsubishi, and Toyota; while I can get datasets with lots of cars and maybe a dataset that is only ford I cannot seem to find a convenient dataset that has all four neatly divided into categories. That trick is called Mechanical Turk, or a specialised data labelling service. I do NLU, not CV, but this sounds like a crap idea for unsupervised or semi-supervised clustering, if that’s what you’re thinking. We have a dead-ish DS thread, and should probably get around merging those together. Maybe merge in the scientific computing thread as well, make a boffin central. And revive LaTeX thread  Fake edit: 1200 images? Just boil yourself a coffee pot. Edit: Should’ve probably finished reading before replying.
cinci zoo sniper fucked around with this message at 11:36 on Jun 9, 2022 |
|
#
?
Jun 9, 2022 07:52
|
|
|
cinci zoo sniper posted:That trick is called Mechanical Turk, or a specialised data labelling service. I do NLU, That's Natural Language Understanding, right? Do you know of some good intro books into that field? I have an interest in linguistics and NLP + NLU.
|
|
#
?
Jun 9, 2022 08:14
|
|
|
quarantinethepast posted:That's Natural Language Understanding, right? Do you know of some good intro books into that field? I have an interest in linguistics and NLP + NLU. That’s it indeed. The area you’ve outlined is really broad, and I don’t know what’s your background like, so maybe take a look at https://web.stanford.edu/~jurafsky/slp3/ ? I’m not sure I know anything up to date that would be more beginner-friendly than this, as far as books go.
|
|
#
?
Jun 9, 2022 09:38
|
|
|
dan jurafsky the years i knew him was kinda 3/4 of the way to being a crying shambling wreck because the neural net peeps were running roughshod over his life's work. he's fundamentally not a neural nets guy and neither is martin iirc. of course they are forced to shove it in any nlu book now
|
|
#
?
Jun 9, 2022 09:47
|
|
|
bob dobbs is dead posted:dan jurafsky the years i knew him was kinda 3/4 of the way to being a crying shambling wreck because the neural net peeps were running roughshod over his life's work. he's fundamentally not a neural nets guy and neither is martin iirc. of course they are forced to shove it in any nlu book now Yeah it’s not a DL book per se, but jumping directly into DL without understanding what you’re trying to do or why is like trying to do a backflip with a motorcycle when you can’t do it water, imo.
|
|
#
?
Jun 9, 2022 11:32
|
|
|
So after having to completely reinstall python/anaconda I managed to get cuda working for the PyTorch CNN; but the "Skorch" version just refuses to work on my GPU. I have device='cuda' in the NeuralNetClassifier constructor and I've confirmed that cuda/my gpu is available and working correctly, at least with PyTorch, but Skorch I have no indication that it's working; GPU usage is 2-5% when working with PyTorch but 0% when it gets to the Skorch version of my network.
|
|
#
?
Jun 10, 2022 04:20
|
|
|
cinci zoo sniper posted:Yeah it’s not a DL book per se, but jumping directly into DL without understanding what you’re trying to do or why is like trying to do a backflip with a motorcycle when you can’t do it water, imo. I've been taking Andrew Ng's deep learning Coursera series so I've got some DL background, albeit the courses could use more practical focus on projects beyond "add these 2 lines which we have spelled out for you to an almost complete function".
|
|
#
?
Jun 13, 2022 05:19
|
|
|
quarantinethepast posted:I've been taking Andrew Ng's deep learning Coursera series so I've got some DL background, albeit the courses could use more practical focus on projects beyond "add these 2 lines which we have spelled out for you to an almost complete function". I had reviewed that course recently and for a practitioner imo it’s only good to tie up disparate knowledge about network-based ML, e.g., if you’re moving from credit risk into computer vision. I really hated the assignments, most of which just had you copy and paste code provided above, full verbatim, and I struggle to imagine how anyone could learn from that. Apparently, though, that was too much already, as the course is now seeming being redone to be simpler. Also, the audio quality was really poo poo. I loved random loud pitching noises in half of the videos, because no one on the editorial side had functional hearing. Not sure what’s a good alternative for it though, for general NN intro. One of the things on my docket is to figure out a replacement curriculum for future hires that may need it.
|
|
#
?
Jun 13, 2022 09:13
|
|
|
cinci zoo sniper posted:Not sure what’s a good alternative for it though, for general NN intro. One of the things on my docket is to figure out a replacement curriculum for future hires that may need it. Allow me to (possibly) help you!  Personally, I used Kirill Eremenko et al's "Deep Learning A-Z™: Hands-On Artificial Neural Networks" to learn the basics. I had no previous understanding of ML at all, and only very basic knowledge of programming, perhaps 80-160 hours total of programming under my belt. The course was pretty good in my opinion, and I learnt the basics. My thesis network probably still uses bits of code from those lectures I did years ago. I don't know how "current" the course is, and obviously it doesn't teach deep insight on deep learning, but a very good hands-on lecture series. Or at least I found it good, I'm not saying it's the best one out there. Oh and just in case anyone isn't familiar, Udemy's "sales" are all bullshit, courses always cost 10-30 bucks no matter what the list prices say. Those very "urgent" 95 % off deals of several hundred dollar/euro courses are essentially fake. There's always a sale going on that ends in a few hours or days. Link: https://www.udemy.com/course/deeplearning/ EDIT: I still use the course every now and then, for example I am looking into RNNs at the moment, and watching the LSTM lectures every now and then. It's going to be a good stepping stone to time-series analysis or whatever the hell I want to do later.
|
|
#
?
Jun 13, 2022 09:42
|
|
|
cs229 for reals has pretty moderate problem sets that they still couldn't deal with when they started coursera with it https://cs229.stanford.edu/summer2019/ps1.pdf fast.ai is decent enough for neural nets stuff. although you won't be doing nontrivial original stuff from it, you should not condign anyone to that fate in a corp anyways bob dobbs is dead fucked around with this message at 09:53 on Jun 13, 2022 |
|
#
?
Jun 13, 2022 09:50
|
|
|
I want to learn machine learning I did skim through this tutorial for PyTorch real quick and got to this point: quote:When training neural networks, the most frequently used algorithm is back propagation. In this algorithm, parameters (model weights) are adjusted according to the gradient of the loss function with respect to the given parameter. The loss function calulates the difference between the expected output and the actual output that a neural network produces. The goal is to get the result of the loss function as close to zero as possible. The algorithm traverse backwards through the network network to adjust the weights and bias to retrain the model. That's why it's called back propagation. This back and forward process of retraining the model over time to reduce the loss to 0 is called the gradient descent. I'm just not really sure how this is any different from newton's method or like basically any numerical method that minimizes an objective/loss function. e: Maybe I need something more technical instead of a quickstart, as my background is numerical methods. Boris Galerkin fucked around with this message at 18:23 on Jun 15, 2022 |
|
#
?
Jun 15, 2022 18:20
|
|
|
Boris Galerkin posted:I want to learn machine learning I would say that you could try going into https://developers.google.com/machine-learning/crash-course raw and seeing how it goes. Depending on particulars of your background, you could give plenty of ML practitioners run for their money. On the 3 libraries, difference is in focus. TF/PyTorch focus on neural network-based ML, while SKLearn focuses on classical ML methods. You could (kind of) think of the difference as differentiable programming vs probabilistic programming. Where to go after the crash course depends a bit on what your plans are. Do you want to work for a particular company or company class? They may have a niche to specialise for where learning classical ML wouldn’t make sense, or the other way around. It could also be the case that your specific kind of numerical methods are actually repackaged as (more expensive) ML somewhere else, if you just want to hop into the job title. If you’re comfortable with academic literature, in my explicitly subjective opinion you could easily do worse than read https://hastie.su.domains/ElemStatLearn/ regardless of what’s down the line for your ML career, though. Some other things you’ll need to take care of likely sooner than later: - SQL - Data engineering fundamentals - Not grimacing when colleagues ask about sick AI features
|
|
#
?
Jun 15, 2022 18:45
|
|
|
Cool, thanks I’ll take a look at the textbook and try to do that Google course in earnest. I mostly want to learn this stuff because the very first time I heard the term “machine learning” I thought “how cute the programmers are learning about linear regression”. And well here we are in 2022 and everything uses ML. Jokes aside just trying to fill in some knowledge gaps.
|
|
#
?
Jun 15, 2022 18:59
|
|
|
Boris Galerkin posted:I'm just not really sure how this is any different from newton's method or like basically any numerical method that minimizes an objective/loss function. Newton's method and other second-order techniques aren't really appropriate for NNs for a couple reasons, not least of which being that the Hessian is so enormous: even if you don't store it you still have to compute with it. Backpropagation is just a way of quickly computing the gradient of the objective function that works with the particular structure of NNs.
|
|
#
?
Jun 15, 2022 19:51
|
|
|
Boris Galerkin posted:I'm just not really sure how this is any different from newton's method or like basically any numerical method that minimizes an objective/loss function. The goal in traditional optimization is to fit the points you have. The goal here is to fit the points you don't have. There are a lot of methods in common but the problems are fundamentally different.
|
|
#
?
Jun 15, 2022 20:12
|
|
|
you can get pearlmutter's trick done for pseudo second order method and lots of first order methods with vague handwaving towards curvatures but not much else
|
|
#
?
Jun 15, 2022 20:39
|
|
|
So here's my current problem for a university project and my attempt to make it go a little faster using my GPU.code:However using skorch it seems to be much slower and not using the GPU at all: code:
|
|
#
?
Jun 21, 2022 00:05
|
|
|
Alrighty, I managed to determine that if I go to the Windows task manager performance monitor, and switch one of the windows to "CUDA" I can actually confirm that CUDA ranges between 41% to 94% usage; yay. So I guess my GPU IS being used, it's still going to be drat slow for reasons unknown to me then. Perhaps I'm loading/transfering the data in a way that isn't optimal? Here's my network for reference: code:Based on googling the input for that first nn.Linear should be something like (??? * imageWidth * imageHeight) where ??? I think relates to the last output size of the last convolution layer? Is that the MaxPool2d since its a 2x2 matrix (4?)? 4*128*128 adds up to 65536 which is what the error is telling me it wants. "(32x65536 and 32768x1000)" as an example for an error I got when I thought it was 32 * 32 * 32. The values otherwise for the Linear and Conv layers are all basically taken as-is from the class Labs and are essentially arbitrary as far as I'm aware; I have no idea if they are good values, I am reading https://towardsdatascience.com/pytorch-layer-dimensions-what-sizes-should-they-be-and-why-4265a41e01fd but their examples are a little different from the code I was given so I'm not really 100% understanding what's going on. If I'm guess right here; originally the CIFAR10 dataset maybe was 32 by 32 images; there's two pooling layers (MaxPool2d) which I read somewhere does some math and cuts the resolution in half? So 32/2/2 is 8. So 8*8, and then the last Conv layer was 64 as its output, which would correspond to the original 8*8*64? Is that right? So for 128by128, that would imply, 32*32*64? That would add up? Is that right? Ideally I'd like to setup the inputs to be either automatic or easy to calculate based on my dataset image resolution and settings; and set it up so CUDA will do its drat job so I'm not twiddling my thumbs with my tiny dataset of just 2,000 images.
|
|
#
?
Jun 21, 2022 03:01
|
|
|
Raenir Salazar posted:So here's my current problem for a university project and my attempt to make it go a little faster using my GPU. With the really big caveat that I am relatively novice and probably wrong, it sounds like there is a bottleneck on data transfer to gpu, causing idle times. Possible reasons: batch size is small, model is too small to get good gpu advantage.
|
|
#
?
Jun 21, 2022 13:16
|
|
|
I think it might be how the data is loaded, because when I use pytorch it's blazingly fast. As an aside, I can't figure this out but now my new problem is no matter what settings I adjust, like batch size, epochs, test data size etc, testing accuracy never gets above 47%. But the training accuracy easily reaches 100% with 20ish epochs. Is this likely a result of bad data? Is 2,000 images total not enough?
|
|
#
?
Jun 21, 2022 17:38
|
|
|
That seems like overfitting. Not knowing about the data here, I'd start by checking to make sure your data is homogeneous: that you don't have a huge set of similar data in training and then a big set of different ones in test. This can happen if you don't randomly separate the two categories and flat copy the data into test where filenames might be grouping classes or similar samples within a class. Your test data could just be the tail end of your class listings. You could also be feeding very high res data, in which case, you would either need a lot more data, or you could just scale them at the input if that's not already happening. Augmentation could improve your acc, but we're talking single digit % typically. I'm mostly familiar with TF, but I imagine PyTorch would have that built into the front end as well. But this disparity points to some more fundamental issue with the data imo. e: I just saw that you're trying to ID cars by mfg. with CNN. Assuming it's a bunch of random images off the net at all sorts of different angles and views, that's going to be hard to get accurate at 2000 samples. But I think you should be able to see better test acc. than that. Lower input res and making sure they're uniformly distributed should really boost that. former glory fucked around with this message at 15:56 on Jun 23, 2022 |
|
#
?
Jun 23, 2022 15:46
|
|
|
Try a validation set after every epoch to see if you're maybe overfitting there.
|
|
#
?
Jun 24, 2022 06:33
|
|
|
At a quick glance it seems like you have an enormous sequential layer there. It might be the culprit here. I would look into scikit's gridsearchcv and random architecture search to tune network size or dropout values.
|
|
#
?
Jun 24, 2022 09:02
|
|
|
I'm a complete outsider interested in gaining an overview of OCR methods and applications; there's a specific OCR application that I think would be unusually straightforward and doable, and which I also think has already been completely accomplished with high fidelity in a nearly identical setting. I unfortunately can't be specific, but it's along the lines of "right now there are tools out there for reading product names and prices from grocery receipts, and we want a tool that can do the same to bookstore receipts." The real applications are actually much closer together and more stable than that example. There's also a preexisting, fully labeled dataset that I believe is quite large (six digits) that could be used to train and refine such an application. The endpoint would be a site where users would upload an image of the text and the output would be generated for their review/errorchecking immediately, then stored. Everything I see and read makes this seem like it would be (relatively) simple, but at the same time, I know I'm close to clueless about any of this stuff, and I'm really just trying to get a sense of how costly/feasible actually doing it would be. Is there a good resource for really intro-level stuff that would give me a sense of how difficult building this out would look like? Discendo Vox fucked around with this message at 06:18 on Jul 3, 2022 |
|
#
?
Jul 3, 2022 06:08
|
|
|
Talking about real life application targets for this: 1) Would the text there be printed by a machine exclusively? 2) Would it all be in English? 3) Could these parcels of text be considered documents (i.e. clean background, standard font, focus on legibility)? 4) Would they be described well by a low number of templates?
|
|
#
?
Jul 3, 2022 11:41
|
|
|
I had an idea once back when I did QA work for (indirectly) WoTC of scanning MTG cards from the god book we were given for key phrases and use grammar rules to try to output all of our test cases/scripts for testing the cards in order to automate our work of thinking of and writing those test cases. My lead discouraged me from doing it so I never went through with it as I wanted to actually be paid for potentially eliminating my own job
|
|
#
?
Jul 3, 2022 14:27
|
|
|
cinci zoo sniper posted:Talking about real life application targets for this: 2) No, a very low group, probably ~1%, will be in duplicated form with another language and in theory there will be a handful not in English. It should be possible to exclude these from the intended use, and/or from the training set. 3) Yes. 4) I’m not certain what you mean by “template”, but the images would be in a handful, maybe six, standardized forms where text is positioned in proportionately consistent relative positions and sizes. The theoretical training data is not currently labeled to separate these formats. Discendo Vox fucked around with this message at 15:16 on Jul 3, 2022 |
|
#
?
Jul 3, 2022 15:13
|
|
|
Have you tried just pointing Tesseract at your data and seeing how well it does?
|
|
#
?
Jul 3, 2022 15:28
|
|
|
|
| # ? Apr 30, 2024 03:10 |
|
|
Discendo Vox posted:1) Yes “OCR” in current parlance lumps 3 different areas: 1) Optical Character Recognition – read the text [in a traditional document]* 2) Scene Text Detection – identify text areas in a [naturally occurring] scene 3) Scene Text Recognition – read the text in those areas 2 and 3 are difficult problems, challenging state-of-the-art methods. 1 can be challenging for difficult handwritings or small languages, but is basically a solved problem for machine print of major languages (unless you're dealing with 50-year-old photos of 500-year-old parchments or similar). Your second point is unlikely an issue, and your 4th point takes care of text detection basicallly - you probably can distinguish between forms using rote heuristics, with no fuzzy ML needed. Thus, I'm not sure you even need anything more than regular developers with experience of integrating off-the-shelf OCR toolkits like EasyOCR, PaddleOCR, Tesseract, respective CV APIs of major cloud providers, or whatever else you have access to, to digitize the collection in question. Caveat - I'm assuming that the text here is text, and not, e.g., physics formulas. Dealing with math symbols or fancy sub/superscripting is not something I've encountered, but should be researchable enough with the aforementioned keywords. *Traditional document – sanely laid-out text on a high-contrast background, typeset in a generic font with large enough letters (relative to image resolution, for meaningful contrast areas), legible spacing, and no fancy formatting. cinci zoo sniper fucked around with this message at 15:52 on Jul 3, 2022 |
|
#
?
Jul 3, 2022 15:49
|
|