|
Here's a more accessible and high level description of some issues by Karpathy: https://karpathy.medium.com/yes-you-should-understand-backprop-e2f06eab496b
|
#
¿
Jun 7, 2023 17:33
|
|
|
|
| # ¿ May 17, 2024 18:36 |
|
|
The Llama models released by Meta are tuneable and people have been using them to make things, such as Open Assistant. The main problem is whether you have enough data to tune it on (and also that it's still non trivial to do in terms of implementation and computational resources). In particular you can't really do RLHF as that requires tons of human resources.
|
|
#
¿
Sep 10, 2023 02:20
|
|
|
Nektu posted:Why is recognizing diseases early a use case for generative AI? On first glance it looks like something for a pattern matching algorithm (which recognizes symptoms). Generative AI is just a fancy new word for pattern matching algorithm, it seems. People seem to use it for any huge scale machine learning models that use the attention mechanism, thus you can also say that pretty much any machine learning task is a use case for generative AI. Alternatively, it may refer to prompt-based interaction with any large-scale model, then I guess it's just an alternative interface to the patterns that are detected.
|
|
#
¿
Feb 26, 2024 19:31
|
|
|
I do NLP research including LLM evaluation but yeah, that doesn't mean I know everything about it and of course I don't have the resources to make my own.
|
|
#
¿
Apr 17, 2024 17:46
|
|
|
I would say that using a LLM is overkill for this use case. As for the context size, it really depends what kind of units of information you are interested in. If it's mainly words, you can use pretrained word embeddings and context doesn't matter as the word contextual meaning will come mainly from the pretraining data and not much from your context. Otherwise you can make document embeddings - either with the few word snippets, or by merging the snippets into full sentences or chapters of videos. It's really up to you at what level you want to have a searchable semantic representation. If more contextual info is needed you could also include for example the video title into each document embedding or whatever.
|
|
#
¿
Apr 17, 2024 20:26
|
|