RNNs and LSTMs from the past did this as well (but cannot be trained in parallel as each token has to be compressed sequentially). Transformers ate their cake.
Newer methods are going back to similar concepts but trying to get past previous bottlenecks given what we've learned since then about transformers.
They're actually not giving it away for free. At least not LLAMA v2. Once a product successfully monetizes and gains over a certain number of users, an official license from Meta has to be negotiated otherwise Meta could revoke usage of the model. At least that's roughly what the license for LLAMAv2 implies.
This is how I understand it too, and it's very interesting. In any case, if your company reaches that size, I'm sure you'll have replaced it with something else by that time. I'm confident many fully open-source, high-quality and specialized models will be available in the next years - even thanks to Meta. Whatever their strategy is, the models are on our machines, legally, and this is incredible.
As someone who uses ffmpeg daily (mostly basic functions), I now rely on chatGPT to approximate the command and fine tune from there. Haven't used too many of the advanced features of ffmpeg so glad someone seems to be covering those use cases as most tutorials dont cover them.
I'll add my anecdotal experience, got covid in 2020, got the vaccine a year later and felt butterflies in my chest immediately after. Went away after a few seconds but after a month discomfort started during intense exercise. Another month later and I went to the ER with mild pericarditis. Its been a long ~year of recovery where I had to keep my HR under 100 and take tons of anti-inflammatories. Got lucky that I had it mild and finally back to normal (minus getting out of fitness), lets hope it stays that way.
What I learned (perhaps too late lol):
* Medical Science is largely empirical (requires statistical nous), unfortunately doctors do not understand statistics, take argmaxes and treat things deterministically in the guise of evidence based medicine.
* Said doctors advise Insurance companies so any doctors that do understand statistics will be bounded within the confines of what the insurance company considers evidence based medicine. This creates wasted work and even frustration as the doctor has to jump through hoops to get things done.
* getting a rare disease or disease with uncertainty requires becoming an expert on it to "correctly" navigate the health system (US here). I have a newfound appreciation for not only those who have to deal with insurance but simply navigating the uncertainty.
Getting a rare disease means you have to become the expert of your own disease in other places as well.
For the “average” rare disease, the vast majority of the people in the medical system have either (a) not heard of it at all (b) had one paragraph (at most, one lecture) about it while studying. (c) have likely not met a patient with said disease, or maybe a couple over a decade or two
The doctors you see have no time to study and be up to date about multiple rare diseases.
Whereas you have the time and the motivation (and hopefully the ability) to understand all the updates, consider anecdotes, etc.
I got the vaccine long before getting Covid (about a year before), and have had palpitations ("butterflies") since the vaccine.
Are the two related? No idea. But it illustrates the fallacy of relying on anecdote. For anything.
That said: we know that the vaccine causes myocarditis in younger men, and the rate is at least in 1 in 5000, and quite probably higher than that [1]. A recent study of Thai boys suggested a much higher rate, on the order of 1 in 100 [2]:
just a reminded because some people don’t know, not advocating for an additional original strain booster or any cause aside from that it definitely needs to be factored into incidence reports
I think your link (or rather the reference in your link) refutes your statement. If I'm reading correctly, beyond age, you also have to subgroup by sex and by the particular vaccine type to find a subgroup where myocarditis risk of vaccine exceeds that of the disease.
Three young men I work with got myocarditis from the vaccine. In each case, the emergency room staff didn't believe that it was a vaccine injury. In each case, they called it "anxiety" and sent them home. Only later were they able to find doctors willing to call it vaccine injury, but who knows how much damage was done in the meantime with the condition left untreated.
The point is, this isn't rare, and you are not alone.
Family members are rationalizing why he might have done this after the fact. Inferring a causal effect from bullying to murdering children from family member testimony seems very biased.
Huh. I talked to some experts and they told me NN loss functions are bowl-shaped and have single minima, but those minima take a very long time to navigate to in high dimensional spaces.
For higher feature counts the real concern is saddle points rather than minima, where the gradient is so small that you barely move at all each iteration and get "stuck".
To add here: for a local minimum to occur all those dimensions (or features) need to increase. This is highly unlikely for modern NNs where you have millions of dimensions. If one of the dimensions is going down but the rest up, you have a saddle point. Since you go down only one (or few) dimensions it takes longer.
Is that really an issue? Its really a missing data problem. Your data is "missing" and facebook can infer with some probability you exist and the people your "shadow profile" might interact with.
If six of your contacts have provided them with the same name and email address, then they know with near certainty that those six people have you as a mutual contact. When they spot an ad profile with the same email address, that comes with a dossier on your browsing and shopping habits, which are used to infer your interests etc. It's not a "missing" data problem, it's an embarrassment of data problem.
That’s pretty much all there is to it, yeah, just a missing data problem. But once you slap the “shadow profile” label on it, it gets “shady” and “scary”. A pretty cool lesson in how to make something fairly trivial and non-threatening seem scary to people.
Note: not currently or in the past employed by Meta or any affiliated companies, so that’s just purely my own take.
Newer methods are going back to similar concepts but trying to get past previous bottlenecks given what we've learned since then about transformers.