Hacker News new | past | comments | ask | show | jobs | submit | slashcom's comments login

There’s lots of work on distillation, smaller models, approximations, etc. People already have simpler forms of these running on smartphones. Models seem to be growing faster than we can make them small though :D


Yes, these things keep us up at night as well :-).


A DGX-2 will do just fine.


Quite a few of my colleagues came back Monday the 27th. I imagine many stay longer since the trip is onerous.



Close to 400 comments. It was a good discussion.

https://news.ycombinator.com/item?id=20662232


yep


fp16 saves a lot of memory and is worth doing. I've not had trouble fine tuning all these models with fp16.


Have you fine tuned 774 successfully using a single GPU?


Zipf’s law


Beat me to it. To expound upon this, Japanese is not unique in basically adhering to Zipf's law. In many organization data sets, including the vocabulary of most languages, the most commonly used word is twice as common as the second-most, and the second-most word is twice as common as the third-most, and so forth.


> second-most word is twice as common as the third-most, and so forth.

Normally Zipf's law refers to the frequency being inversely proportional to rank - i.e. the 3rd most common element would be 1/3 as frequent as the first, not 1/4th.


Links to papers:

https://arxiv.org/abs/1811.00207 - Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset

https://arxiv.org/abs/1811.01241 - Wizard of Wikipedia: Knowledge-Powered Conversational agents

https://arxiv.org/abs/1810.10665 - Engaging Image Captioning Via Personality

https://arxiv.org/abs/1902.08654 - What makes a good conversation? How controllable attributes affect human judgments

And a demo of a bot that they've produced: https://www.facebook.com/Beat-The-Bot-212188996195556/


The demo bot keeps repeating: "You do not currently have access to this page". If anyone has access and can paste a few examples of conversation, it would be useful to understand if results are good for production bots.


From the paper:

"$50,000 was divided among the human participants based on their performance to incentivize them to play their best. Each player was guaranteed a minimum of $0.40 per hand for participating, but this could increase to as much as $1.60 per hand based on performance."

So the humans weren't betting their own money, but they still made more money if they won.


That should tell you how grad students are paid.


Indeed it may somewhat be a result of financial illiteracy, but today's young people are significantly better educated than previous generations (https://en.wikipedia.org/wiki/Educational_attainment_in_the_...).

With the cost of education, healthcare, and housing all rising significantly faster than inflation, it's possibly just harder for many to save up that emergency fund. Indeed, it's particularly difficult to justify having an emergency fund if you're also facing five figure 8% interest student loans.


A difference I've noticed between myself and many of my friends is that I view savings as "Money you don't touch", whereas they view it as "Money you haven't spent yet"

That difference in perception leads to me having a constantly growing bank balance, while they have developed a habit of saving then splurging, repeatedly. Which isn't really building wealth, but more buffering for large pre-planned expenses.


I mostly have your attitude, but it isn't in fact correct. Money saved is to be splurged with latter. However my splurging will be on emergencies or a nice retirement. Anything leftover after the above should be on toys today (though I might need to save for them)


In my case I don't particularly want more toys, which makes my approach easier for me than for others. I personally view it as not needing to work as hard.

I have enough money, and have enough fun, so I can relax a bit rather than pushing myself for more income.


I think a higher number of people taking out five figure 8% loans that aren't used to purchase an education making it relatively painless to pay back, is solid evidence of less financial literacy. There are still ways to get a college degree without taking out 5 figure loans. You just have to go to a cheaper school and work while in college. It's what I did.


I mean, that’s the thing though. Undergrad in state tuition plus R&B for NCSU and UTexas (programs I got in state tuition for) is at 9k and 10k per year with 5% interest for undergraduate, and more like 6-8% for MS students.

You can be employed in a high paying sector like tech after going to an in-state engineering school, and still l end up with $2k tacked on in your first year of employment. So then what is more rational: pay off the loan as fast as possible, or build up an emergency fund with a generous 2% ROI.


The education system does do a poor job of educating people on financial matters, however.


There was just a post on HN the other day about how the top priority for Millenials and Gen Z is travel, so I would say that the younger generations are not significantly better at financial literacy and in fact may have expectations even more out of line with their financial reality than other generations.

https://news.ycombinator.com/item?id=19963476


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: