Hacker Newsnew | past | comments | ask | show | jobs | submit | more eshack94's commentslogin

Are there any higher-context 70B finetunes yet with good benchmarks?


As well as being a pain to search, it's complete black hole for information.

The trend of more and more projects and companies using Discord as their primary forum for support, docs, and discussion boards is probably not one that will be looked back on as having been a good idea.

I worry that there's so much valuable information stored in prior chats that will simply disappear if Discord ever goes away.

In addition to this, this valuable information is not accessible/searchable via the web without a Discord account. Plus there's the 100 server limit for free accounts, which can become a problem depending on how many projects/communities one is involved with.

That being said, Discord is convenient and cheaper to license/host communities than Slack, which I think is part of the allure.


Does anyone know (or is it publicly known) when the 400B variant will be released?


It would be awesome if all these newspaper archives were freely available via Archive.org or a similar service. Seems strange to need a subscription to view old newspapers. Perhaps part of the fees pay for hosting costs and the work that goes into compiling so many sources and slapping a easily navigable UI on top of it, etc.


Finding papers and digitizing them takes a a lot of work. Is nice that sone not for profits like archive.org do that work for no cost but expecting others to do it for you for free is what seems strange to me.


Archive.org, for example, does a lot of good work. But without too much hyperbole, there's essentially an infinite amount of paper with lines on it in libraries, historical societies, and private collections around the world. It's never going to all be digitized.


BetterSnapTool is great for this. Check it out.


Angel + Zeus = God. o_O

This little game is too entertaining. I hope the author open sources it, or writes a technical blog post on how it's implemented.


How many greek gods have you managed to get? I got Zeus, Hera, Athena, and Poseidon, plus Achilles if he counts.


I got Dionysus from God and drunk


This isn't universally true. Some people truly enjoy experiencing other cultures and parts of the world for its own sake. Not everyone who travels for pleasure and curiosity are doing it "for the gram". That being said, there are also many people who are.


> It’s Tuesday morning. The year is 2009. You’re just waking up after a long and boozy New Year’s Eve with friends.

I digress, but I think the first day of the year in 2009 was Thursday, January 1st, 2009.

Well-written post. I share the sentiment and I find myself longing for new ways to find creative/interesting content on the web. Seems like there are too many gatekeepers of content these days and it's hard to keep up with blogs and niche interests. More difficult than it used to be, at least.


> it's hard to keep up with blogs and niche interests. More difficult than it used to be, at least

It is hard to keep up with niche interests! I blame it on being 36 with real responsibilities instead of 22 and in college.

I suspect that has a much bigger impact than the state of the web/internet today. My younger more energetic coworkers tell me about all sorts of fun and wonderful things they discover and deep-dive on TikTok. Just as I used to on blogs. The format is different but the variety and serendipity remains. If anything, "kids these days" have way more content and creators than we did.


Way more content, maybe, but on platforms that are not made for long term retention and curation, but for attention span of a fruit fly and optimized for engagement. The content might get the quick giggle or wow, but then it has passed. TikTok and similar are not the kind of platform that I would search answers to questions on or that I would use to follow a hobby in depth. Perhaps my hobbies don't lend themselves to being represented by TikTok shorts or whatever they call them there.


And yet I listened to a podcast once where a tax accountant explained that Instagram Search is her strongest lead pipeline.

At my dayjob we do women’s health, actual clinics with real doctors. Many of our users come from Instagram and TikTok ads. Because yes people will in fact choose their doctor based on a good Instagram/TikTok presence. In fact any time I mention the brand to female friends who live in our target markets they go ”Oh yeah! I’ve seen your ads on Instagram”. It’s never search, or a billboard, or a blog, or youtube, or even me telling them about it. They recognize us from Instagram and Tok.

It’s a wild world out there my friend. Makes me wanna yell at clouds every day.


Backing this up. Google is presently most feeling threatened by TikTok, not OpenAI.

Because an entire generation of new American adults does not use web browsers, like much at all.

Want a burger? You probably open Chrome, go to Google or Kagi and type “Burger $myCity”

People under 25 use TikTok and Instagram and just look for “burger” and are blasted by 300 10s videos of real people munching and smiling. Like a perfect commercial and entirely crowdsourced.

That’s the new internet. The kids only know ‘content’. They don’t know what the fuck an HTML file is.


I'm curious to what extent this is honest to God actually true. Maybe the very first time I ever move to a new city and want a burger, my first thought is find some directory service telling me where burgers can be found. Right now, I have a kitchen and a grill and would make the burger myself as a first choice, and if not, I've lived in the same house for seven years now and have a great dive bar a block away I can walk to that my wife and I have hung out at forever where we know the owner and staff and they make terrific burgers, better than anywhere I've been in the city in the now nine years I've lived in this city.

Do people really just perpetually not know where to get something they want in the place they live?


> Do people really just perpetually not know where to get something they want in the place they live?

No but a) people travel and b) the young post-college demographic is usually new to the area. By virtue of being young and freshly out of college. They really don’t know the city yet!

Personally when I travel my search for burgers goes straight to Apple Maps.


Wonder what that accountants ability to keep clients around looks like vs. Intentful Google searches, and what that market would look like.

You have to take the serious consideration that winning customers from tiktok is going to be a wildly different persona than from google.

Churn and burn practices are for folks who've not seen the 5th year of their used to be sustainable market crash when arbitrary platform dynamics change and they don't realize they've been working with the wrong type of client that whole time.

I work in Healthcare as well. It's just a giant farm so folks will take anyone who is alive and insured. I could see TokTik do well there.


I, and I'm sure there are many others in their 30's who would agree, prefer to get my information in written form. Pictures/diagrams are fine, but I don't want to watch a 10-15 minute video, or even a 2 minute video to get information I can read in less than 30 seconds. "Kids" these days seem to prefer the video medium much more. I don't know why, but I find it interesting that reading scores have also tanked a lot in the last 20 or some odd years.


S. P. Somtow nailed this 43 years ago in _Mallworld_. It's completely confusing to me also.


Wow, I never heard of that, sounds like I need that on my kindle ASAP. Thanks!


Too many nitpickers nowadays too.


After long and boozy night? It might as well be Tuesday :)


For you, that was the worst hangover of your life. For me, it was Tuesday.


This is the most accurate and clear-eyed take I've seen on GPTs so far. They might be useful, but they're not magic and they're intended to enhance OpenAI's moat-building operations to make it harder for people/companies to walk away from the platform amid future competition.


I believe GPTs are an attempt by OpenAI to generate training data. How can you get data at level N+1 when you have a model at level N? You give it more resources - more tokens (CoT), more dialogue rounds, code execution, web search, local KB, human-in-the-loop. A model with feedback from human and tools can do so much more. And by training on this data they can incorporate these skills in the next generation. It's like RLHF in the sense that the training data contains portions generated by the model itself (specifically model errors) and feedback. It's on policy data, generated with the involvement of the model, not something you can scrape from the web.

Let's do an estimation - if they have 100M users and each of them generates 10K tokens in a month, that's 1T tokens per month. In a year they have generated 12T tokens, which is very close to the GPT-4 training set size of 13T. Looks like they can generate serious data with this method. They don't even need to train directly on it, they could rewrite it as high quality training examples, without copyright and PII risks, because LLMs are great at rewriting and rewording and MS has already shown that synthetic data is better.

Google lost the start and they don't have the human-AI chat logs OpenAI sits on. So they are trying to do the same trick but without the human in the loop. Hence the declarations that Gemini will use some techniques from AlphaZero. They are teaching models by feedback too.


Thanks for sharing this anecdote.

Your mother seems like a very curious and inquisitive person, who actually wants to learn about something before gravitating to strong opinions about it. That's respectable.

Seems like she also challenged you (in a positive way), by asking you to reformulate your explanations until they were approachable but not too ELI5. That takes a lot of patience and solid communication from both participants. Seems like a win-win situation in the end, enabling her to gain a solid understanding of the fundamentals, while allowing you to refresh your own understanding, and strengthen your technical communication skills.

Kudos to both of you, and thanks again for sharing.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: