Honestly, it's not our problem. Once a service becomes so vital it cannot be terminated without any meaningful process. My meta developer account is suspended and none of my appeals are responded to . Who can I talk to? Nobody. It's wrong.

Have a similar problem. Ran meta ads for about a week, account shutdown. asked to verify ID. Jumped through the hoops. Got a notice perma ban, decisions final.

I still don't know why. I've talked to their chat bots many times. I can't run facebook or instagram ads ever again is seems. (also can't boost posts and some other things mentioned)

Considering they are doing some big push for small business success with meta / facebook / instagram, and our AI helps small business do amazing things...

That just seems like fraud to me considering the experience I have gone through.

Part of me thinks if I had a business that made most of it's money from advertising that I would help people make ads that align with their policies - I got so many generic - follow our terms, community guidelines - nothing explains what they heck is wrong. No way to fix it. No way to keep running ads.

This is your business, this is your AI, this is your policy, and you are trying to promote a welcoming space for small business and a dependence on your AI?

What would be the non Mac computer to run these models locally at the same performance profile? Any similar linux ARM based computers that can reach the same level?

Framework Desktop is the closest one with the MAX 385/395 chip. It's mostly about the memory being fast enough rather than just CPU/GPU oomph.

The 64GB model is 2240€ base and the 128GB is 3069€ base + all the stuff you need to add to make it an actual computer.

As a comparison the 64GB Mac Mini is 2499€ here and a 128GB Mac Studio is 4274€.

Note though that that a MAX 395 has half the memory bandwidth of a M4 Max chip, and the memory bandwidth is going to be the biggest limiting factor, so you'll likely be getting around half the tokens/second with that Framework Desktop.

There's a reason why it's cheaper than the Mac equivalent and it's not all because of Apple's premium pricing =)

But it's still the easiest and cleanest way to get decent local AI speeds on a non-Mac.

Not even close. If you want to run this on PC's you need to get a GPU like 5090 but that's still not the same cost per token, and it will be less reliable and use a lot more power. Right now the Apple Silicon machines are the most cost effective per token and per watt.

It's odd no manufacturer jumped on this wagon to offer a competitive alternative.

Is there even enough market for this?

These models are dumber and slower than API SoTA models and will always be.

My time and sanity is much more expensive than insurance against any risk of sending my garbage code to companies worth hundreds of billions of dollars.

For most, it's a downgrade to use local models in multiple fronts: total cost of ownership, software maintenance, electricity bill, losing performance on the machine doing the inference, having to deal with more hallucinations/bugs/lower quality code and slower iteration speed.

Actually yes. For example, I run local models for ingested documents, summaries, etc. The local models are fine, and there is no need for me to pay for tokens. Performance is adequate for that purpose as well. There are many other cases where I run at scale, time is flexible so things can move slower, and I rather keep it all in house. I'm not even getting into areas where data cannot leave the premises for legal reasons. Right now I'm limited with GPUs mostly. But if that world of local models on Apple silicon is so "good", there is room to expand it to other fruits...

> These models are dumber and slower than API SoTA models and will always be.

Sure but you're paying per-token costs on the SoTA models that are roughly an order of magnitude higher than third-party inference on the locally available models. So when you account for per-token cost, the math skews the other way.

Intel’s doing interesting things with their Arc GPUs. They’re offering GPUs that aren’t super fast for gaming but are relatively low power and have a boatload of VRAM. The new B70 is half the retail price of a 5090 (probably more like 1/3rd or 1/4 of actual 5090 selling prices) but has the same amount of memory and half the TDP. So for the same price as a 5090 you could get several and use them together.

Is it feasible to run LLM inference comparably without CUDA or Rocm? How much of the cost performance goes away?

I wonder if the Snapdragon X Elite already caught up with the Apple's M series in that regard - does anybody know?

Mavis Beacon thought me to touch type on my Amiga as a kid, and I'm grateful to her. In recent years I've been using a Logitech membrane keyboard. It's great for a membrane but mostly it's quiet. I work at night and that's a hard requirement. I'm looking for a mechanical as quiet as the Logitech and it's a struggle. Feels like there is a gap in the market for (really) quiet mechanicals.

There are definitely quiet mechanical switches out there. I'm pretty happy with my Kailh LP Whale silent tactiles. My old IBM style was so loud at night.

Consider Topre, e.g. a Realforce R4 keyboard.

It's OK, we're doing badly across the board, not just computing. :-(

I don't disagree, but I can't join you wishing for it to happen. It's going to be bad for all of us.

The longer it's delayed, the worse it will be.

I am going to need a mac to perform some app build for ios sometimes. I was thinking to get a used mac mini for that. Would the neo be a cheaper, but still viable option?

What an interesting and very apt choice of name!

Edit: I feel like it needs the addition that Coccinelle was the stage name of Jacqueline Charlotte Dufresnoy (1931), a French performer and probably the first celebrity who underwent a transition from male to female.

Here is what worries me the most at the moment: we're in a period of hype, fire all the developers, we have agents, everybody can code now, barrier is not low - it's gone. Great. Roll up a year from now, and we have trillions of lines of code no human wrote. At some point, like a big PR, the agent's driver will just say yes to every change. Nobody now can understand the code easily because nobody wrote it. It works, kinda, but how? Who knows? Roll up another few years and people who were just coding because it's a "job" forget whatever skill they had. I've heard a few times already the phrase "I didn't code in like 10 months, bruh"...

Then what?

Not saying I'm not using AI - because I am. I'm using it in the IDE so I can stay close to every update and understand why it's there, and disagree with it if it shouldn't be there. I'm scared to be distanced from the code I'm supposed to be familiar with. So I use the AI to give me superpowers but not to completely do my job for me.

I think the idea is that by the time those trillions of lines of code start to cause maintenance problems, the models will be good enough to deal with those problems.

We'll see, I guess...

That won't solve the problem that humans will lose the skill to write code. It will become a hobbyist pass time. Like people listening to 8-tracks now...

Why is that a problem?

Sewing by hand is mostly a hobby but it doesn't seem to be a problem for the textile industry. Planting seeds by hand in a field is a hobby too. So is cutting down trees with an ax.

It sucks but our skill set is (or will soon be) worthless. If it's a consolation, the same is true for anyone who makes money with their brain.

That won't be a problem -- again assuming the vision comes to pass -- any more than the inability to write Latin and Greek holds anyone back today.

But Greek and Latin don't underpin our entire planet's infrastructure... Software does.

A better analogy might be to point out that microprocessors run everything, but almost no one needs to know assembly language anymore.

That's almost me. I've always used Facebook as a tool to keep in touch with friends around the world. My friend list is 95% people I know in real life. A small fraction of them still posts. I also get a lot of slop in between. The filler posts. I am waiting for a Facebook resurgence or a Friendster comeback.

When your surname is McCracken, you're kinda obligated to write an adventure game...