I guess we're just going to be in the age of this conversation topic until everyone gets tired of talking about it.
Every one of these discussions boils down to the following:
- LLMs are not good at writing code on their own unless it's extremely simple or boilerplate
- LLMs can be good at helping you debug existing code
- LLMs can be good at brainstorming solutions to new problems
- The code that is written by LLMs always needs to be heavily monitored for correctness, style, and design, and then typically edited down, often to at least half its original size
- LLMs utility is high enough that it is now going to be a standard tool in the toolbox of every software engineer, but it is definitely not replacing anyone at current capability.
- New software engineers are going to suffer the most because they know how to edit the responses the least, but this was true when they wrote their own code with stack overflow.
- At senior level, sometimes using LLMs is going to save you a ton of time and sometimes it's going to waste your time. Net-net, it's probably positive, but there are definitely some horrible days where you spend too long going back and forth, when you should have just tried to solve the problem yourself.
My most recent example of this is mentoring young, ambitious, but inexperienced interns.
Not only did they produce about the same amount of code in a day that they used to produce in a week (or two), several other things made my work harder than before:
- During review, they hadn't thought as deeply about their code so my comments seemed to often go over their heads. Instead of a discussion I'd get something like "good catch, I'll fix that" (also reminiscent of an LLM).
- The time spent on trivial issues went down a lot, almost zero, the remaining issues were much more subtle and time-consuming to find and describe.
- Many bugs were of a new kind (to me), the code would look like it does the right thing but actually not work at all, or just be much more broken than code with that level of "polish" would normally be. This breakdown of pattern-matching compared to "organic" code made the overhead much higher. Spending decades reviewing code and answering Stack Overflow questions often makes it possible to pinpoint not just a bug but how the author got there in the first place and how to help them avoid similar things in the future.
- A simple, but bad (inefficient, wrong, illegal, ugly, ...) solution is a nice thing to discuss, but the LLM-assisted junior dev often cooks up something much more complex, which can be bad in many ways at once. The culture of slowly growing a PR from a little bit broken, thinking about design and other considerations, until its high quality and ready for a final review doesn't work the same way.
- Instead of fixing the things in the original PR, I'd often get a completely different approach as the response to my first review. Again, often broken in new and subtle ways.
This lead to a kind of effort inversion, where senior devs spent much more time on these PRs than the junior authors themselves. The junior dev would feel (I assume) much more productive and competent, but the response to their work would eventually lack most of the usual enthusiasm or encouragement from senior devs.
How do people work with these issues? One thing that worked well for me initially was to always require a lot of (passing) tests but eventually these tests would suffer from many of the same problems
And just a tip for who may be intersted: Claude Opus with Extended Thinking seems to be very good at converting existing code into TLA+ specs.
I've found multiple bugs for personal Rust projects like this (A Snake game that allowed a snake to do a 180 degree turn), and have verified some small core C++ components at work with it as well (a queue that has certain properties around locking and liveness).
I tried other models but kept getting issues with syntax and spec logic with anything else besides Opus.
MSR has a very clear and accessible tutorial on quantum computing for anyone interested in getting up to speed with the fundamentals: https://www.youtube.com/watch?v=F_Riqjdh2oM .
Either Android or iPhones can be customized. The parent has to take the time to sit down and set it up.
The iPhone has a lot of parental setting customization. You can disable certain built in apps, prevent installing anything from the App Store or just prevent making purchases, set screen time restrictions, and a whole bunch of other things [1].
Android has similar settings with Family Link [2].
Imbue | Engineering Manager | San Francisco | Full-time
Imbue builds AI systems that reason and code, enabling AI agents to accomplish larger goals and safely work in the real world. We train our own foundation models optimized for reasoning and prototype agents on top of these models. By using these agents extensively, we gain insights into improving both the capabilities of the underlying models and the interaction design for agents. We aim to rekindle the dream of the personal computer, where computers become truly intelligent tools that empower us, giving us freedom, dignity, and agency to pursue the things we love.
This is very similar to the method I use and, after conversation with him, the method Art Benjamin uses. It's trivial to do in 10 to 15 seconds, though it requires practice, a very small amount of memorisation[0], and a small amount of mental arithmetic. I use it regularly, partly to keep in practice, and partly because once you have the skill you find it's surprisingly useful.
JH Conway used a different technique[1] which I have swutch to when computing days in the current year. It's quicker and easier, but I find that it's harder to compute "The Doomsday" for other years (it's Thursday this year), so I revert to my standard method[2].
Example: Today is 2024/05/24
Years since 2012 is 12
Leap years is 12/4 = 3
Magic month number = 2
Date is 24
Add mod 7 is (12+3+2+24) = (-2)+3+2+3 = 6 = Friday
[0] A "magic number" for each month. There are mnemonics, and they can be computed from first principles if needed. It's also easy to remember a few and then compute the others.
144 : Jan, Feb, Mar
025 : Apr, May, Jun
036 : Jul, Aug, Sep
146 : Oct, Nov, Dec
[2] All values computed mod 7. Take years since 1900. Add number of leap years since 1900 (so (YYYY-1900)/4 rounded down). Add the magic number for the month. Add the day of the month. Then Sunday=1, Wednesday=4, etc.
Because the calendar repeats every 28 years, for more recent dates you can start with 2012 instead of 1900.
Monitoring is done with two systems, one in-house in-band that we might open source one day (was called Epic, currently called Atlas). The other is AppDynamics running as a SaaS application with no dependencies on AWS. There is some useful overlap for when one or the other breaks, we merge the alerts from both (plus Gomez etc) but they have very different strengths as tools.
The queries are the same as what you'd type into the Gmail search bar to filter emails so you can test them out in Gmail and then add them to the list. I've added a comment to the gist with an image showing the trigger to run it daily. You can run it manually from the Google Script editor too for testing.
Edit: Forgot to mention I removed a bunch of queries and only left a few in as examples.
FYI, I used a tool I develop [1] to look into this, and came to the following conclusions:
- Biden will sign the bill (>95%)
- The Commerce Secretary will start the 12 month clock (>95%)
- TikTok will challenge the ruling (>95%), which may delay the clock starting to delist or divest, but they are unlikely to win (20%)
- And if they lose, they will succeed in selling to a US company (75%)
- The sale will be for $30-50B (CI 50%), assuming it does not include certain ByteDance IP
- Walmart and Oracle won’t compete to buy it this time. Microsoft or Amazon are the top contenders, along with a consortium led by Susquehanna or Steven Mnuchin, possibly in partnership with Snap or X.
Scott Garrabrant wrote a nice series on Less Wrong which explains/derives the Kelly Criterion, and relates it to other areas (Nash bargaining, Thompson sampling, Bayesian updating), and how they all follow the same underlying logic (optimising the geometric mean) https://www.lesswrong.com/s/4hmf7rdfuXDJkxhfg
It's definitely worth taking the time to set up a credit freeze with the three big agencies (Experian, TransUnion, Equifax). Initially setting it up is a pain in the butt and is rage-inducing, as you have to provide a bunch of personal data when the whole problem in the first place is that they're careless with your data.
However, once you've got it set up, it's very easy to freeze and unfreeze them. Just keep all the URLs, usernames, and passwords in a secure note somewhere, and any time you need to apply for credit, unfreeze them for a day or a week.
I used to have all sorts of identity theft problems (people taking out credit in my name) but freezing my credit has solved it.
https://nakashimawoodworkers.com (new commissions around $7K-$15K for a coffee table, $20K-40K for dining table, plus shipping; older Nakashima pieces are highly valued in the art world and sell anywhere between $15K-$300K)
Edit: Also, to echo what someone mentioned below, if you're interested in solid wood furniture you should find a local woodworker.
Another edit and thought: I used to own a lot of IKEA furniture and as I've gotten older, have slowly replaced those pieces with items from Knoll, with custom pieces from local woodworkers, with a few pieces from the studios listed above. A lot of people are commenting on the cost, and yes they're expensive and could be considered luxury goods.
But if you like art and design and you care about quality, you save for what you want to buy. I wanted to be surrounded by great craftsmanship, so instead of buying "stuff" and instead of spending money on lots of subscriptions and services, or constantly upgrading phones and computers, I buy one piece of nice furniture every year. I believe the more you appreciate the things around you, the more they begin to influence your own work, and your sense of place.
I regularly see a lot of IKEA furniture on the side of the road and in dumpsters. I think this is the difference between buying "things" and having "possessions" but that's a discussion for another day.
I travelled to Yellowstone National Park. I wandered into the back country office and spoke with the ranger. She was super friendly and informative. One interesting fact she shared is that Yellowstone is almost 4,000 sq. miles and receives 5 million visitors a year. 99.9% of those visitors never travel more than 50 feet from the main road. This means that most of those visitors experience less than 1/10th of 1% of the actual park.
Why do I bring this up? Because this is how most worker's experience their organization. Stay in your lane, get that promotion, best case you get your boss's job. But how did that job come to be? Who setup the training that you took? Most people can't even describe where the money in their business comes from.
It is a tremendous advantage to explore your organization fully. Visit its other offices and learn what your colleagues do and why they do it. Especially as an engineer. You can literally write your own ticket. Last year I was bored and I started to break down our cloud spend. This took me on a little detour. That detour involved a team that was following a process I could not understand. Turns out they didn't understand it either. I little reorganization yielded a $385,000/yr cost optimization. It took me just a couple days. Chances are you swim in a sea of complacency too.
The right and wrong direction are up for debate. Everything is an opportunity cost, so if the product is not successful or making money, then shifting resources away from it to something more productive may sink the product faster, but might not have been the wrong for the business.
I've not known many executive decisions like that (right or wrong) being made without any input from technical side of the business, perhaps not Joe shitkicker in the trenches but technical leads. and I've very rarely encountered a technical person who concedes that their project or product or team should be given fewer resources. Almost always the complaint is that if only they were given more resources, they could have been successful, which rarely understands business and market realities and often confuses cause and effect.
And the egos and empire building and infighting and so on are definitely a thing that happens, and they are definitely not restricted to management. Engineers and technical people are some of the worst offenders!
I'm not trying to absolve executives and managers here, but I do think some engineers need better perspective and yes some empathy with the business side. Almost no new engineer needs to be told that their leaders are poor -- they'll hear it from their coworkers in their second week on the job and continuously after that until they retire. What I've rarely if ever heard is "damn we shipped a real piece of crap here, it must be embarrassing for our executives and sales people who have to sell it and answer to our customers, we could have done much better".
I’m surprised the YC application hasn’t been more of a focus in this discussion.
The following questions are pulled from the application (perhaps an older version):
What is your company going to make?
Why did you pick this idea to work on? Do you have domain expertise in this area? How do you know people need what you're making?
What's new about what you're making? What substitutes do people resort to because it doesn't exist yet (or they don't know about it)?
Who are your competitors, and who might become competitors? Who do you fear most?
What do you understand about your business that other companies in it just don't get?
How do or will you make money? How much could you make?
How far along are you? Do you have a beta yet? If not, when will you? Are you launched? If so, how many users do you have? Do you have revenue? If so, how much? If you're launched, what is your monthly growth rate (in users or revenue or both)?
—-
These questions cover product description, value prop, market analysis, financial projections and more, all of which are the core of a “business plan”.
Sure, the application may be structured in a different way, but the sentiment that PG has never read a business plan seems very disingenuous.
You might look to improve y'all's process by looking at datatdog's interview process. I have never felt more appreciated and well treated than interviewing there.
1) they always give feedback
2) they have more generic positions, get you in the door to some small filter interviews, and then shop you around to find the right team for you, instead of the reverse approach where people shotgun resumes across your company trying to get in the door. The problem with the recruiter and multiple HMs I talked to at stripe is they didn't seem to care about getting people to work at stripe, only getting people to work in their org which didn't have open positions for X.
3) incredibly quick and responsive through the process. My recruiter at stripe did this!
For those looking for a more exercise-oriented approach to cryptography, the Cryptopals challenge (https://cryptopals.com/) is an excellent way to discover cryptography step by step with 64 exercises with an increasing difficulty level
It's great to have you as a customer. Make sure you give product feedback to our team!
Most of the money in SaaS is in large clients in the enterprise. Almost all large SaaS businesses have been built that way (Salesforce, Adobe, ServiceNow, Workday). Once you figure that out monetizing smaller companies goes way down in priority and it's a better strategy to give your product away for free.
For us in particular:
1) It was a great way to grab attention from Mixpanel and others in a crowded market.
2) A lot of those companies become large customers over time when their needs become bigger and more complex. Doordash, Instacart, and Rappi all started out that way and are now huge customers.
3) A lot of those companies and people at those companies get acquired by larger companies over time. Under Armour, Capital One, and Twitter were all companies where Amplitude was brought in through acquisition of a smaller company.
4) It's not that expensive relative to your overall cost base. I believe 8% or so of our server costs go to our free plan, which is significant, but worth it.
We've never received pressure to do that, our venture capital shareholders are very aligned towards winning the market over the period of decades. We did get some stupid (IMO) questions about gross margin as we went public but no one ever gets down to the level of messing around with your pricing plan and free tier. If we were owned by private equity though it'd be a very different story. Those guys are experts at wringing blood from a stone.
Can you link me or someone else link me what you would consider the "ultimate kanban guide"?
I work in an environment that had no project management tools or methods being used other than emails and I've finally gotten buy in on Kanban and I want to make sure I guide my team properly.
From Wikipedia description: Anscombe's quartet comprises four data sets that have nearly identical simple descriptive statistics, yet have very different distributions and appear very different when graphed. Each dataset consists of eleven (x,y) points. They were constructed in 1973 by the statistician Francis Anscombe to demonstrate both the importance of graphing data before analyzing it, and the effect of outliers and other influential observations on statistical properties.
Forget the Hunger Games, here is the dystopian shit right now:
For a medium level leetcode problem here is the time split (for which 20 min are given typically):
1. 4 min => Read/interpret/parse the problem statement
2. 5 min => Formulate the solution.
3. 8 min => Actual code
4. 3 min => Edge cases/review
---------------------
Notes:
* #2 You should think of various algos, DS and lock on to a particular one. If you cannot improve upon the time complexity, you need to make a split decision to either 1) continue to think more or 2) choose the next best and move to #3
* All you can do is steal one min from other phases but at the cost of less time in that phase.
* Think aloud in #2, #3 & #4
* At times even less time is available in case follow-up questions are to be accounted for.
* Code structure & consistency is important. You will be penalized if you write big monoliths.
* Additional conditions when GTFO is triggered: ["Solved the problem in 23 min instead of 20", "Did not think about that 1 scenario even though you took care of most of the important ones", "Solved the DP problem in top-down instead of bottom-up.. loser"]
----------------------
Optimizations:
* #3 should be reduced as much as possible. Choose a lang, know its standard lib by heart. This gives more time for #2
* Think of edge cases while writing code in #3
* At #2 one, you should be able to describe a solution (in your mind) detailed enough to convert into a code e.g Do DFS. Keep a max_val variable. Update max_val on values from left & right sub-tree.
* Develop intuition of doing some #2 work while in #1
-------------------------
What actually happens (or should happen) when someone practices leetcode a lot?
1. Reduce #1 time.
2. #2 => Quick recall of high level patterns already encountered
3. Able to generate multiple solutions.
4. #3 => Translate that pattern as fast as possible. E.g. There is a standard template for BFS, DFS, two pointers, etc
--------------------------
So the entire prep can be divided into two parts:
1. How soon you can zero-in a solution and convert into an outline?
2. How fast you can vomit that into a code?
Once you are comfortable in writing any code fast, you do not have to code every problem in leetcode. I see a problem and only solve it in mind and then check whether the approach was ok or not. I know I can code it fast
Edit: So for me if I have solved 200-300 LC problems, what it really means is I have actually coded in like 40-50 and remaining are just mind puzzles. If I do daily 5 mind solves, I can easily cover 200 in 40 days. I finish this daily routine in 30-45 min. (Hey .. I said this is dystopian)
------------------------
High level view:
* Practice incrementally and not in 2-3 months sprint. Think of it as daily exercise.
* This is just the way it is. May be I will do my part to change the system from inside when I am in an influential position. If you are able to find companies which do not do this shit, really good for you.
* Know that as senior dev, the margin for error is even less. So more important to practice incrementally.
--------------------------
I do not approve of this. Its bad. I am just a guy dealing with the "system".
So, you have an email address at Gmail which you're using to authenticate everywhere, which is your username at most services, and possibly your main point of contact.
Then you mention something innocuous, I don't know, a cucumber maybe, on YouTube in a comment, and now your account is gone because apparently cucumbers are a part of some neo-nazi slang or a secret pedophile ring slang, and you got reported into oblivion.
Since your email address was also a part of getting into your bank account, now you've got a problem of getting your money, too.
The hoops you have now to jump are worse than changing spelling of your legal name or changing the fucking citizenship.
Still, the internet hivemind is raving that "it's not censorship if a private company is doing it", "they are a private company so they have a right to boot anyone off".
Of course another cohort of smug internet users will say: just self-host your email! Well, good luck getting your email deliverable if you're a Joe Schmoe and not one of the big players in the first place.
The big companies pulled the blanket so much that without using their services, you may as well not exist on the internet. They've made it extremely difficult, at least. They got, in some sense, bigger than many individual governments in regulating information channels. And yet they dodge all responsibility on the premise of being "private companies".
Every one of these discussions boils down to the following:
- LLMs are not good at writing code on their own unless it's extremely simple or boilerplate
- LLMs can be good at helping you debug existing code
- LLMs can be good at brainstorming solutions to new problems
- The code that is written by LLMs always needs to be heavily monitored for correctness, style, and design, and then typically edited down, often to at least half its original size
- LLMs utility is high enough that it is now going to be a standard tool in the toolbox of every software engineer, but it is definitely not replacing anyone at current capability.
- New software engineers are going to suffer the most because they know how to edit the responses the least, but this was true when they wrote their own code with stack overflow.
- At senior level, sometimes using LLMs is going to save you a ton of time and sometimes it's going to waste your time. Net-net, it's probably positive, but there are definitely some horrible days where you spend too long going back and forth, when you should have just tried to solve the problem yourself.