Sometimes I hear a 10-20hz pulsing sound (so pretty fast, chirping sound) emanating from around my brainstem / back of neck at the level of my ears and a slight feeling of some liquidy movement inside. I always thought it was connected to CSP movement but maybe it isn't. But if it isn't I can't see what else it can be as it happens when I'm completely stationary.
There are also documented effects of your camera quality, and even where your camera is placed (as you tend to look at your screen during a call so your gaze is always relative to it).
probably not very relevant in a zoom meeting but there are people hypersensitive to the eye movements of someone reading from a teleprompter as well! When I figure someone is reading from a script with their eyes moving, it really affects my focus for some reason. Here is someone trying to find the ideal setup to prevent it: https://www.youtube.com/watch?v=8LkRMtWfhn4
first you need to find a way to differentiate human thinking from machine thinking. you need to basically show how human thought is not the result of statistical learning and inference but something completely different. if it is not possible, the distinction is moot, something that only appeals to emotion.
we are working on <impossible problem stumping humanity>. We have considered the following path to find a solution. Are we on the right track? Only answer Yes or No.
It is more obvious when taken to extreme: With the current feedforward transformer architectures, there is a fixed amount of compute per token. Imagine asking a very hard question with a yes/no answer to an LLM. There are infinite number of cases where the compute available to the calculation of the next token is not enough to definitively solve that problem, even given "perfect" training.
You can increase the compute for allowing more tokens for it to use as a "scratch pad" so the total compute available will be num_tokens * ops_per_token but there still are infinite amount of problems you can ask that will not be computable within that constraint.
But, you can offload computation by asking for the description of the computation, instead of asking for the LLM to compute it. I'm no mathematician but I would not be surprised to learn that the above limit applies here as well in some sense (maybe there are solutions to problems that can't be represented in a reasonable number of symbols given our constraints - Kolmogorov Complexity and all that), but still for most practical (and beyond) purposes this is a huge improvement and should be enough for most things we care about. Just letting the system describe the computation steps to solve a problem and executing that computation separately offline (then feeding it back if necessary) is a necessary component if we want to do more useful things.
Not only that but also LLMs "think" in a latent representation that is several layers deep. Sure, the first and last layers make it look like it is doing token wrangling, but what is happening in the middle layers is mostly a mystery. First layer deals directly with the tokens because that is the data we are observing (a "shadow" of the world) and last layer also deals with tokens because we want to understand what the network is "thinking" so it is a human specific lossy decoder (we can and do remove that translator and plug the latent representations to other networks to train them in tandem). There is no reason to believe that the other layers are "thinking in language".
Haxe is amazing, has macros etc. A force multiplier if you are a solo developer for sure. But damn, you feel kinda alone if you are using it. Not everything is an npm install away which negates your velocity gains from using a saner language.
I RARELY/NEVER have to build an app so fast that i just fart out whatever broken code as fast as my fingers can type. IF i get a project like this with a deadline of "yesterday" i politely just refuse. I will be wasting my personal time, and the clients time. And the result will be a broken mess that will eventually take more time to fix, than it would have if i in fact did it "correct" from the get go.
That said Haxe has externs, enabling you to target JS/PHP and use the rich ecosystem both langauges have. The best part of externs is that IF i only use 4 things from given package, i statically KNOW i only use these 4 things, and can refactor more easily, or even build the thing i need myself.
When I'm talking about velocity, I'm not talking about coding fast, but being able to write DRY, flexible yet easily maintainable code that can weather future requirements / refactorings easily. Personally, I'm also talking in the context of my own projects so nobody is breathing down my neck or pressuring me with time. I just want to write good code that is a joy to maintain for years to come.
yes but still, a local model, a lightning in a bottle that is between GPT3.5 and GPT4 (closer to 4), yours forever, for about that price is pretty good deal today. probably won't be a good deal in a couple years but for the value, it is not that unsettling. When ChatGPT first launched 2 years ago we all wondered what it would take to have something close to that locally with no strings attached, and turns out it is "a couple years and about $10k" (all due to open weights provided by some companies, training such a model still costs millions) which is neat. It will never be more expensive.
