GPT-3. I assumed that's what most people mean with ChatGPT. Usually, people specify for 3.5 or 4.

Is GPT-4 significantly better with Japanese?

GPT-4 is dramatically better with everything except speed. The minutia most people in this thread are complaining about is almost completely solved by GPT-4.

For example I used it over like 30 minutes to conceptualize, solve, and write some code that draws a graphic for a simple physics problem (0-shot) that I could intuitively understand but had no (physics and math knowledge) tools to calculate properly and it was a great experience.

It let me pick and change how what properties i wanted represented on the graph, knew what center of mass means, how to calculate it for a weird object, got something that feels correct to me, drew it out with various representations where i used color, size, shape to represent the various distances, weights, clusters, etc.

My non programmer friends have used generative networks to make designs for prints, incredible and generally accurate folk art, full mobile games.

People are sleeping on this. Don't be them.

Specific examples don't generalize though. My experience with ChatGPT (GPT-3) is that it is vastly better at dealing with English than Japanese. That is still probably true with GPT-4, but that doesn't mean there hasn't been progress in GPT-4 with Japanese, which a sibling comment says is the case. But it's not because it does better at $task that you can extrapolate to a different one. People have reported GPT-3.5 being better at some things, after all.

I cannot speak for Japanese, but I'd say that GPT-4 is better at Russian.

That the model is better in English is no surprise given that most of its training corpus is in English. In fact, based on the sentence structure of the output when it speaks in Russian, it's clear that what's happening there is some kind of real-time translation from English.

That aside, I have yet to see any task on which GPT-4 wasn't at least as good as, or better than, GPT-3.5. I'd love to experiment with that. Do you recall any specific examples?

Yeah that's fair, it may be that Japanese ended up getting pruned out. I sincerely doubt that though, it seems very, very unlikely.

It's night and day in Japanese.

I guess I should give it a spin then.

GPT-4 still unfortunately makes up words. Which is not totally surprising considering its tokenizer.

ChatGPT is 3.5, and the one mostly used is 3.5-turbo, that's the only free one and it's the default if you pay (though you can pick the legacy 3.5, or 4).