I think this somewhat misses an important nuance. Japanese PCs had to be different early on because of the complexities of the written language. All of the important characters could be handled in just a few bits (7 or 8) and low resolution in Western markets, with different fonts and character maps dropped in to support a few different alphabets.
But in CJK countries, things were much harder and the entire I/O system had to be significantly more capable than what might pass for usable elsewhere. This meant larger ROMs, larger framebuffers, higher resolution displays, more complex keyboarding systems, the works. Everything was harder and more expensive for a long time. A common add-on was ROMs with Kanji (Chinese derived characters) support in the same way a person in the West might buy a new sound card or get a VGA card. Except this was just so you could use your new $1200 computer (in today's money) to write things on.
Back then, given limited memory, you also ended up with a ton of different display modes that offered different tradeoffs between color, resolution, and refresh. Because of the complex character sets, these Japanese systems tended to focus on fewer colors and higher resolution while the west focused on more colors at a lower res in the same or less memory space (any fans of mode 13h?). The first PC-98 (the 9801) shipped in 1982 with 128k of RAM and a 640x400 display with special display hardware. The equivalent IBM-PC shipped with 16KB of RAM and CGA graphics which could give you a display no higher than 640x200 with 1-bit colors but was mostly used in 320x200 with 4 (terrible) colors.
Even with similar base architectures, these formative differences meant that lots of the guts of the systems were laid out different to accommodate this -- especially in the memory maps.
By the time "conventional" PCs were able to handle the character display needs (sometime in the mid-90s), they were selling in the millions of units per anum which drove down their per unit prices.
The Japanese market was severely fractured and in a smaller addressable market. Per unit costs were higher, but the software was largely the same. Porting the same businessware to half a dozen platforms cost too much. So now the average user of the Japanese systems had a smaller library of software which was more or less a copy of what was on IBM PCs, on more expensive hardware -- market forces solved the rest.
(btw, the FM Towns, IIR, also had specialized graphics hardware to produce arcade-like graphics with tiles and sprites and so on, making it even more different)
Some of this history also informs why home computing lagged in Japan compared to the West despite having all of the other prerequisites for it to take off.
Excellent summary. A few additional comments from personal memory:
I have lived in Japan since 1983, and I started working as a freelance Japanese-to-English translator in 1986. I wanted to produce clean-looking text in English for my clients, so after a few months using a manual typewriter I took out a loan and bought a Macintosh with a dot-matrix printer. If I remember correctly, it cost six hundred thousand yen. The Mac could not handle Japanese; when I needed to write Japanese text, such as for notes to clients, I wrote by hand. I eventually bought a dedicated Japanese word processor for writing clean text in Japanese.
Around 1992, I bought a modem and went online, first to a local foreign-run BBS and then, a couple of years later, the Internet. Many of the first friends I made online were Japanese-English translators like myself, and some of the most active discussion groups I took part in were about the Japanese language and translation.
The display of Japanese characters in our online discussions was a problem for a long time. Even as more and more of the participants became able to type Japanese on their own computers, they were using a variety of OSs and character encodings, and the Japanese parts of their messages, when posted online, would be corrupted more often than not. When discussing a particular Japanese expression, we would have to romanize the Japanese and, sometimes, explain what kanji were used.
Here’s are two examples from posts to a translators’ mailing list in 1998:
> While this handbook uses "åòå≈ê´" for "robustness", the systems engineers I work with prefer "ÉçÉoÉXÉgê´" <robasutosei>.
> Ruth, the kanji for taikou are tai (as in taishi - Crown Prince) and kou (as in kugurido - the radical is mon with gou inside (gou = au/awasersu). Does this help? The dictionary meaning obviously does not make sense here.
This made it impractical to discuss longer texts or to have our discussions in both English and Japanese.
It was a great relief when, around 2000 or so, the encoding issues were gradually resolved and we became able to write Japanese freely in our online discussions.
(Addendum: I am still in touch with some of the people on that mailing list, including the Ruth mentioned above. In fact, last month I attended a party in Yokohama in honor of her and her husband’s 55th wedding anniversary. Several other friends I first met online in the mid-1990s were there, too.)
I would imagine things begin to improve around 2000 due to the broad adoption of unicode? I remember there being an absolutely huge number of encoding systems for the various CJK languages back then, but I think Windows eventually guessed/settled on UTF-16 IIR.
I didn't live in Asia during this time, but was heavily involved in writing some multilingual capable desktop windows software and was very aware of these challenges. I remember one colleague who worked on our Chinese language material having to buy an expensive copy of a Chinese British telegraph code book.
> I would imagine things begin to improve around 2000 due to the broad adoption of unicode?
Yes. I don't remember the exact timing, but it was the switchover to Unicode by Windows and Mac that finally resolved the character encoding conflicts (mostly).
I don't know how much attention this got outside Japan, but there was significant opposition here to Unicode for a while from some Japanese intellectuals. Handwritten and, sometimes, typeset Japanese has traditionally allowed for a lot of minor variations in the forms of kanji. A person might write their name with an extra stroke in one character, for example. Some of those variants are ignored in Unicode as well as in the Japanese encodings on which it is based, and some people kicked up a fuss about it in the Japanese press. I remember reading rants accusing Bill Gates of trying to suppress Japanese culture by imposing a homogenized character encoding through Windows.
The controversy has long since died down, though there have been efforts to develop more comprehensive character sets of Japanese and other Asian languages, such as TRON Code and Mojikyō:
Oh it gets attention on Hacker News to this day. Any time there's a thread about something Unicode one or another of our Japanese colleagues will show up to make a fuss about how Unicode is quite insufficient for Japanese, for this reason or that.
In 2024 I'm out of sympathy for this. There's an entire tertiary plane for extended Hanzi/Kanji/ideographs, if there are still missing glyphs please take it up with the Unicode Consortium, because there's plenty of room and it is reasonable to support one (1) character encoding now.
But it is in fact Microsoft's fault that the farcical attempt to squeeze "CJK" into a two-byte encoding was even attempted. They were firmly committed to the idea that UCS-2 could ever be viable and we're stuck with various consequences of this fatuous premise to this day.
In fact, I occasionally run (currently maintained) CJK commercial software on my en-US Windows installation and I still run into ??? from time to time and have to guess what the text is supposed to be. It’s a shitshow.
Heck, the stupid Windows application for the "my number card" has this problem, and it is an official application coming from the government itself! How is it possible that in 2024 we still have this problem?
For the record: this happens because text encoding problems was solved by broad adoption of not Unicode but by localized Windows 9x. Windows NT and apps on it to this day uses whichever encoding used by equivalent ancient regional versions of Windows 95 for language/region it booted in, e.g. cp1252/ISO-8859-1 for American English and cp932/Shift_JIS for Japanese.
True unification into Unicode only happened during the mobile shift in the form of iOS/Android.
Between 2010 and 2020 I've ordered something from Amazon around 5 times, and each time they spelled my legal name wrong, each time in a different way (due to encoding issues). And I'm an european, living in a large country. So I'm not surprised CJK languages have this problem.
Yeah and a part of e-tax literally asks you to have Japanese Windows in fine print or it fails silently without any warning messages (Fixed by changing chrome locale to Japanese thankfully). Gotta love Japanese websites. I just go into the office and hand write the forms because it beats trying to debug cryptic issues.
Many Japanese PC games and desktop applications aimed at the home market, from small to large developers, also do sort of these things. Just look up the word "AppLocale".
There is a system wide setting that changes all non-Unicode text encoding to another code page e.g CP932 for Shift-JIS. Third party tools are available to do the same conversion on a per application basis.
It’s not as bad as trying to load some really old CJK web pages on mobile devices: few mobile browser has an accessible option to select character encoding and there appears to be none on iOS. The only option is to change the system language and that didn’t always work for more obscure character codes.
I spent a while playing with `iconv` commands to solve your mojibake, reinterpreting bytes in and out of Shift-JIS, but I didn't get it - i'd love it if anyone managed to figure out the exact encoding,
Going way off the beaten path, this post makes me realize how lucky the West was to be able to develop practical computers when they did. And it also has me speculating how challenging it could be for an alien civilization to develop modern computing, depending on the complexity of their interface with reality. English is surprisingly utilitarian and straightforward for a first pass at a computing device. But one could imagine some alien civilization with far more foreign concepts of communication struggling to develop practical input devices and displays during a primitive computing era.
It's not the English language but the latin alphabet that helped there. It had already been optimised in ancient times to a limited set of easily readable characters, and then was further standardised by the introduction of the printing press. Going from a collection of metal pieces to a collection of bitmaps is way easier than adapting a handwritten languaje (which already had problems adapting to the printing press).
To add to this, there are some specific characteristics or acceptable practices within European alphabets (by the time that computers were invented) that helped.
- The total number of characters needing representation is small. In the most limited case, you can get by with just the majuscule (upper case) characters. This means you can represent all letters in just a few bits. Representing all upper + lower + numbers + basic punctuation for almost any single language in Europe, even non-latinate languages is possible in just 7 or 8 bits!
- The glyphs are linearized in a consistent direction. So data storage to display logic is very simple. This is opposed to languages like Korean where characters are assembled into syllables and text can flow left to right horizontally or right to left vertically.
- Most European languages have both print and cursive forms. By the 20th century it was acceptable to use the disconnected print form, which made display logic much simpler (just copy the 8x8 bitmap in ROM to the screen). But importantly, special semi-ornamental characters, like ligatures, were acceptable to separate.
- A more or less universal base-10 number system meant that we only needed to include 10 digits in the character set. Arithmetic only requires a handful of symbols. So you can include a pretty big subset of math in your character encoding. This is opposed to other systems that use distinct characters in ways similar to Roman numerals (e.g. Chinese numerals which also have financial and lay versions of counting).
- Punctuation and simple arithmetic characters had been basically boiled down to about a dozen glyphs.
- If you use 8-bits for your character encoding, you have so much space you can also include pretty much every character variant needed for all Latin using languages, allowing you to hit an addressable market of billions of people with the same 16k ROM.
>which already had problems adapting to the printing press
as i recall the printing press was developed in the East first. Each page was essentially a wood cut and pressed on to the page, the major innovation that Gutenberg in the west had was movable type more than the press.
And even then, Chinese had already used movable type ~400 years before Guttenberg, possibly even for "serial numbers" on printed banknotes. The oldest surviving book printed with movable type was published in the late 1370s in Korea: https://en.wikipedia.org/wiki/Jikji.
Even woodblock printing, while it was seemingly know in Europe or at least the Byzantine Empire since the 11-13th centuries only became heavily commercialized and widespread around the same time as Gutenberg's press pretty much in parallel with it (for playing cards, illustrated books etc.).
There was something special about Europe in the 1400s. I'm not downplaying his skills/ingenuity but Gutenberg's greatest achievement was doing what he did at the right time and place. It seems there wasn't enough demand for books/printed materials that would have justified the needed investments anywhere else.
Another thing about the development of movable type. I've heard that Gutenberg's main contribution was the development of an alloy that made metal type feasible, and that the (earlier) Korean solution was to use ceramic type.
They also tried bronze and copper as well (mainly for money which used a mix of plate and movable type).
The transition from wood to bronze metal type might have independent in Korea, though. They even had a pretty extensive printing industry as far as we can tell but it was fully controlled by the state/emperor and non official printing was illegal.
I'm certainly not an expert but at least in China's case the type of inks they used didn't really work well with metal or even ceramic type which probably made the process relatively inefficient.
Butyeah, Guttenberg invented a new alloy, ink, moulding method which made the process much cheaper and more effective.
I think at this point we understand that 'inventing' or 'discovering' means coming up with the solution/discovery that took off in modern culture/usage in a way that shaped our current world.
Your analysis is correct. I recall in the mid 80's a number of Japanese PC's emerged. They all seemed to want to make a walled garden so they could sell add-ons into it for extra $$, while, as you say 'the hills are alive' with mass made items for the emergent PC standard enabled by Microsoft's provision by sale/licence of MS-DOS while preserving their ability to sell fully compatible same-ware. There was only a small hurdle, the IBM BIOS, which IBM lawyered to death any and all copiers. Soon clean-room original BIOSes were created by a few companies and the gold rush was on.
IBM had lost all hope, but sold enough to business and governments to make a good business. They tried a walled garden with OS2 and their MCA(Micro-channel architecture) bus. All US PC makers jumped on this, millions of plug in cards were made - buyers - crickets, crickets, crickets, I suspect many billions went down that rabbit hole. I recall one scrap salvager processing tens of thousands of Zenith MCA boards for gold a few years later.
The Japanese finally saw the light and focussed on laptops and those had a good run, but finally they faded, IBM also made their think-pads, but eventually sold to Lenovo. Us makers - Dell, Apple and HP persist - I am not sure which are US made? https://en.wikipedia.org/wiki/List_of_laptop_brands_and_manu....
>(btw, the FM Towns, IIR, also had specialized graphics hardware to produce arcade-like graphics with tiles and sprites and so on, making it even more different)
I originally thought might have confused it with the X68K, whose it's closeness to arcade hardware such as Capcom CPS-1 allowed for near (but not fully, contrary to popular belief) perfect ports possible at the time. (Capcom even sold a Genesis/SNES controller converter for X68K for the Street Fighter II ports.) However, the FM Towns does have its own share of arcade ports, mainly involving Sega, Taito and Capcom titles, such as After Burner, Operation Wolf and Super Street Fighter II, respectively.
On FM Towns side, their commercial games list appears to be intermediate between X68K's "arcade at home" experience and PC-98's graphics that prioritize RPGs and visual novels (often smut in nature) due to PC-98's graphics capabilities that benefitted static graphics. (No PC-98 mention/discussion is complete if eroge is not brought up at one point anyway.)
The X68k was an absolutely amazing line of machines. Beautiful industrial design in and out.
The FM Towns relied on Intel CPUs, but the rest of the hardware was designed more or less like any other Japanese system of the era with specialized graphics and sound hardware. I believe it also was capable of booting off of CDs and running the software directly without having to install anything to an internal hard drive.
The specifications of the graphics and sound hardware were bleeding edge for sprite-based hardware of the time and included some really incredible graphics modes that far exceeded normal VGA plus had some really arcade level sprite hardware which the IBM-PC graphics hardware never offered.
The audio hardware was likewise well in excess of contemporary PC soundcards. It also mirrored arcade level hardware in capability.
In some ways, the FM Towns was as much a general purpose Personal Arcade as it was a Computer.
>The FM Towns relied on Intel CPUs, but the rest of the hardware was designed more or less like any other Japanese system of the era with specialized graphics and sound hardware.
Even in the larger commercial computer space, Japan always liked to sorta do their own thing. Aside from a couple other companies, they were always big Itanium backers for example.
I was an analyst during that period and Japan was always something of an outlier. (Europe was to some degree as well. But less so.)
Regarding Itanium, that could be just by chance because Fujitsu is the main (or almost only) seller of supercomputers in Japan partnered with HP. Also interesting however is how they switched to it from SPARC and actually kept using SPARC longer than the west.
Taiwan had similar problems for Traditional Chinese in the early days. Their solutions were specific hardware cards in the beginning, then software-only OS compatible with IBM. It is not necessary to reinvent the wheel.
DOS/V (Not to be confused with the similarly-named MS-DOS 5.0) is what made standard PCs able to run Japanese software. It provided a software emulation of a full Japanese text mode with Kanji, just requiring VGA.
But in CJK countries, things were much harder and the entire I/O system had to be significantly more capable than what might pass for usable elsewhere. This meant larger ROMs, larger framebuffers, higher resolution displays, more complex keyboarding systems, the works. Everything was harder and more expensive for a long time. A common add-on was ROMs with Kanji (Chinese derived characters) support in the same way a person in the West might buy a new sound card or get a VGA card. Except this was just so you could use your new $1200 computer (in today's money) to write things on.
Back then, given limited memory, you also ended up with a ton of different display modes that offered different tradeoffs between color, resolution, and refresh. Because of the complex character sets, these Japanese systems tended to focus on fewer colors and higher resolution while the west focused on more colors at a lower res in the same or less memory space (any fans of mode 13h?). The first PC-98 (the 9801) shipped in 1982 with 128k of RAM and a 640x400 display with special display hardware. The equivalent IBM-PC shipped with 16KB of RAM and CGA graphics which could give you a display no higher than 640x200 with 1-bit colors but was mostly used in 320x200 with 4 (terrible) colors.
Even with similar base architectures, these formative differences meant that lots of the guts of the systems were laid out different to accommodate this -- especially in the memory maps.
By the time "conventional" PCs were able to handle the character display needs (sometime in the mid-90s), they were selling in the millions of units per anum which drove down their per unit prices.
The Japanese market was severely fractured and in a smaller addressable market. Per unit costs were higher, but the software was largely the same. Porting the same businessware to half a dozen platforms cost too much. So now the average user of the Japanese systems had a smaller library of software which was more or less a copy of what was on IBM PCs, on more expensive hardware -- market forces solved the rest.
(btw, the FM Towns, IIR, also had specialized graphics hardware to produce arcade-like graphics with tiles and sprites and so on, making it even more different)
Some of this history also informs why home computing lagged in Japan compared to the West despite having all of the other prerequisites for it to take off.
graphics
https://www.pc98.org/
memory maps
https://radioc.web.fc2.com/column/pc98bas/pc98memmap_en.htm
https://wiki.osdev.org/Memory_Map_(x86)