Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Chinese Text Analyzer https://www.chinesetextanalyser.com/

Tokenizes chinese text into "words" for learning purposes and then renders the text in a GUI where you can click on a word to get the definition. It's not perfect, but a LLM fine tuned for it will eventually result in much better "tokenization".



The same thing for Japanese: https://ichi.moe/




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: