I experimented with something similar previously and it doesn’t really work. It ... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

artdigital on April 5, 2023 | parent | context | favorite | on: OpenAI Tokenizer

I experimented with something similar previously and it doesn’t really work. It usually can’t decompress it properly

kleton on April 6, 2023 [–]

The example given is obviously gpt's own generation. It might not be expected to roundtrip arbitrary text.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact