Yes but I'd argue that this is a case that shows the construction "JavaScript library" to be non-intersective :)
My colleague James is tall and he's a basketball player. But, in one pretty straightforward sense at least, he's not a tall basketball player.
Edit: I thought of a better analogy. Compare Lodash and uBlock. Lodash is clearly a "JavaScript library", uBlock is clearly not. Why not, given that like Lodash it's just so much JavaScript? Because uBlocks' intended users don't use it as a JavaScript library. They use it to modify the default behavior of the chrome browser to something more to their liking, not as a tool for more easily expressing themselves when writing JavaScript code. I'd argue the relation of htmx to HTML is very much like that of a Chrome/VSCode extension to Chrome/VSCode: its purpose is to change the built-in behavior of some tool (HTML) used by the user, whereas a library at bottom serves an expressive purpose.
htmx's angle is that the developer don't interact with the javascript, so it's not a javascript library. Otherwise one could say that html is a C++ library because the markup is interpreted and run by the browser (written in C++).