I think that if you use passportjs because you don't understand how to implement authn yourself, then you're no any better off from a security standpoint.
To me, passportjs might be useful if you need to plug into 3rd party auth APIs, but I don't really see the point. Authentication is a core part of your application and you should always know exactly how it works.
If you can't store an authn secret with confidence, how can you do anything with confidence?
I would disagree on the grounds that authentication is, generally speaking, a well-solved problem for the level most applications require. It's a better use of your time to use a library that's well-used and well-understood rather than rolling Yet Another (Probably Bad) Authentication Framework.
I will concede, however, that the most basic forms of authentication that I've used are so close to the metal that they're usually already built into whatever you're using to do communication.
To me, passportjs might be useful if you need to plug into 3rd party auth APIs, but I don't really see the point. Authentication is a core part of your application and you should always know exactly how it works.
If you can't store an authn secret with confidence, how can you do anything with confidence?