PolarSSL and MatrixSSL definitely seem far off the beaten path, but many projects use GnuTLS (both as one of the more well-known non-OpenSSL codebases and because it has a GPL-compatible license). I'd be interested to know if you're concerned about it in particular.
There was a GnuTLS vulnerability introduced in 2000 was discovered in 2014 due to an audit. To summarize there was a refactoring that had no accompanying test coverage that had the effect of inverting a check.
Bugs happen to everyone, but the process that led to this one is really concerning. (OpenSSL certainly has bad process too but as the GP mentions, more people are hammering on it.)
This blog post has more (including an LWN article about it):
Every security library has had vulnerabilities, and I'd be more concerned about libraries that don't (since it implies nobody is looking). Does GnuTLS seem significantly more prone to vulnerabilities than other implementations?
Another option is wolfSSL (https://wolfssl.com/wolfSSL/Home.html) which is GPL-compatible, but also has a commercial license option. They have an OpenSSL compatibility layer, but are not a derivative of OpenSSL.
My experience with their software has been very positive, and they have avoided the majority of recent insecurities. Plus they have great support for anyone working on open source projects.
