I could be wrong, but it seems that the way Secret is designed it cannot be truly anonymous. They need to be able to decrypt your secrets to show them to others, and they need to be able to associate them with your account to show them to you when you log in. This means that Secret knows who you are (to the extent that your login credentials reveal your identity) and what you've posted. If coerced or attacked there is no reason to believe that posts to Secret will in fact be kept a secret.
Well, all of your contact information is sent as a hash to the Secret servers. I'm not an expert on cryptography, but I think the idea is that if you send your information hashed, and all of your contacts hashed, and then all of your contacts send their info hashed, you only have to find out where the hashes match up and you won't actually need to know what the actual contact information is (the phone, email, whatever).
Everything sent over the wire to their apis is encrypted, but using an easily reversible fashion. The encryption key is a combination of a static salt and the user's session id. The session id is also sent to the api as an http header, so it's pretty easy to decrypt that anyway.
I'm not convinced there is any ultimate way to build a secret type app that allows for perfect secrecy. Ultimately anything build on top of the internet has a chain of IP address that have to exist in order to send message back and forth. Add a system with a server in the middle and it doesn't seem like there is any way to avoid someone (or a chain of someones) to figure out the far endpoint. If you can get at least some information leakage (cell phone connections, other IP accesses, etc) I would think you could eventually figure out the device on the other end. The only hope you have to keep the source secret is to make it too difficult or expensive to uncover so no one bothers.
It all depends on what are the powers your attackers have. If your attackers can monitor the whole web, inject packets as he wish and control every computer in the world - there probably isn't any way to offer secrecy.
On the other hand, if we're talking about an adversary with a more limited capabilities - maybe he can access only 50% of the computers in the world and that cannot crack encryption at will - Than there are papers talking about theoretically sound systems that can give you anonymity with very high reliability.
If they just store secret without anything else, that would be pretty hard to get back to the original author though. I don't know a lot about security, but if they remove the whole "linked friends" part, I would qualify it as safe.
On the other hand I guess the whole point is knowing your friends secrets, not just people secrets.
Anyway given how Secret works, if you are the only common contact between two people who receive the same secret, it is pretty easy to find the originator. It seems if would be pretty easy to create an app/website where you share you contact list and the secrets you receive, to find who the secret originator is. There is probably an opportunity here to create a "Reverse Secret" app. If this is really a secret, don't share it...