In their second publication they actually use the probabilistic unit, but computing those requires running an EM algorithm for each layer of capsules: https://openreview.net/forum?id=HJWLfGWRb
I doubt this will be a real problem because I've seen brilliant people excel in provincial universities with very little means and talent around. Internet education has gotten pretty good and probably reflects most of the experience of studying at a top university with the best researchers in the field. People might be 5% worse prepared. Nothing to be concerned about.
I still find it incredibly hard to tell whether this is overblown hype or legit scientific progress. There is no indication whatsoever that this approach scales to deep feature hierarchies and that is likely what you need to compete on hard tasks like classification on ImageNet. Given the amount of money at play (several hundred millions of dollars), writing 70 pages, making code publishable is certainly an obvious way to get the most out of the hype.
The problem is that anxiety is warranted in this case. You can’t do anything about solving the problem for everybody, but you can potentially save your own life:
- Work like a madman to get into the 1%.
- Move to a region that is self-sustaining and is little impacted by global warming.
Honest question: in the event of some kind of widespread collapse of civilization, there will be hundreds of millions of starving people. For the purpose of this question, I don't think it matters much whether its a 'fast' or 'slow' collapse.
Assuming you have some kind of self sufficient farm and a bunch of guns and other defenses, does it seem likely that you can keep what food you have and what food you can produce in the future from a large number of starving people?
It's not like hordes of people are going to come at your farm like a zombie invasion. The starving people in this scenario will be scattered, scared, and lacking the ability to share information and coordinate themselves.
You could be right...but I encourage you to think that through very carefully. Even in most worst case scenarios of climate change killing most food production, it's a slow process, taking quite a few years.
The people who might eventually come to take by force what someone has as a result of exceptional planning and execution could be the same people who have been politely asking for some extra food for years. And those hungry people have been thinking about it and talking to other hungry people about it for a good long time.
Also: the local government and pseudo-government entities might insist on redistribution. And that redistribution might be entirely legal.
> evolution of evolvability and then a seemingly unrelated subject: the evolution of robustness.
Isn't evolvability only about robustness? What other criteria would improve evolution? Mutations just happen, so the question is how well the phenotype can deal with these mutations. If it can incorporate mutations well, it can tunnel to different useful phenotypes and therefore is robust.
Agree, but it’s not not obvious to most people. Robustness itself if a murky subject: are you robust to mutation, to the environment, genetic noise, genetic background? Also if you are too robust then you can’t evolve. What we want to say with evolvability is that we’re more predisposed positive selection.
I think, the main effect of sexual reproduction is that, much like GANs and competitive self-play, it creates species-internal competition: Both sexes need to impress, which makes cheating an obvious strategy (makeup, steroids, Shakespeare quotes, LISP etc., but many such examples can be found in the animal world), and hence both sexes also need to be able to detect cheating. Some species are rather asymmetric in that regard. For example, in humans it is mainly women who attract (they masquerade as fruits [make up is likely a cross-cultural phenomenon; and, well, breasts] tapping into male food gathering circuitry); men compete in hierarchies trying to impress and women select men from the top of the hierarchy. Complex dynamics emerging from this likely lead to the immense growth of the human cortex.
Sexual reproduction basically outsources some of the selection effort to the cognitive apparatus of the species itself, thereby introducing a massive amount of additional selection signals (mainly by the much increased necessity to model other minds, namely minds of the opposite sex). Many of these signals promote traits that are useful for survival (mainly intelligence and health).
I was thinking the same thing. I think it would be interesting to explore this area more and try to model it computationally.
Your point about physical features made me think of how physical attractiveness plays into human development as well.
Research has shown that beauty in humans is defined as physical symmetry. So "novelty" in our case might be defined as someone who is really ugly - the elephant man.
So in this case, beauty wouldn't really fit into the robot walking example as it's neither fitness (moving the foot forward) or novelty. More a different type of fitness that increases the odds of reproduction.
My guess would be that symmetry is a simple heuristic measure of physical fitness. Visual attraction is basically a strong regularizer that restricts the search space to phenotypes with particular traits. Asymmetry means that the joints wear out more quickly and muscles might not coordinate optimally leading to less strength and a reduced ability to hunt and to fight predators. AFAIK it is also a quite robust predictor of all kinds of diseases because it often means that the growth signalling is out of tune throughout the system. Visual selection basically performs environmental selection more immediately and more effectively: an asymmetric person might still survive, but its offspring has a lower overall chance to survive. The teaching signals of that are much weaker.
Wait, that can’t be wrong because that is literally what DO does. It is a convex hull regularizer around the network activations using noise. That is also why dropout does not solve susceptibility to adversarial examples: It merely extends the regions that the NN generalizes to outward; but that is limited because high-dimensional spaces are counter-intuitively large and the noise required to cover a descent fraction of the “unmapped” space would completely prevent learning. AFAIK, Yarin Gal merely provides a Bayesian interpretation of the noise.
IIRC, his "Bayesian interpretation of the noise" actually shows that dropout performs approximate integration over model parameters. As he says, dropout doesn't work because of the noise but despite the noise.
Actually, GANs reach state of the art in anomaly/outlier detection and drug/molecule prediction, so there is certainly more to it than just artistic applications:
Totalitarian systems are quicker to come up with the rules (in this case for self-preservation). And when time matters, then it has obvious advantages.
Businesses can react quicker because internally they run like dictatorships. That's why I am always wary of people wanting to run government like a business.
