I wouldn't say so, pretty common. It used to add a layer of security. You should take a look at an casino app.
Did you know that every chip on a Chip & Pin bank card is powered by a Java Virtual Machine that when you go to tap or insert in to a card reader it's activated.
I mean the only claim is no propagation, you always need a gradient of sorts to update parameters. Unless you just stumble upon the desired parameters. Even genetic algorithms effectively has gradients which are obfuscated through random projections.
No you don't. See Hebbian learning (neurons that fire together wire together). Bonus: it is one of the biologically plausible options.
Maybe you have a way of seeing it differently so that this looks like a gradient? Gradient keys my brain into a desired outcome expressed as an expectation function.
Nope that update with the rank one update is exactly the projected gradient of the reconstruction loss. That's not the way it is usually taught. So Hebbian learning was an unfortunate example.
Gradient descent is only one way of searching for a minima, so in that sense it is not necessary, for example, when one can analytically solve for the extrema of the loss. As an alternative one could do Monte
Carlo search instead of gradient descent. For a convex loss that would be less efficient of course.
The one that is not used, because it's inherently unstable?
Learning using locally accessible information is an interesting approach, but it needs to be more complex than "fire together, wire together". And then you might have propagation of information that allows to approximate gradients locally.
Is that what they're teaching now? Originally it was not used because it was believed it couldn't learn XOR (it can [just not as perceptrons were defined]).
Is there anyone in particular whose work focuses on this that you know of?
If there is a weight update, there is a gradient, and a loss objective. You might not write them down explicitly.
I can't recall exactly what the Hebbian update is, but something tells me it minimises the "reconstruction loss", and effectively learns the PCA matrix.
There is no prediction or desired output, certainly explicit. I was playing with those things in my work to try and understand how our brains cause the emergence of intelligence rather than solve some classification or related problem. What I managed to replicate was the learning of XOR by some nodes and further that multidimensional XORs up to the number of inputs could be learned.
Perhaps you can say that PCAish is the implicit objective/result but I still reject that there is any conceptual notion of what a node "should" output even if iteratively applying the learning rule leads us there.
Yes there is a weight on every connection. At least when I was at it gradients were talked about in reference to the solution space (e.g. gradient descent). The implication is that there is some notion of what is "correct"for some neutron to have output and then we bend it to our will by updating the weight. In Hebbian learning there isn't a notion of correct activation, just a calculation over the local environment.
In genetic algorithms, any gradient found would be implied by way of the fitness function and would not be something to inherently pursue. There are no free lunches like with chain rule of calculus.
GP is essentially isomorphic with beam search where the population is the beam. It is a fancy search algorithm. It is not "training" anything.
True, genetic algorithms are only implied, but those implied gradients are used in the more successful evolutionary strategies. So while they might not look like it (because it's not used in a continuous descent) they still very much work like (although they represent a smoother function than) regular back-prop gradients when aggregated.
GP glancing at the pseudo-code is certainly an efficient way to dismiss an article, but something tells me he missed the crucial sentence in the abstract:
>"We believe this work takes a first step TOWARDS introducing a new family of GRADIENT-FREE learning methods"
I.e. for the time being, authors can't convince themselves not to take advantage of efficient hw for taking gradients
(*Checks that Oxford University is not under sanctions*)
Check out feedback alignment. You provide feedback with a random static linear transformation of the loss to earlier layers, and they eventually align with the feedback matrix to enable learning.
It's certifiably insane that it works at all. And not even vaguely backprop, though if you really wanted to stretch the definition I guess you could say that the feedforward layers align to take advantage of a synthetic gradient in a way that approximates backprop.
"Years of works of the genetic algorithms community came to the conclusion that if you can compute a gradient then you should use it in a way or another.
If you go for toy experiments you can brute force the optimization. Is it efficient, hell no."
An estimate of how many satellites are needed to serve 500,000 users in the continental United States would be 500000 / (8x296) = 211 satellites when using UT-1.
With a uniform distribution of satellites over the Earth’s surface, and taking into account only the U.S. territory (2.55 percent of the Earth’s surface),the constellation should include 315 / 2.55% = 8,274 satellites.
bought a ten years old company, a division of a public company, some million dollars.
got an overly complex, over 30 micro services architecture, over usd20k in monthly cloud fees.
rewrote the thing into a monolith in 6 months. reduced development team in half, costs of servers by 80-90%, latency by over 60%
newer is not better. each micro service must be born from a real necessity out of usage stats, server stats, cost analisis. not by default following tutorials.
It’s telling that you revised both application architecture and org structure to be simpler and more efficient.
Microservices are sometimes a reflection of the org; the separation of concerns is about ensuring everyone knows who’s working on what, and enforcing that in the tech.
(Not defending that, it’s often inefficient and can be a straight jacket that constrains the product and org)
I've seen the opposite: single monolithic codebase, where the different bits of functionality eventually end up tightly coupled, so it's actually pretty difficult to extract bits into a separate service later, so a different type of architecture isn't possible even if you wanted to split it up.
Why do that? Well, when a big Excel file is uploaded to import a bunch of data, or when some reports are generated, or when a crapton of emails is being sent, or when batch processes are sending data over to another system, both the API and the UI become slow for everyone. Scale it vertically, would be the first thought - for a plethora of reasons, that doesn't work. There are bottlenecks in the DB thread pool solution, there are bottlenecks in the HTTP request processing, there are bottlenecks all over the place that can be resolved (for example, replacing HikariCP with DBCP2, oddly enough) but each fix takes a bunch of time and it's anyone's guess whether something will break. Suddenly updating the dependencies of the monolith is also a mess, something like bumping the runtime version also leads to all sorts of things breaking, sometimes at compile time, other times at runtime (which leads to out of date packages needing to be used). Definitionally, a big ball of mud.
Can you just "build better software" or "write code without bugs"? Well, yes, but no.
I've seen plenty of cases of microservices also becoming a chatty mess, but what strikes me as odd is that people don't attempt to go for something like the following:
* keep the business functionality, whatever that may be, in one central service as much as possible
* instead of chopping up the domain model, extract functionality that pertains to specific types of mechanisms or workloads (e.g. batch processing, file uploads or processing, data import etc.) into separate services, even if some of it might still use the main DB
Yet, usually it's either a mess due to shoving too many things into one codebase, or a mess due to creating too much complexity by attempting to have a service for every small set of entities in your project ("user service", "order service", ...).
> single monolithic codebase, where the different bits of functionality eventually end up tightly coupled, so it's actually pretty difficult to extract bits into a separate service later, so a different type of architecture isn't possible even if you wanted to split it up.
In my experience, this is the time to refactor the monolith, not try to introduce microservices.
> grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too
The document upload & send data to db + render UI are some of the primary functions of SharePoint. All done within the context of the same ASP.NET worker process with no UI slowdowns for everyone.
It's inherently async & multithreaded, of course.
What you're describing sounds like a single threaded sync solution, which we'd all agree will cause UI lag and/or timeout. But it doesn't have to be that way with a monolith.
> But it doesn't have to be that way with a monolith.
Tell that to some old Java Spring app running on JDK 8 (though I've also seen worse). It would be cool if I didn't see most of the software out there breaking in interesting ways, but it's also nice when you at least can either limit the fallout or scale specific parts of the system to lessen the impact of whatever doesn't behave too well, until it can be addressed properly (sometimes never).
Whether that's a modular monolith (same codebase, just modules enabled or disabled during startup based on feature flags), microservices, anything really, isn't even that relevant - as long as it's not the type of system that I've also seen plenty of, "singleton apps", that can only ever have one instance running and cannot be scaled (e.g. if you store sessions or any other long lived state in RAM, if you rely on data being present on a file system that's not mounted over the network and can't be shared with other instances etc.).
Your suggestion aligns well with how Ruby on Rails tends to handle this. All of the stuff in your list of workloads would be considered “jobs” and they get enqueued asynchronously and run at some later time. The jobs run in another process, and can even be (often are) on another server so it’s not bogging down the main app, and they can communicate their success or failure via the main database.
I run the tech org for an insurance company. We've got a team of about 30 folks working on a system that manages hundreds of thousands of policies. Apart from the API's we call (a couple of them internal, belonging to our Analytics team), it's one big monolith, and I don't see that changing anytime soon. At our scale, a monolith is more stable and performant, easier to understand, easier to deploy, easier to test, easier to modify, and easier to fix when something goes wrong.
