Not GP, but an idiosyncratic data model that reflects a non-reality really encumbers momentum by introducing integration friction, arguments on the “right” way forward and generally causes a lot of noise. It can also make the team look like bozos when explaining the model to outsiders.
Apologies for the noob question but how would FastAPI/Flask know that the job has been successfully completed? Would the worker have to persist ml inference results somewhere and the FastAPI server poll it periodically?
This is a very good question, with a lot of different answers depending on your use-case.
One approach is to translate the synchronous call into an async call plus polling on the webapp side. You push onto a queue, with the callback queue in the message body. But that gives you problems when you want deploy a new version of your webapp - existing connection will be disrupted and the state lost.
Since you need to deal with retries, anyway, you can move the logic into the client itself. It will get the request id on the initial response and then ask the service for results.
You see, this solution can vary wildly depending on your scalability, durability and resiliency requirements. And on your budget. Its not wild to expect the response to be big, so you might want to upload it to s3. You might use websockets, too. Technology gives you a lot of options here, of different levels of complexity and scalability ;)
Thank you for the response. Would this pattern be appropriate for something like ML inference if I’m looking for sub 200ms response time and the results objects are small? The polling interval would need to be very short, and would that become an issue?
Depending on how much time it takes to query your model. If it takes significant portion of your time budget, you could just put a reverse proxy in front of your models and make it route the traffic. I would architect your workers around homogenous traffic as it would be easier to calculate the capacity, and route the traffic, via paths, on the proxy.
In that case, you trade off safety whilst the setup is still simple. But if querying the model takes less that 100ms I would argue whether thats cpu/memory intensive at all. Remember that you still need to allocate and populate the memory, and Python doesn't free the memory back to OS.
How are they going to address the problem of connection creep? (Over time the number of people you add to your friends network increase and cause context collapse)
The linked pdf contains notes explaining HFT circa ~2011. The About section explains it's sourced from the author's blog, and was also used as lecture notes for an undergrad course. I've not read the whole thing, but at a minimum, the brainteasers contain some good nerd-sniping content.
For those wanting more recent additions, the author seems to have more updated thoughts here[1], as linked on the LinkedIn profile to which their former blog redirects.
“ Piki selects music from a database of roughly 5 million songs and incentivizes users by giving them $1 for every 25 songs they rate. The Piki interface plays a song, and then gives the listener the ability to rate it after different amounts of time. Specifically, the user can “dislike” the song after 3 seconds, “like” the song after 6 seconds and “superlike” it after 12 seconds.”
These people are training on a completely different set of features than what Spotify is using. For example, listen time is likely to be a important feature where a short time or a skip will be a pretty good proxy for a dislike.
In other words, the authors conclusions are conditioned on only using the features outlined in the article, like, dislike, and super like. And I’m not sure it will generalize for Spotify.
Cuz when u have as much market power as Apple, you don’t really have a choice in granting someone market access to your platform. A refusal is potential grounds for a antitrust case. Assuming ofcourse, epic stays within the lines of apple’s other rules which hasn’t been challenged in court.
Another way to put this is: nobody working in this field has control over what they are building, and can't make any promises about how it will behave.
It's like launching a rocket ship that you can't test in a physical simulation first. Something is going to go wrong that can't be predicted because we lack the understanding, but this is seen as an acceptable risk.
Which is inherent to the problem. The domain is the space of _all possible natural images_, it’s so big, it’s ridiculous. Fundamentally, these techniques do not analyze images like humans do, but are rather trained to pick out any salient signal it can latch on to. That seems to be “primates are mostly like humans but darker”, which is superficially true but a pretty weak definition as it includes dark skinned humans.
Primates are not darker than humans. Primates is the Order, they came in all colors, including brilliant blue, white with black stripes, orange, yellow or red.
We live in a world when everybody is offended by nothing. The problem is that nobody should be offended by being called a Gorilla. In the same way as nobody is offended by being called an eagle or a wolf. Is a wonderful animal, smart, strong, protective and gentle. What if some idiots used the term pejoratively five generations ago? We know better. Societies can change.
If white people is not being classified as primates, the algorithm should be corrected so they are. Not fixed excluding black people from humanity.
People should be educated also to understand that an AI algorithm is returning probability, not truth
It's probably because humans are primates – but the AI systems often have to treat “human” as a completely separate category as “primate”, so they have to draw weird, complex boundaries around “primate” (actually “all non-human primates”). When the “primate” classification is stronger than the “human” classification, the system says “primate” rather than “human”, and if it's predominantly been trained on “pictures of white Americans are not pictures of primates”, its “primate” definition might not be skewed to miss everyone else.
I expect you'd get better results if you allowed the system to call humans “primates”, then accept “human primate” as “human” when parsing the output. (That is, leave the “is_primate” output line floating while training on pictures of humans.) I don't know whether that would work, though.
These models typically don’t have hierarchical labels like that, and they apply a softmax to their output - which means /one/ label will be considered correct. (A softmax means taking the exp of your predicted scores, then divide by the sum.)
I know – but there's no technical reason they shouldn't have more complex relationships between labels (assuming you can train that, which I don't know). If we can't get better training data, at least trying to fix the problem at the algorithms (instead of slapping crude filters on the end of them) would be nice.
I agree in spirit but disagree in practice, I think. Like we said previously, the domain is humongous so even establishing meaningful relationships between labels and sublabels is extremely difficult. Many cases are likely ambiguous too, our human understanding isn’t actually hierarchical - it’s much more elusive. It’s a square peg round hole type problem really, humans don’t really think in terms of labels in the first place, we mostly use them for the purposes of language.
nobody working in this field has control over what they are building, and can't make any promises about how it will behave. If you are working on machine learning as an employee of Facebook, Google, etc. shouldn't you assume the worst possible outcome from your efforts? I often wonder how people manage this cognitive dissonance.
A large and complex task is well suited for the talented people behind Facebook. But even a list of basic searches that exercised the engine would have caught this. No excuses imo, particularly since it happened to Google[0] not that long ago. Was nobody paying attention?
