Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
FinRL: The first open-source project for financial reinforcement learning (github.com/ai4finance-foundation)
98 points by ghgr on March 27, 2022 | hide | past | favorite | 52 comments


Quick hot take: when the project has a Venmo QR code for soliciting donations, you can safely assume they have not actually figured out how to make lots of money on the securities markets.

Which is not me looking to trash it. I'd just encourage people to be aware of the context. This is an academic project which has apparently only been tested on hindcast data. Real life trading may present challenges that were well outside their scope of research.


I'd agree with the skepticism. That said, it takes a minimum of 25k to get started using something like this in the real world due to the PDT rule (honestly more than that so you don't dip below 25k). It's possible they just aren't ready to throw that kind of capital at it yet.

Also even if they aren't using real money that does not mean they are limited to historical data. You can run algos on live data even without actively trading. What you don't get is how your actions affect the market itself.


> Also even if they aren't using real money that does not mean they are limited to historical data. You can run algos on live data even without actively trading. What you don't get is how your actions affect the market itself.

That equals to useless toy software in my book. But it's just my opinion.

Edit: Look, I don't want to trash the devs for no reason, but the amount of "finance" academic stuff that claims to produce successful strategies is just ridiculous. At some point people just get tired.


You may well be right that it's a toy (and judging from their docs it seems like it), but it's basically the first step in using an algo for trading. You always test on live market feeds first.


What's presented here is not a full blown strategy, it's just a framework for building signals (+ some of example signals) from which a strategy could be built, or to complement an existing strategy.

The signals presented are too weak to beat their cost and OF to generate money if traded, you would need at least 50 others blended together.

Its like listening to a bass player alone and thinking "I should put that guy in a live concert, he will earn millions", while the reality is you would need a guitarist, a drummer, a singer, etc.


Investopedia says that it only required when day trading on margin.

https://www.investopedia.com/terms/p/patterndaytrader.asp


Cash settlements take several days. So while that is correct, a margin account let's you use funds from the sale of a stock immediately instead of waiting for the settlement to occur. You aren't taking out any margin (with interest) in this case, just using the funds you already have (but haven't settled). Otherwise a cash account limits how much you can trade very quickly since you are waiting on trades to settle.


Equities settle T+2. Without a margin account, Reg T requires that cash be tied up until a trade is settled. It’s not viable to day trade like that.


Doesn't the exchange match people with another person who is buying / selling it? Every single cryptocurrency or video game exchange I have used has settled pretty much instantly.

How hard is decrementing the shares of one user and incrementing the shares of another.


> Doesn't the exchange match people with another person who is buying / selling it? Every single cryptocurrency or video game exchange I have used has settled pretty much instantly.

There are multiple exchanges where stocks are traded, and order routing is typically transparent when using retail brokerages. The exchanges only provide the matching engine, while brokers handle custody. The process of transferring the ownership between brokerages is the clearing step which is handled by the DTCC.

Cryptocurrency exchanges provide both the matching engine and custody, so transferring ownership can be done instantly by updating records in the exchange’s database. The clearing step occurs when transferring coins to another wallet or exchange, when the transaction actually hits the blockchain.

> How hard is decrementing the shares of one user and incrementing the shares of another.

It’s surprisingly difficult. Clearinghouses (centralized) and blockchains (decentralized) exist to solve this very problem.


You can settle as quickly as you want if you’re engaged directly with your counterparty. But yes, T+2 is wildly outdated and will likely move to T+1 in the next few years. Longer term, anything less than same-day settlement both wastes capital (of BDs who need to post collateral while their customers’ trades wait to settle) and risks contagion (since each additional day of settlement gives a party more time to go bankrupt) IMO


Don't you mean the opposite? Shorter is better since there is less risk?


Yes, my phrasing was poor. I meant to say “any reform that does not go as far as same-day would be bad”


> How hard is decrementing the shares of one user and incrementing the shares of another.

whose database are you incrementing/decrementing in? Real life stock trades happen across multiple exchanges with multiple intermediaries (brokers, market makers, end users). Without a blockchain, this is not trivial to coordinate safely


>whose database are you incrementing/decrementing in

Your own. An an exchange you own 10 total shares and in your database you store who those 10 shares belong to.


I really can't help but wonder - say I want to create a trading strategy that I actually want to use. Is there really any point in learning an open source framework, where I can assume that the most profitable strategies it can produce are already being used by someone? I think I'd rather start by learning the underlying frameworks like TensorFlow, that this one seems to wrap, and produce something that has at least a minimal chance of being somewhat original.

That being said, I have more fundamental doubts that as a private individual, with only off-the-shelf hardware, one could produce any meaningful signals with DL that actually beat the market.


> Is there really any point in learning an open source framework, where I can assume that the most profitable strategies it can produce are already being used by someone?

In practice, yes, because the methodology itself is only as good as the data you feed it.

There's only so much you can do with e.g. ML. The framework gives you that methodology. You can fiddle with it, fine tune it, etc, but that's often just marginal gain. The real juice is in the features that you prepared, and the data you have available for these features.


Hmm, but seen under this angle, it seems even worse, no? The tool lists about a dozen well-known public data sources that it's compatible to work with, you can't expect to gain any edge from those.

Sure, you could learn how to fit your proprietary data into this tool, but that goes back to my point, why not learn how to work with TensorFlow or whatever ML framework directly? They're not that difficult that you'd need a wrapper.


As someone with 10+ years of exp in the industry, I had no clue what this was actually doing after reading the github page. Had to dive through the examples and videos.

The alphas showcased seem weak: 1.5 max gross SR in sample, unresidualized, unconstrained and standalone.

That would need to be blended with a hundred un correlated others to have a decent portfolio.


What's the best way for someone doing and ML and optimisation PhD to get into quant research/Trading? I've been playing with paper trading, trying to use creative data sources to find alpha and algotrading micro amounts of Cryptos for a while now and am seriously considering doing this for my post PhD life if I can't find a nice tenure track (a complication is that I do not want to leave Switzerland until I have citizenship here, so any advice or introductions would be very appreciated!)


Get an internship at a reputable prop shop or hedge fund (that specialize in quant finance). There are plenty in Switzerland.


Aren't they going to impose a strong non-compete agreement on you, so that once you've actually learned something you are handcuffed to them?


You can't really impose strong non competes on interns. You don't show them the good stuff either. Just enough to get/give an impression for later recruiting.


As mentioned in comments, it's pretty rare to give a strong non compete to interns.

But interns aside, the whole industry is built around these competes, so mechanism are in place to circumvent them.

Some HF will happily buyback the non compete, some will offer a welcome bonus to compensate, some will be ready to wait, some will make an offside deal with the departing company, etc.


Do you have a list/advice to find them? I only found a few with offices in Switzerland, and not being in the industry I don't fully trust my judgement of what is "reputable"


What does unresidualized mean?

What do the need to do to the software to “residualize” it? If that’s even the right expression.


It means to remove a set of factors from the signal - or the returns of the signal, which would impact its performance or exposure, and are not meant to be captured by the signal.

Imagine a "I have no preference" signal which gives a weight of 1/n to each asset of the portfolio - you have 100 stocks in your basket, you give 1% to each. If you would compute its performance, you would see that this portfolio would roughly make 10% average returns over 20 years. That seems good right?

Well no it's not, because these 10% don't come from anything meaningful that your signal did. It came because you gained exposure to the market, and that exposure "carried" your signal performance.

> What do the need to do to the software to “residualize” it?

Well residualizing overall just means that you want to regress your signals on a set of factors that would carry its performance for no justified reason. In practical terms, you could start by doing a linear regression of your signal against such factors (say beta, country, sector) and only keeping the remaining residuals (alpha).


Can you recommend books on the topic to learn about this kind of stuff?


https://www.amazon.co.uk/Quantitative-Equity-Portfolio-Manag...

Caveat: it’s not exactly “easy reading”, and you might want to have the “three blue one brown” YouTube channel on standby.


Watched every video of them, I am okay with math. Thank you very much! Btw, since quant industry is full of secrets - is there any source on some actual money figures - how much firms make, what are the sums that are required to be shuffled, what are the frequencies of trades, profitability figures? Anything to get a slight glimpse into inner workings of this. Unfortunately googling turns out nothing or scams


Hedge funds are a very diverse group: from small shops managing tens of millions to the large funds managing tens of billions.

The very good ones probably make around 20% a year (meaning that they earn around $40M per year per $1B that’s under management).

HFT companies are a different story. The biggest ones make around $1B/year (e.g Virtu, Flow Traders which are public), with a few of them even bigger.

There are many tier two companies making tens - few hundred million dollars per year.

So none of them are really big (compared, to say, a big bank), but they do make a lot of money per employee, and the salaries reflect that.


Thank you!


> is there any source on some actual money figures - actual money figures - how much firms make

Yes there are a number of public sources, especially for UCITS funds that have some regulatory requirements to post publish their portfolio composition.

But you probably won't be able to make much sense of these figures, as it needs to be interpreted relative to the orientation of the fund.

See, a hedge fund is not selling "performance" per se, it's selling a reward for assuming a specific kind of risk on your behalf. Lots of people don't understand that nuance, and that's where you see all these messages like "haha my index fund outperformed hedge fund X".

Say you're a super wealthy company, idk, let's take an insurance company for the sake of the example. You have 1 billion dollar lying around and you would like to earn some passive money on it.

You can't just give that money to whatever hedge fund manager will give you the most performance, simply because you, as an investor, already have exposure to a bunch of factors. You cannot afford to have that money exposed to the same risks.

- Suppose your insured clients are mostly from Europe, it would be a bad investment to place that money on something that is exposed to the European economy. It would mean that if some bad macroeconomic factor impacts the European economy, thus making Europeans companies at risk, your dear investment would fall at the same time.

- You probably will need to be able to withdraw a part of that money at some point. Even if not, you will need to have some sort of balanced books to keep track of. Something along the lines of "the overall company reserve should cover 20% of the insured goods of clients". That means you cannot invest on whatever yields good performance, you also have to make sure the volatility of this investment won't put you at risk.

There are thousands of considerations like that, each client will have a different set. This leads to a bunch of different "orientations" of funds. Each client will typically have some kind of preferred allocation that he will balance between multiple funds accordingly.

Some examples:

- CTAs (trend followers) will provide good performance, at the expense of a strong market exposure and volatility.

- Market neutrals will provide pure alpha (no market exposure) but with lower volatility and returns.

- Arbitragers will provide very good returns, very low volatility, but with very small capacity.

- Macro funds will provide returns uncorrelated to the market, but suffer very low vol in certain circumstances.

There is no point in comparing the returns of different orientations of funds. Even comparing the returns of same kind of funds is not really relevant, to have a full picture you would need to k ow exactly what this kind of fund is supposed to deliver in terms of volatility, exposure, returns, etc.

> what are the sums that are required to be shuffled

Usually this is an output of your strategy, not something you decide a priori. Quantitative funds win "on average", so you want to trade as much as you can for the law of large numbers to kick in, until the trading costs catches up.

> what are the frequencies of trades

That completely depends on the kind of fund, and the regulatory enveloppe with which the fund is sold.

Typically that will go from a few microseconds for the best arbitragers to weekly/monthly rebalancing for large fundental funds.


Much appreciated for the information!


And thank you for asking the question that allowed for such a great answer.


It comes from the “residual” of a linear regression.

So you take some obvious variable like the market basket of stocks, do a regression, and then do your machine learning on the residuals.

The reason is that you don’t get “credit” for doing whatever the market did, you only get credit for doing even better.


Quants: So much fancy vocabulary just to consistently underperform your favorite mutual index fund :)


Do you really believe the world is that simple?

It poses you no problem to live in a world where your outsider opinion on a topic you probably just read about on the internet, is right versus a whole industry of very smart people managing hundreds of billions?

At no point do you wonder that if thousands of smart, successful people and companies do something, then maybe that something is not as absurd as some internet blog posts may suggest?


I have done research in financial modeling and indeed have a family-member that manages his own successful hedge fund, so the comment was somewhat tongue-in-cheek. I do consider myself an outsider, though, as I’ve been out of the field for sometime.

Still, if you’ve worked in hedge funds you’ll know two things: connections and luck account for almost everything. What remains are mostly linear models that almost never outperform your basic mutual index funds.

Those very smart people moving billions do get things very wrong, though often at an imbalanced risk to themselves (an imbalance in their favor, mind you), so I don’t put much stock in such sweeping appraisals.


I'm curious how efficient is RL for trading. Given that RL is extremely data-hungry I'd say that this approach is likely to result in an overfit model. Does anybody have positive experiences with this?


```Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'.``` lulz


Why lulz? Bad indicators, too few?


These are naive and unrefined technical indicators.

- there's only price/volume, it's very poor in terms of information. Smells like an academic with only access to a free database.

- it's really the classical technical analysis 101 indicators, studied 10000 times in research articles.

- you would need wayyyy more indicators to get a meaningful model.


Technical analysis (Bollinger bands and the like) would be treated similarly to horoscopes by many people who know how to work a Bloomberg terminal.


So then why does Bloomberg build in so many technical indicators into its product?

Look, I share your skepticism of technical analysis, but there are many traders who do not. And when managing traders, you might want to let them trade on TA because it can give them a reason to take action. Otherwise they can suffer from analysis paralysis.


I've never understood how someone had the nerve to trademark "Bollinger bands", when all they are is just a measure of 2 standard deviations from a rolling mean of share price. It's like renaming 1+1=2 after yourself and and trademarking it's use.


This is self defeating if you open source an approach like this and it is effective, then the laws of arbitrage says that the market will remove any additional benefits of using it.


> This is self defeating if you open source an approach like this and it is effective, then the laws of arbitrage says that the market will remove any additional benefits of using it.

This assumes perfect information. The effects of law of arbitrage are not instantaneous.


It would have to be the DOMINANT approach for that to happen.


smells like a solution in search of a problem


Based on the name, I was hoping it was a roguelike.


I might be able to use these algorithms in my automated trading system: https://tradecast.one. I'm not sure how well they perform, but something to consider.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: