I am one of those traders, and I'll happily share my observations.
(Before you ask -- no, there are no any good books and tutorials on HFT trading. You need to grokk it on your own, or receive the knowledge by direct transmission if you are an employee of HFT firm. However, there is "The Problem of HFT" by Haim Bodek, which contains some nice essays from the insider perspective and even some basic examples, though too specific within US equity markets).
1) Bitcoin exchanges are the perfect playground to learn the basics of HFT. All trading data is free and public, most traders are not sophisticated enough, you can start as small as you like (being small is an advantage in HFT), lots of low-hanging fruit are to be picked. HFT proliferation and arms race is barely started on most exchanges.
2) The algorithms themselves are simpler than most people think. HFT trading is mostly unrelated to arbitrage (inter-exchange arbitrage is much slower, and it is a different discipline).
3) The engineering issues are _harder_ than most people think. Even more so with inter-exchange arbitrage.
4) Bitcoin markets are small, and HFT potential capacity is quite limited. Most algorithms I came up with work perfectly well on small scale (and running successfully in production), but I am not making any riches out of it yet.
One thing I don't understand about the potential for crypto HFT is how to deal with the long transaction verification times. With bitcoin, it can take a tx several mins to be verified on the blockchain. I've dabbled in bitcoin HFT for a few years, but am always stuck when it comes to this point.
There are a ton of arbitrage opportunities between exchanges, but the long transaction times are the equivalent of IEX's "speedbump." How do you deal with this?
*Note: IEX is the exchange started by Brad Katsuyama, one of the main figures in Michael Lewis' outstanding "Flash Boys."
Blockchain is used for deposits/withdrawals only. Actual trading is happening within exchanges with their own order matching engines, outside the blockchain.
I.e. blockchain operates analogously to bank transfers. They are even slower, but they are not used for trading, only for funding and settlement.
I wrote a bot but could never make money due to the transaction fees. I wanted to make 15-20 trades per day. Instead, had to limit it to 3 or 4, meaning a lot more burden on each trade to be right.
The funnest part of writing a bot is using all the historical data to observe how the bot would have done during that period. I ended up checking thousands of permutations of parameters by brute force to determine the best ones, but in the end transaction fees killed me. I didn't have the confidence to do it at a large enough scale to make money I guess.
Since the exchange functions as a market maker, do they also make money on the spread ? Or is the Ask and Bid an exact match ? Maybe there is no way to know for certain without two accounts.
Also do you trade on multiple exchanges to perform inter-exchange arbitrage or for some other reason ?
Same experience here, several years ago. I'd love to know how professionals manage to avoid problems with transaction fees. I guess it is just taken into account during prediction, but it does seem to limit the number of tentative transactions you can make, even if you cancel them.
It's actually fairly simple: basically change the metric of a trade worth taking from "one with expected profit > 0" to "one with expected profit > expected fee". It does limit the number of potential profitable trades, and hence fees are sometimes used as a way for exchanges to reduce trading volume. Or to divert it; some Chinese (non-bitcoin) exchanges for instance charge greater fees for intraday trading, to discourage speculation.
Folks, is this really deserving the downvotes it receives? Parent is merely querying on his assumptions. Please, let's try to be kinder on these types of well meant and non-hostile comments.
This is why it's important to wait and test your model against out of sample data. I.E. if you back tests up until today, give it some time and test it against what happened from now until then.
>The funnest part of writing a bot is using all the historical data to observe how the bot would have done during that period. I ended up checking thousands of permutations of parameters by brute force to determine the best ones, but in the end transaction fees killed me.
That's a sure way to find a bad strategy. You end up doing the equivalent of overfitting. If you look at enough variables, the chance of there not being spurious correlations is basically zero, so you're guaranteed to land on sets of parameters that look good with historical data but fail going forward.
The writing does not exclude proper backtesting. A more favorable reading would have him/her use a holdout or test set that is in the future of the historical data used for training. Then there would be no problem with that approach in regards to overfit/poor generalization.
There is still the problem of the market not reacting to his trades. It would be very easy for me to show a trade of $1m making money, but very hard to execute such a trade.
Bitcoin Auto Trader. Not HFT I've done inter-exchange/inter currency arbitrage technically, most platforms take on the order of hours to extract so it isn't remotely HFT.
1. Private Wallets: Most your coin should be in wallets you control. Your profit margin should be transferred immediately to a cold wallet that your bot doesn't have keys too.
2. Only move coin to an exchange when you for a transaction, and immediately extract it once the exchange is complete. Yes this forces your swaps to have a fairly non-trivial margin.
3. My API tool scrapes several exchanges. But every 24 hours I have to manually input which exchanges to actively consider swaps between. The goal of this is to avoid exchange bad news.
Even with all these safeguards I managed to lose ~$400 when Mt Gox shutdown.
Now that I'm employed I mostly do day/casual trading on NASDAQ tech stocks based on personal tech news/market timing [1]. I went from ~110% yearly gain to ~380% yearly gain. It requires 1/100th the effort and leaves me time for FOSS projects.
[1] Yes I'm aware this is generally considered the worst strategy ever.
I make maybe 1 market movement per quarter (divest/invest). Most my trading is re-enforcing the positions I already have monthly (to avoid per trade costs).
Most my investments boil down to
1. Mid-quarter purchasing if a silicon good ran into shortage issues (not triggered by supply chain but demand). Not dased on official outlets but on local stores/discussion boards.
2. Post announcement hype deflation bidding on products I feel will succeed, or will succeed for technical reasons.
3. Very high risk long duration options on things I dislike or companies losing a sense of direction/product.
4. Connecting the dots on inter-coprate deals. X company buys fab plant from Y company.. 2 years later X company announces data center with Z feature.. 1 year later Y company announces Z feature, no other company does.
5. Watch OSS commits, know the products/markets/technical details to the degree of people working on the products.
6. If its in a magazine/blog you already missed the market.
7. The only internet companies worth investing in are monopolies.
I was doing this on the now defunct Cryptsy with the popular alt coins. Using relatively basic exponential moving average calculations. There just really wasn't enough volume to be doing it constantly. I was lucky to get a few trades in a day that made sense. I made money, and lost money. Basically breaking even. Mostly just a learning exercise. At this point though, all of the big players are getting their hands in. Not really worth it anymore.
Are you using some sort of leverage (haven't heard about leverages for btc though)? Looking at btc price movements, it seems that you need either 1) very high bankroll or 2) loads of transactions to make it work, as minute/hour price movements are rather low, and even lower for smaller time ranges. (maybe that's a lame question, I have no trading experience, just curious and reading a bit about it).
There is leveraged trading, but using the leverage exposes you to directional risk, and in HFT you want to have as little directional risk as possible :)
On OKCoin, my flagship algo generates more than 4000 trades per day (and many more order placements). However, OKCoin is glitchy.
4000/day? I'm struggling to reach half of this :-) Can you share if your infrastructure is yours or are you using any sort of cloud setup? Can you share a bit about it?
Most of the exchanges (including OKCoin) are using cloud infrastructure. I place my instances in the same cloud and region (meaning, the same datacenter).
Nothing too fancy, I rent usual EC2 (or analogous on other cloud) "medium"-sized instances, with 4-8G RAM. All my code is written with Scala/Akka, so it is packed into a single JAR file.
(If you reach 2000 trades per day, even on OKCoin, it usually means that you are well into HFT territory, at it is generally impossible to stay profitable any other way. I might not know something about other options, though.)
1/ How do you think you or others can build an advantage if everyone is playing almost the same game? If I understand well, the arbitrage opportunities you are seeing, are seen by almost anyone.
2/ What tools/APIs/etc do you think are interesting to build? Since I am a cofounder of a softwae development company in the cryptocurrency space, I am looking for new insights but not have experience in the HFT space.
OKcoin endless errors in documentation and lack of sane reporting, using multiple channels for market data flow (e.g. Websocket for incoming book updates and FIX or HTTP for outgoing orders), trying to account for many many possible failure modes.
The average performance is currently +0.3%/day. Losing days are rare (about 1 in 2 weeks, with about the same average loss). I have survived multiple crashes and liquidity flight events, including the Bitfinex mishap.
Is it HFT or algorithmic trading? In my view HFT is a based on arbitrage between exchanges. If so my experience is negative. The price fluctuations are too small to compensate for fees (0.5%) and most of the exchanges have no arbitrage within say 250ms interval.
HFT is quite rarely about arbitrage, in fact (at least not directly).
I don't do arbitrage (there are easier ways that I hadn't fully exhausted yet), but between e.g. OKCoin and GDAX 30-second delays are common every day.
The best way of dealing with the fees is not paying the fees. ;)
My tax situation is quite complicated, as I am a resident of multiple countries. However, usually taxes are paid on some periodic basis on capital gains, no need to include all trades in the report. :)
(Before you ask -- no, there are no any good books and tutorials on HFT trading. You need to grokk it on your own, or receive the knowledge by direct transmission if you are an employee of HFT firm. However, there is "The Problem of HFT" by Haim Bodek, which contains some nice essays from the insider perspective and even some basic examples, though too specific within US equity markets).
1) Bitcoin exchanges are the perfect playground to learn the basics of HFT. All trading data is free and public, most traders are not sophisticated enough, you can start as small as you like (being small is an advantage in HFT), lots of low-hanging fruit are to be picked. HFT proliferation and arms race is barely started on most exchanges.
2) The algorithms themselves are simpler than most people think. HFT trading is mostly unrelated to arbitrage (inter-exchange arbitrage is much slower, and it is a different discipline).
3) The engineering issues are _harder_ than most people think. Even more so with inter-exchange arbitrage.
4) Bitcoin markets are small, and HFT potential capacity is quite limited. Most algorithms I came up with work perfectly well on small scale (and running successfully in production), but I am not making any riches out of it yet.