HN Books @HNBooksMonth

IMO Google and Facebook have their hands remarkably clean relative to the rest of the Adtech industry.

I've heard some crazy shit about how much money comes from adtech to fund blackhat data brokers. Adtech buys hacked databases on underground markets, but more than that they fund supply chain attacks to get highly intrusive adware into popular apps. They frequently buy up applications that have a wide install base on phones and browser extensions, and then on the next update, request maximum privileges and use it to loot as much as they can from user systems.

It's a symbiotic relationship. Shady ad networks are often used by criminals for narrowly targeted attacks (advertise this crafted phishing site to women aged 25-35 in the greater Dallas Fort Worth area who are recently married). Those criminals use that access to obtain more private data which they sell to adtech companies. It's a pretty gross business.

In other news, HFT isn't bad because it's HFT, it's bad because order matching services have a bunch of shady, undocumented order types that are designed to allow HFT firms to specifically extract winnings from retail investors. They are absolutely economic parasites, and no one has any incentive to stop them.

https://www.amazon.com/Flash-Boys-Wall-Street-Revolt/dp/0393...

https://www.amazon.com/Dark-Pools-Machine-Traders-Rigging/dp...

⬐ deadmutex

> bunch of shady, undocumented order types that are designed to allow HFT firms to specifically extract winnings from retail investors

Ehn, most of those order types are to try to beat other HFTs. When talking about order types, it's HFT vs HFT.

For those that don't know, this is Josh Levine's blog. He started Island ECN and is prominently featured in Scott Patterson's Dark Pools https://www.amazon.com/Dark-Pools-Machine-Traders-Rigging/dp....

You can see the old homepage here http://josh.com/oldindex.htm

Some of his code (through acquisitions) lives on at the Nasdaq exchange. He's a fascinating person in the history of electronic markets.

⬐ RubberShoes

One of my favorite books. Highly recommend

⬐ westoncb

Anyone one else want to weigh in on that? Sounds intriguing, but Amazon reviews suggest it's ruined by technical inaccuracy.

HFT reduces spreads. End users end up benefiting from lower prices. If you wanted to make HFT less of a game, adding much higher precision would help. So instead of pricing in increments of a penny, which is a lot, try, say, 1/100,000th of a penny (8 digit precision on the price).

If we accept financial services as a whole to be OK, then HFT certainly is. I'd suggest the book "Dark Pools": https://www.amazon.com/Dark-Pools-Machine-Traders-Rigging/dp...

It describes how automatic trading started. If you're a programmer, you'll probably have your mouth hanging open realising how up-for-grabs so much money was. At least that's how it sounded on reading it.

⬐ alimw

> HFT reduces spreads. End users end up benefiting from lower prices.

Is this really true? It's hard to understand how the fact that one guy can trade with a delay of two microseconds instead of three will benefit the market as a whole. I can tell you it is commonly thought that HFT's profits come at someone else's cost, and that someone else is precisely your "end user". See "front-running".

⬐ MichaelGG

Front running is when you take your customers' orders, then use that knowledge to cheat them. For instance, your customer gives you an order to buy a stock. You then go and buy a bunch first, then sell to them at a profit.

Front running has nothing to do with speed. And since HFT firms don't have customers generally, they simply cannot front run.

The profits have come at the cost of other market makers - they're all HFT now. There's also the ability to adjust pricing quickly, so that when some large firm tries to buy or sell a large block, they don't get to avoid price impact as easily.

⬐ dredmorbius

Front-running is precisely what Michael Lewis's Flash Boys is about.

The trades intercepted by HFT don't constitute marterial market information which could change the behavior of the trader being front-run. They can only serve, much as with Sysiphus, to take an existing transaction opportunity from them, after having been committed to it.

⬐ bogomipz

By end users you mean HFT trading firms?

"adding much higher precision would help. So instead of pricing in increments of a penny, which is a lot, try, say, 1/100,000th of a penny (8 digit precision on the price)"

So start gaming the price to fix gaming the trades?

⬐ MichaelGG

I mean anyone not acting as a market maker.

It's not gaming the price. Using 1 penny increments is slightly less silly than 1/8th but makes about the same amount of sense: not much. There's no technical reason, it's purely a regulatory issue. (Well sub $1 stocks in the US are allowed 0.001 pricing.)

⬐ matt_wulfeck

Sweet the spreads are reduced. Thanks HFT! I've been up all night worrying about them.

Everyone profiting from HFT are trying desperately to convince everyone else that they provide a service and benefit and they just can't understand why normal folks aren't buying it.

The amount of power of HFT to move markets is frightening. Like, for example bogus assassination tweet sending the stock markets plunging $136 billion in two minutes*

* http://www.bloomberg.com/news/articles/2013-04-23/fake-repor...

⬐ Dwolb

People also might care if most of their retirement rests with some large insitutional investor.

So if you invest with an active mutual fund, HFT is either a) detecting any large move the fund makes and pushes up the price of the asset before the trade completes or b) making you use a more costly service to complete the trade secretively to hide it from HFT detection.

Passive funds might trade less but their moves are so widely broadcast you don't really need algorithms to detect them.

⬐ bluecalm

As an average individual investor who thinks that keeping significant part of my net worth in stocks is a good strategy I do care about spreads and I don't care about market plunging $136 billion for a day or two. Why would I? Why would anyone? The company is still there, standing, producing, paying dividends. I couldn't care less about it plunging 50% for few hours because some suckers decided to sell some stocks for half the price. Maybe I can even benefit by buying then.

HFT is a godsend for your average investor who can now save money when buying/selling stocks instead of giving it to greedy traditional brokers on their overpriced services.

⬐ twic

As an average individual investor, you're buying and holding for the long term, in which case you actually don't care about shaving razor-thin slices off spreads.

⬐ bluecalm

I do care. If I buy stocks to hold and sometimes re-balance the difference between spreads between liquid stocks (as in markets with active HFTs) and not liquid stocks (as for example in market in my country) is huge. Those things add-up.

I also want efficient arbitrage done by HFTs. I don't want to sell or buy being unaware of some important news. HFTs ensure I get a fair price without following the news every time I want to buy/sell something. The more efficient the market, the more liquidity and the lower the spreads - the better it is for me as an individual buy and holder.

⬐ barrkel

As an average Joe, I feel exactly the opposite. Spreads don't affect me much because the commission on a purchase or sale is so high and I trade infrequently. High volatility is what would make me more dyspeptic.

The book DarkPools does a decent job of describing the rise of machine based trading that eventually replaced pit traders.

http://www.amazon.com/Dark-Pools-Machine-Traders-Rigging/dp/...

Good riddance, the markets are better off in almost every way but removing pit traders.

  - end of day's reconcile, 

  -machines aren't tempted to "lose" trade tickets for trades that would have been loses.  

  - machines don't hold clients trades while they jump infront of them.

  - liquidity is much, much better off now.   Every flash crash has an almost instantaneous rebound of prices back to their "true" values.

To paraphrase Churchill. "“Computerized trade matching is the worst form of trading, except for all the others.”

To the city of Chicago's credit. Even thought they lost pit trading for futures and almost all options, they've still managed to keep the bulk of those trades via the electronic CBOE and being the center of HFT.

I would guess that when an industry gets as disrupted as pit trading was, then the center of gravity would tend to move. In this case, Chicago did well to hang onto it.

⬐ a3n

> machines don't hold clients trades while they jump infront of them.

Does machine trading actually bring anything with it that discourages that? I would think that it would make it easier to do that as a defacto business model.

⬐ serve_yay

No, it just looks different now. (It looks like spending crazy amounts of money on a wired connection to be 4ms ahead of the other guy.)

⬐ jeffreyrogers

Well, it's illegal, and presumably it's harder to cover up a machine doing it (since it's in the source code) than a human who could just deny doing anything wrong.

⬐ kjs3

Immutable activity logging.

⬐ yummyfajitas

Yes - if you put multiple queues into your software, that's pretty easy to spot and prove in court. It's a bit harder to prove if the queue lives in your broker's head.

⬐ pavel_lishin

> the markets are better off in almost every way

In which ways are they not better off?

⬐ jackgavigan

> I would guess that when an industry gets as disrupted as pit trading was, then the center of gravity would tend to move. In this case, Chicago did well to hang onto it.

Absolutely. At the beginning of 1997, the London International Financial Futures and Options Exchange (LIFFE) had 65% of the market for trading German Government Bond futures (known in the market at Bunds), its most-traded product. Eurex (the German futures exchange in Frankfurt) had the other 35% of the market. Throughout 1997, Eurex slowly grew its market share by slashing trading fees and extending access to its electronic platform to clients in the US and London. By contrast, LIFFE was slow to embrace technology, preferring to rely on the traditional, open outcry trading floor model. However, in January 1998, for the first time, Eurex captured a larger share of the Bund market than LIFFE. This marked a tipping point and by the end of 1998, virtually all Bund trading had shifted to Eurex (and stayed there).

Coincidentally, the Streetwise Professor himself wrote a paper on the topic: http://www.bauer.uh.edu/spirrong/dtbtip2.pdf

To be honest, I was a bit disappointed with this article. I'm aware of the author, I own two of his books, and he certainly knows what he's talking about.

However, there wasn't really too much actionable advice in it. I don't think I actually saw any advice as to how to become an algorithmic trader in the post.

I think I actually wrote a better answer in a previous post here:

https://news.ycombinator.com/item?id=8699260

The sad fact is that if you want to get into algorithmic trading you really have 3 choices.

1) got to MIT, get an undergrad in math/engineering, apply to TradeBot, Virtu, Getco, Jane Street.

This provably works as this is how these firms hire, sadly its not very applicable for most people.

2) Get into a small and successful fund that was previously a prop shop( traders without any algorithmic tooling and then start to build it out yourself.

This can work, but its a very hard and long slog. You'll be creating everything from scratch and wont' have alot of people to bounce technical ideas off of. This might be the hardest way to break into the industry, as you'll essentially be creating a new company inside of an existing one, but it is possible, as this is how I did it.

3) Get hired in a technical capacity with a major algorithmic trading firm and move up.

The key here is to not be in a strictly technical capacity for more than a few years. The industry has a tendency to box people into their current roles.

You have to make people aware of what your goals are. Shadow the best traders you can find. Be mentored by the technical talent who writes the strategies. Get as close to the money as possible!

I lied there is actually a secret option 4)

You can go it alone and trade your own money. I really don't recommend this to people as you need a minimum of $50,000 to $100,000 to do this well. Its hard as you won't have anyone to bounce ideas off of or to lean on when times are tough.

The biggest problem I've seen with going it on your own is that since 2009 we've been in a huge bull market. Everyone is making money. We haven't had a challenging market for 5 years so if you've been trading for less its hard to know if its you who is making money(alpha) or if its the market(beta).

I really don't try to time the market but I have a feeling that late next year people who have been trading their own strategies will start to find out what its like to trade in a bear market.

Someone privately messaged me about math. For each of these options, I find math, specifically stats, to be very important. The hard part is getting programmers to learn stats. There is an old Simpsons episode where Homer is trying to learn about marketing. He starts with a huge book on marketing reads it for a few minutes and then goes to a beginners guid to marketing before finally looking the definition of marketing up on the internet.

Don't be afraid to learn math this way. I usually recommend people read chapters 2-5 of Introduction to Statistically learning

http://www-bcf.usc.edu/~gareth/ISL/

If you are flying through it then graduate to Elements of statistical learning http://statweb.stanford.edu/~tibs/ElemStatLearn/

If you are working hard to understand Intro to statistical learning then go to Kahn Academy and spend 2 weeks doing all their stats lessons.

I feel like I'm repeating my self but there are no free lunches. You need to work to learn the material. Don't be afraid to go back to basics.

On his book recommendations, Trading and Exchanges is awesome. Michael Lewis' Flash boys is not, read Scott Patterson's DarkPools instead http://www.amazon.com/Dark-Pools-Machine-Traders-Rigging/dp/...

If you are determined to read Flash Boys then atleast read the counter argument by a HFT https://news.ycombinator.com/item?id=8577237

Its a much more enlightening read and is only a few dollars:)

⬐ zeeshanm

>> got to MIT, get an undergrad in math/engineering, apply to TradeBot, Virtu, Getco, Jane Street.

Regarding Tradebot, they are known to hire from local schools out in Kansas. In fact, Dave Cummings once said it's like half-way to India strategy. You don't have to get an MIT kid and pay them loads of money when you can get a local school kid pretty much do the same.

I used to be in this field, so I can say from my experience, at least, such jobs come out of relationship building with the men of influence. Sometimes it means being ruthless about following up until the right moment comes (e.g. they have a headcount).

But I hear you on that. Having gone to MIT may help.

⬐ None

None

⬐ None

None

⬐ busterarm

With some places (DE Shaw & similar), having gone to MIT or an Ivy is just about the only way to get your foot in the door if you're new to the industry.

Also, having a Ph.D. helps, and if somebody wants to go this route I would steer them towards Two Sigma over the competition.

⬐ jeffreyrogers

Even then it's still really hard. I'm at one of those schools and many of my friends don't even get interviews with the best firms. My impression is that the supply of capable people for these jobs is much higher than the capacity to absorb everyone who is capable of doing the job.

⬐ busterarm

Yeah, it's hard to really pin down. They're all really image-conscious and they're looking for specific things -- I don't even know what sometimes. It also really helps to meet with their recruiters that come around your campus and get feedback from them on this. They have a difficult job that they'd love to make easier.

Also keep in mind that they have an image to maintain and some of them are really hurting right now. They may have to look like they're hiring when really they aren't.

(looking over your profile) You might be better off at some of the smaller, specialized HFT shops out there (I hope you like Chicago) if you really know your way around linux kernel programming. That's a really small industry though.

⬐ jeffreyrogers

Thanks for the advice :)

⬐ nacho_weekend

I had a brief stint with a HFT company and it certainly seemed like 90%+ of all the traders were cherry picked from MIT/Harvard. I believe a few came from Notre Dame? There were definitely outliers, but the vast majority came from those two schools (from what I recall). The Computer Science part of the company was a little more diverse (Carnegie Mellon, Stanford, Caltech, Northwestern, UIUC). All top schools.

⬐ chvid

I have been interested in this subject for years.

The strategies I actually can get to work yield maybe 10% per year with 5% variance. Allowing me to take out around 5% per year.

For it to pay as a fulltime job and cover office expenses I would have to take out 150.000 euro per year; meaning I would have to have a working capital of 3 mio. euro.

So for me this cannot be anything other than a hobby project.

⬐ wglb

This is the best comment on the entire post. TFA seems to be from the point of view of someone who writes books about it rather than actually doing it. Major clue is recommending Flash Boys. And the other link you point to is also good.