HN Theater @HNTheaterMonth

The video version [1] is also pretty awesome, though its code itself is a bit outdated now. Explains a lot of very practical issues that you might not find in most academic textbooks, but you encounter every day in practice.

[1] https://www.youtube.com/watch?v=vq2nnJ4g6N0

For those interested in a deeper dive to just deep learning, "Tensorflow and deep learning - without a PhD" is really good, and covers a lot of material in a single 2hr talk.

https://www.youtube.com/watch?v=vq2nnJ4g6N0

⬐ mlevental

the deep nets and conv nets stuff was excellent. i wish the explanation of rnns was a little better.

⬐ null0pointer

+1 for this. Well worth the 2 hours.

I watched a great video on Tensorflow (link below). It mostly introduces very basic deep learning concepts, but there are a few key moments in the 2 hour+ video, where he explains what to do if something goes wrong. It's definitely not a "science", but with enough experience in deep learning, you can intuit what's going on inside the black box, and there are best practices on what to try next.

For example, he goes through a few examples where a neural net has too many weights, or too little data or improperly connected nodes. All three result in problems, but the problems exhibit themselves in slightly different ways and with expertise you can start identifying them.

https://www.youtube.com/watch?v=vq2nnJ4g6N0

This is a good introduction focused on tensorflow. https://www.youtube.com/watch?v=vq2nnJ4g6N0 (Tensorflow and deep learning - without a PhD by Martin Görner)

The ML/DNN rabbit-hole goes deep. If the video above leaves you wanting more, http://www.deeplearningbook.org/ does a good job on drilling into more specifics for the various techniques used. The examples on the tensorflow webpage are also very good.

Amazing work; it makes using AI and Deep Learning accessible for everyone here really. If you haven't seen it check this out for an intro:

https://www.youtube.com/watch?v=vq2nnJ4g6N0

I wish AMD graphics cards were supported fully. I really think AMD should find a way to work with the Tensor Flow team on this...

⬐ amenod

I agree 100%. I'm not sure what AMD is thinking, but without support from major ML tools there is no chance of competing against NVidia in this space - and this space will grow larger and larger.

⬐ syntaxing

I totally agree with you as well. I was looking for a new graphics card and was debating between the GTX1050 or the RX480. I ended up getting the 1050 since it has CUDA and CUDANN support even though the RX480 has better specs.

⬐ cityhall

It's worth pointing out Tensorflow is basically Google's clone of Theano, including a lot of the same design decisions. They've improved some things but it's not like Google handed us the secret to fire here. It's just a good implementation of the same things a lot of people have been working on for years.

⬐ p1esk

TensorFlow is not a clone of Theano. It's based on the earlier Google's platform DistBelief, mostly known outside of Google as the engine behind 2012 Youtube cat videos paper. Like DistBelief, TensorFlow was designed from the ground up to be scalable across multiple nodes.

Theano, on the other hand, seems to be focused on the optimizations for the single machine, single GPU code. It only recently got the ability to run each function on a different GPU.

⬐ lern_too_spel

DistBelief was a CPU-only special purpose neural network system that would have been difficult to modify to support arbitrary neural architectures like theano. TensorFlow is not based on DistBelief in any meaningful way other than that they were written by mostly the same people.

⬐ andy_ppp

To be truthfully honest it doesn't matter either way or even if there is something "better" out there (if Theranos was...).

TensorFlow has already become the winner from my reading around it so I'm going to continue learning it rather than another framework until I've become fairly proficient. By which time why change?

⬐ p1esk

TensorFlow does not make AI or DL "more accessible". It's not easier to use than Theano. Both have good documentation, and both have lots of code examples/model implementations.

If you're looking for something that would make it easier for you to learn DL, you should try Keras - it's a higher level library, which can use both Theano and TF as a backend.

For Deep Learning, start with MNIST. The 1998 paper[1] describing LeNet goes into a lot more detail than more recent papers. Also there's an excellent video from Martin Gorner at Google that describes a range of neural networks for MNIST[2]. The source code used in his talk is excellent[3].

[1] http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf

[2] https://www.youtube.com/watch?v=vq2nnJ4g6N0

[3] https://github.com/martin-gorner/tensorflow-mnist-tutorial

⬐ jypepin

thanks! definitely keep this for when I'm starting to get to DL!

⬐ fnl

Starting with DL? That's like learning calculus before geometry... YMMV?

⬐ paulsutter

Convolutional nets for digit recognition are certainly easier to learn than ML generally, but I wasn't suggesting to start with Deep Learning. I was suggesting that when studying Deep Learning, to start with MNIST.

It was easier for me to start with the LeCun 1998 paper than to watch all the theorem-proving in the online courses, but that's just personal preference.

⬐ fnl

Sure, but I take it the original comment wasn't exactly by someone with some ML background. And getting to grips with log likelihoods, (cross-) entropy, linear/logistic regression, evaluation metrics, and maybe even some Bayesian statistics might be rather helpful before jumping on the DL bandwagon.

While there are far too many hardcore statisticians and academics who love their theorems more than anything, not all classes are that way. I think I'd have loved it if I could have learned ML from today's MOOCs, instead of those theorem provers and formula speakers I had to deal with (and pass real-life exams you can't repeat every 8 hrs...)

⬐ paulsutter

I don't have an ML background and I had no problem understanding the LeCun 1998 paper. Naturally, the more ML one knows the better, I'm just encouraging people to dive in and try without getting intimidated.

⬐ fnl

Anecdotally, one astonishing observation I often make is that "breakthrough" papers [1] are nearly universally among the most accessible, clear and easy to follow. From Watson and Crick on DNA in MolBio, to Backpropagation by Hinton in ML, to Cox' survival model in Statistics, the most significant advances often tend to be the "easiest" to understand (in hindsight only, naturally).

[1] http://www.nature.com/news/the-top-100-papers-1.16224