Hacker News Comments on
Intro to TensorFlow for Deep Learning
Hacker News Stories and CommentsAll the comments and stories posted to Hacker News that reference this url.
This appears to be a rehost of the Udacity course Intro to Tensorflow for Deep Learning.
⬐ mcleonardThanks for linking to the actual course! (I'm the lead for Udacity's ML/DL content)
> That’s not really how any of this works—you can’t simply throw a bunch of unrelated data at some algorithms and expect usable output.
That is never what I claimed. First, note that I took a pretty (IMO) balanced view and indicated that this is still a hard setting. Second, note that I did indicate that sufficient training (i.e. labeled) data would be required.
This is what was possible in 2016: https://www.theverge.com/2016/2/25/11112594/google-new-deep-...: "The new deep-learning program churns through millions of photos to determine the best match."
Also see project of a fast.ai participant: "Which of the 110 countries a satellite image belongs to?" (point 13 here: https://forums.fast.ai/t/deep-learning-lesson-2-notes/28772)
> (There’s also no such thing as “deep learning”.)
> Yes, Google does have a lot of images of various locations from a top-down perspective, but that isn’t helpful for accurately determining a location from the images that Europol collects. You might be able to narrow it down to a probably country based colors and design patterns, but that’s hardly sufficient and not solid enough evidence to actually do anything.
Maybe not completely, but again: being able to narrow it down would already be an incredible help, especially for outdoor pictures (which were also shown in the article's video). I never claimed that a model would completely replace the human process.
Also, I find the downvotes (not saying you) on my initial comment to be in pretty bad form. I'm not Jeremy Howard or Andrew Ng, but don't think I was blowing smoke, and work in the area of data science and ML.
1. Udacity: Machine Learning
2. Deep Learning Summer School Montreal 2016 https://sites.google.com/site/deeplearningsummerschool2016/h...
2. selfdrivingcars.mit.edu + youtube playlist "MIT 6.S094: Deep Learning for Self-Driving Cars" (https://youtu.be/1L0TKZQcUtA?list=PLrAXtmErZgOeiKm4sgNOknGvN...)
3. Coursera: Machine Learning with Andrew Ng
4. Standford Cs231n (https://www.youtube.com/watch?v=g-PvXUjD6qg&list=PLlJy-eBtNF...)
5. Deep Learning School 2016 (https://www.youtube.com/playlist?list=PLrAXtmErZgOfMuxkACrYn...)
6. Udacity: Deep Learning (https://www.udacity.com/course/deep-learning--ud730)
I created a blog (http://ai.bskog.com) to have as a notepad and study backlog. There I keep track of what free courses I am currently taking and which one I will take next.
Although video courses are good. Everyday life makes it sometimes difficult to listen to videos on youtube while for instance doing chores around the house or working out, because you often need to a. see the slides/code examples, and b. put it into practice right away... therefore, podcasts are good to give you a flow of information.
Linear Digression, Data skeptic and (thanks to this thread i now discovered Machine Learning Guide)
Don't be discouraged if there is stuff you do not understand or feel like: i can never remember these terms or that algorithm. Just be immersed in the information and stuff will fall into place. And later when you hear about that thing again it will make more sense. I tend to use a breadth first approach to learning, where i get exposed to everything before digging into details thus getting an overview of what i need to learn and where to start.
I've enjoyed the Deep Learning Nanodegree . It is a 6 month survey course of deep learning techniques and applications. Some of the lessons and instructors are better than others but overall the content and projects have been good.
This course is not to be confused with the one-off deep learning course they offer , which I have not taken but have heard is not as good.
⬐ NoneNone⬐ trevynI signed up for this, but didn't realize that they pull access to course material if you don't complete on their timeline. I asked for a refund because of this and they refused.⬐ js09so you thought of quitting a course even though they gave us 6 months to finish it?
their approach is good to ensure people follow deadlines just like you would have in a school/college setting because that causes maxomim graduation rates
FWIW, Udacity has had a Deep Learning with Tensorflow course for a while. (Note that I'm somewhat ambivalent about Udacity - a lot of it is copy-and-paste stuff, though it does help to get you started.)
⬐ king_magicIMO the Udacity course is very poorly authored and taught. I would not recommend it to someone who has a primary goal of learning TensorFlow unless you have a pretty deep understanding of deep learning to begin with.⬐ dnauticsit's very poorly organized. I'd say you get maximal utility out of it if you're like me and for some things (I don't always do this) have to pay for the privilege of being pressured to do something to completion, and are reasonably confident you can figure things out yourself (good maths background, minimally "modestly capable programmer", maybe have tinkered with ML).
Hopefully future iterations will be better.⬐ rallycarreI have taken a 'certification' and a 'class' with Udacity and I wouldn't recommend it to anyone. The courses are shallow and do not challenge the student and the certifications hold no significance.
I prefer EduX courses as they are more like 'classes' and I feel are slightly more respectable to put on a resume.⬐ jonperlThey have a nanodegree class that is much better. I have been taking it the past two months and I highly recommend it https://www.udacity.com/course/deep-learning-nanodegree-foun...⬐ jjuelI have watched a lot of Siraj's YouTube videos. He is a very entertaining speaker, but never felt like I actually learned a ton. Is the nanodegree on Udacity better?⬐ king_magicWhat areas have they improved in?
I was really disappointed by multiple Udacity courses; not saying they can't pull it off, but I've been burned enough by Udacity that I wouldn't consider paying for any courses from them at this point.⬐ FabHKYeah, I took most of the "AI for self driving cars" course taught by Sebastian Thrun himself, and was torn.
He's no doubt an extraordinarily competent researcher in his field, but he was clearly a beginner to Python (either that or he didn't care), and the Python code (while bringing the concepts across) was so poor as to be distracting.
Udacity / Google course by Vanhoucke I think is the most popular for learning TensorFlow proper - https://www.udacity.com/course/deep-learning--ud730
⬐ martin_gornerDefinitely recommended. It's the best course available in a 3 month format.⬐ RockyMcNutsthey say 3 months? I would have thought each of the 4 modules was 1-2 full days if you want to blast through it. Maybe 8 weeks of Sundays if you do it that way. But you can definitely spend a lot of time on them and on the TensorFlow docs.
LAFF or the Andrew Ng Machine Learning courses are true semester courses, but I'm not actually sure this is.
I think it's one of these: https://www.udacity.com/course/deep-learning--ud730 https://www.youtube.com/playlist?list=PLbF0BXX_6CPJ4TgDuCDIN...
This might be the closest thing there is to it: https://www.udacity.com/course/deep-learning--ud730
A bit off-topic, but does anyone know how good this course is?
⬐ findjashuaFelt very rushed for a beginner, okay if you have some background.
Personally, I found Stanford dl courses (image classification, nlp) to be much more suitable for beginners.⬐ papaveri was quite put off by it. i feel like the teaching technique is pretty poor and the focus in on all the wrong things. mainly the tech gets in the way for learning. i don't want to figure out how to learn numpy when i'm trying to learn how to understand deep learning, that in itself is hard enough. i quite after a week (i did the stanford course first and this was going to be my second).
i would recommend the coursera course by andrew ng. i had an amazing time. the code stays out of your way and he walks you through the algorithms and explains the theory very well.
i just started the fast.ai by jeremy howard, and literally have been blown away but the course. it is AMAZING! by lesson 3 i'm able to build cnn models and score on top 20% in kaggle competitions. not bad for a complete novice. HIGHLY RECOMMENDED.
once im done with the fast.ai course i may look back around to google's deep learning course. i think it may be easier for more experienced users to digest its info.
Edit: added fast.ai link
Geofrey Hinton (one of the pioneers of deep learning) offers a machine learning course on Coursera: https://www.coursera.org/learn/neural-networks
And Udacity offers a course on deep learning: https://www.udacity.com/course/deep-learning--ud730
If you specifically want to learn about tensorflow then you can enrol for this course by Google https://www.udacity.com/course/deep-learning--ud730
Hey, I signed up just to reply! If you want to begin to understand the stuff for fun I recommend the Udacity cluster of data science courses - intro to data science, machine learning, and eventually deep neural networks. I took AI at Stanford and learned a TON but for getting your hands dirty fast the Udacity courses are more appropriate. They have a bunch and they are well segmented so if you want to divert into e.g. data visualization, there's a class for that.
Your learning will culminate in this course: https://www.udacity.com/course/deep-learning--ud730
How about this one?
There's also the Geoffrey Hinton class on Coursera, although I'm not sure if additional sections of it are being offered per-se. But you can still enroll in it and watch the videos and stuff. I don't know if it's any more recent than ang's class, but it goes into more detail in some areas and covers slightly different topics. At worst, it's a good complement to the other ang class.
I really like their high-level tutorials as in here: https://www.tensorflow.org/versions/0.6.0/tutorials/ (with descriptions and general intro to neural networks, not only - a specific library).
There are also Jupyter Notebooks: https://github.com/tensorflow/tensorflow/tree/master/tensorf... (used as exercises in the Udacity course: https://www.udacity.com/course/deep-learning--ud730).
There is another one from google. You could try with the link in bellow
⬐ mikebotazzoHi mamuninfo, I'm looking for the NIPS video,not another tutorial for deep learning. Anyway, thanks for the reference.
Check out this course: https://www.udacity.com/course/deep-learning--ud730
⬐ dyaroslaThank you for the link! Out of curiosity, have you personally gone through it or just heard good things?
⬐ imhIf you want more than a 4 lecture course, I recommend Nando de Freitas's course. It's very high quality and free.⬐ cbgb⬐ staredTo be clear, the posted course is not a survey course in machine learning. It is instead a more practical course on using TensorFlow to build deep neural network architectures useful for certain tasks.
The link the OP posted is a (great) survey course dedicated to machine learning as a whole, which includes methods other than deep learning.When it comes to the course itself (I've just started it) it looks nice, but the (initial) questions tend to be vague.
E.g. in the first question with code I had to reverse-engineer what they mean (including passing values in a format, which I consider non-standard (transpose!)). The first open-ended questions were entirely "ahh, you meant this aspect of the question".
Otherwise, the course (the general level, pace, overview) seems nice.
The IPython Notebook tasks (i.e. the core exercises) are nice.⬐ ganeshkrishnanI think intro to machine learning https://www.udacity.com/courses/ud120 is the prerequisite to this course⬐ jokoon⬐ mauritsI was just beginning to give it a try, it just requires you to type the code that is shown on video. Poor way of teaching something, it seems at first. I sense this course is just to teach me the tools of the trade, not really enabling students to fully understand what they're doing.
On the other hand, some months ago I watched the ML course by Andrew Ng, and I still did not understand how to test a simple linear regression for myself, so I did not really understand it, and stopped watching the course.⬐ rahimnathwaniYou might like the book 'data science from scratch' by Joel Grus. He uses Python without ML libraries and explains clearly what's going on.⬐ wrsh07From the yc reading list 2015 [http://themacro.com/articles/2015/12/yc-2015-reading-list/], they recommend [for Neural Networks] this book: http://neuralnetworksanddeeplearning.com/
It's more about understanding than "learning tools."⬐ pjscottThat's an exceptionally clear, well-written book, and I recommend it without reservation. And it's free online, so anybody who's curious can just check it out.⬐ pconnerNN&DL is great for neural nets specifically, it doesn't really cover other branches of ML. That being said, I found it very easy to understand with no prior ML knowledge, all you need is some calculus and linear algebra experience.⬐ thecrazyoneCould you share a link to the YC reading list 2015 . I'm surprised no one asked.⬐ datawranglerI really liked the NN&DL course. Another I've been meaning to check out is this book: http://www.deeplearningbook.orgFor people interested, Stanford has an excellent online course on deep-learning with an emphasis on convolutional networks. 
It comes with video, notes, all the math, cool ipython notebooks and will let you implement a deepish network from scratch. That includes doing backprop through the svn, softmax, max-pool, conv and ReLU layers.
After that you should be more than capable to build a 'real' net using your favourite lib (Tensorflow, theano etc).⬐ staredWhile TensorFlow may be not yet as mature as Theano or Torch, I love their tutorial: https://www.tensorflow.org/versions/master/tutorials/. It's clean, concrete, and more general than introduction to their API. (Before I couldn't find anything comparable in Theano or Torch.)
In any case, I regret waiting so long for learning deep learning. (I thought that I needed to have many years of CUDA/C++ knowledge (I have none); but in fact, what I need to to know the chain rule, convolutions etc - things I've learnt long time ago.)⬐ DrNukeYes! Andrew Ng's coursera + kaggle.com + this deep learning course by Google is a very nice -and free- foundation.⬐ magicmuHow accessible is a course like this with no prior knowledge of linear algebra? I know it's listed in the pre-reqs, but with a good head for math and lots of calc, is it something that could be picked up along the way? I'm normally pretty bold about stuff like that, but I know it's a core part of deep learning / ML. If it's really necessary, if anyone has any resources for linear algebra run-throughs it would be greatly appreciated!!⬐ pdm55⬐ cpal90Simple explanation of the basics:
Linear Algebra using Python:
https://www.coursera.org/course/matrix - Coursera: Coding the Matrix
http://codingthematrix.com/ - Website: Coding the Matrix
The simplest software for linear algebra would be GeoGebra, http://www.geogebra.org/. For instance, to enter a matrix just have the spreadsheet view open, enter the numbers, highlight the cells, then choose the option "Create Matrix". To enter a vector start writing "vec" in the input bar at the bottom and intellisense gives you the option to choose "Vector[<Start Point>, <End Point>]". Choose this. Fill it in, for example, "Vector[(-3, 4), (1, 2)]" (Hint: Use Tab to move between options in the input formula, here to move between "<Start Point>" and "<End Point>]".) Voila the vector is drawn! You can even draw a vector with just two clicks in Graphics view, if you first select the "Vectors" tab at the top (the symbol is a line with an arrowhead). The GeoGebra software is really incredible for learning/doing Linear Algebra, Calculus and Statistics. A real godsend.⬐ staredLinear algebra is simple (but ubiquitous - so you need to feel it, not only "sort of follow it").
Again, it is simple, so maybe you can even take course and look up Wikipedia when needed (but for me it is hard to guess you level, current knowledge, etc).
In any case, this ML course assumes some ML knowledge.⬐ magicmu⬐ mrborgenYeah, your last suggestion was what I was leaning towards. Because of the ubiquity of linear algebra, though, this seems like as good a time as any to start getting a good handle on it. Thanks for the resources!⬐ vowellessI like this writeup: http://betterexplained.com/articles/linear-algebra-guide/Haven taken this course (though I will), but check out Andrew Ng's Coursera's ML course. He gives a crash course in linear algebra. It helped me a lot in my endeavours in ML & neural nets. I also like to learn as I go along, and this intro was just about enough to get started.⬐ aswanson⬐ jskonhovdNg is an awesome lecturer, very easy to follow.⬐ pramodliv1I did not find the crash course in Linear Algebra to be useful. I had to complete Strang's course (ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/) before doing Andrew Ng's course , after which most of the course seemed easy.I would start with the nano degree program if you don't have any prior knowledge of ML.
Udacity has a Linear Algebra review course, but I don't believe it is public for now. I had taken a linear algebra course before I took the GT ML class, but I wasn't a expert by any means. I don't believe you will need a deep understanding of linear algebra before taking this class. Singular value decomposition might come up. I think if you are familiar with everything in the following pdf you should be fine.
If you are motivated, you will do fine. Good luck!⬐ magicmuAwesome, thanks for the pdf!⬐ datawranglerUdacity has a couple free linear algebra courses.
And another here: https://www.udacity.com/course/intro-algebra-review--ma004Hi guys one question sorry if it's answered somewhere but why does the title say "Free" course? is it free cause of the trial period or the whole course is free as of now?
If the whole course is free are there more free courses on this site?
Thanks for the reply.⬐ cowsandmilk⬐ alok-gI believe all the courses on udacity are free. They make money through offering nanodegrees, partnering with companies, and a partnership with Georgia Tech to offer an online master's degree in computer science.
So free to take, money if you want degrees.⬐ cpal90Great thanks for the reply!Will the projects/assignments be workable on Windows, or would I need Linux et al for these?
And if not natively (using docker/VMs), would they be able to use NVidia CUDA card on my system? And how much disk space would be needed.
Thanks.⬐ wodenokotoDoes this use tensor flow?⬐ isoos⬐ ntnlabsFrom the site: "Complete learning systems in TensorFlow will be introduced via projects and assignments."⬐ wodenokoto⬐ ganeshkrishnanI must have gone blind, thank you :)⬐ th0br0Purely video-based though, no materials at all, no transcript. That's a no-go for me.⬐ lqdc13Videos are also really short. If you don't already know how neural networks work, you won't learn it here.
This, I think, is more of a library demonstration than anything.⬐ nbouscalIt does start really basic, with a single logistic classifier. I think you'd have to be pretty motivated to learn how neural networks work from this course, but it seems possible. If you don't know any machine learning at all, then you probably wouldn't be able to.Yes it does. I wonder if it's overkill but Google has to push tensorflow to the market some how⬐ pramodliv1Why do they have to push the library. Are there any competitive advantages?⬐ EstragonSo they can spend less time onboarding new engineers.Yeap, it's dead :)⬐ it_learnsesWould it be beneficial for me as a developer to take these machine learning courses? I took a course in the uni a while back and know the general techniques, but I'm not sure how it would help me in my career unless I'm doing some cutting edge work in the field or focusing on a machine learning career, in which case wouldn't I need to be pursuing a postdoc or something in it?⬐ alexott⬐ jorgecurioIt's enough tasks where you need to have understanding of the ML algorithms/workflows/tools, and be able to implement production system that integrates them into real systems, generating value for companies. In many cases you need to have very good domain knowledge & software development skills in addition to understanding of ML. And in ML-related systems, the big part of implementation not ML itself, but a lot of supporting stuff (figure 1 from "Hidden Technical Debt in Machine Learning Systems" paper (https://papers.nips.cc/paper/5656-hidden-technical-debt-in-m...) quite useful for understanding).
I personally took several ML courses from coursera/udacity/edX, and they helped me when I decided to move to another group that works on the ML-related projects.⬐ argonautNo. Honestly, no. Do it because you think it's interesting. Very few companies do deep learning (Google, Facebook, and Microsoft come to mind - it might be useful if you work at these companies already). That number will undoubtedly grow, but the vast majority of ML/data science positions deal with stuff like linear regression, PCA, logistic regression, decision trees, random forests, maybe svms/boosting if you want to get fancy. Take an ML survey course like Andrew Ng's course to learn about these. Also basic statistics/probability.⬐ drcodeIt sort of makes you wonder if we're looking at a future where something like 5 teams composed of 1000 of the top researchers each are going to build the premier ML systems, and those teams can then solve most generalized ML tasks.
Most other programmers wouldn't be able to contribute much value in a world that worked like this.
This perhaps mirrors how the chip market works, which similarly involves a limited number of researchers involved in advanced manufacturing techniques that are highly specialized and mostly a mystery to other people in the technology field.⬐ conventionalmem⬐ eachroBut there may be a larger market to hire people who know how to use the tools they design.⬐ drcodeThe caveat is that in the long term, ML systems are generalized systems that function independently and won't necessarily always remain in the form of an "API tool" that traditional programmers will interface with.Machine learning only going to be more and more relevant in the tech industry. Eventually you're going to have to deal with some sort of data analysis, just because there is little to no barrier from data collection to data analysis.
I'm not sure that a deep learning course would be a good first course. But an intro course on linear regression and basic probability/statistics would be worth looking into.⬐ imhI'd recommend really mastering basic statistics if you aren't going to go all the way with learning data analysis. It's surprisingly subtle and more widely applicable to a broad range of careers.⬐ goffley3If you can make the time than learning new things and taking courses is always a good idea. You never know where you're going to end up as a developer. Who knows you may end up changing the course of your career. Also machine learning and AI are all becoming big fields.⬐ it_learnses⬐ staredTBH, I like machine learning in terms of its applications, but I have no desire to go into the field in order to do research, or deal with statistics, etc. I would rather just use it as part of my software that I am building. To that extent, how helpful is it for me to take these deep learning type courses?Likely. Data analysis (of which ML is an important part) is needed in many places, from entry-level to top-level.
I am a data science freelancer and I mostly do projects for IT-dominated companies. First, I was surprised that such companies need some external help with relatively simple tasks; only later I discovered that top-notch performance in webdev (or even: algorithms) does not mean that someone is able to do simplest data analysis.
For data science / ML - I know a lot of openings in which they are looking for "data scientists", but what they mean is software engineers with at least a slight idea what is data analysis.
When it comes to deep learning in particular - I don't know.⬐ zo1>"For data science / ML - I know a lot of openings in which they are looking for "data scientists", but what they mean is software engineers with at least a slight idea what is data analysis."
I've been wanting to get into this field recently. Do you have more info about these openings, perhaps?⬐ staredNow I don't track offers (I get contracts through recommendations/networking), so I may be not up-to date. My background is different (PhD in quantum physics), so for me stats/data/ML is simple, but software architecture, algorithms - not as much.
When 3 years ago I was looking for data science internships, most of interview were strictly in software engineering. (I got into a more data-analysis oriented.) Even when I applied to Google a year ago (and failed), all non-trivial questions where in software engineering (some with data-oriented paradigms, tough).
Look at https://medium.com/@rchang/my-two-year-journey-as-a-data-sci... - the taxonomy of "Type A Data Scientist" vs "Type B Data Scientist" is useful. You want to apply for the "B" or even - software engineer in a company which deals with data and is open to shifting roles.
Going back to the interviews: I see that the set of questions is entirely different. E.g. if the first question is "how to invert a binary table" or "how to test if a black-box number generator is fair". But sometimes it is not clear from the job opening.
If you are interested in my background: http://p.migdal.pl/2015/12/14/sci-to-data-sci.htmlI fucking love Google, it's the greatest company there is. Thank you for this free course, incredibly high quality and very enjoyable to watch.⬐ fiatjafUdacity is kinda ridiculous, making us answer some stupid questions every 5 minutes. I'm not in school anymore (by the way: no one ever learns anything in school).⬐ j2kunI'm going through the course right now, and the instructor is saying some strange things, clearly (to me) ignoring that what he's saying is only true in very specific contexts.
For example, in the video I just watched he said "the natural way to compute the distance between two vectors is using cross entropy." And then he goes on to describe some unnatural features of cross entropy. The truly "natural" way to compute distances between vectors is the Euclidean distance, or at least any measure that has the properties of a metric.
I can understand this is a crash course and there isn't time to cover nuances, but I'd much rather the instructor say things like "one common/popular way to do X is..." rather than making blanket and misleading statements. Or else how can I trust his claims about deep learning?⬐ sytelusEuclidean distance is not a good measure in higher dimension: http://stats.stackexchange.com/questions/29627/euclidean-dis...⬐ NoneNone⬐ tibbeBecause he wrote one of the most successful ones?⬐ GigabyteCoin⬐ master_yoda_1"Just trust me, I'm smarter than you are." isn't the best stance for a teacher to take.⬐ j2kunMy goal is to understand it for my own purposes, not to put on blinders and replicate his work.I think you forget to read: ================================ Prerequisites and Requirements
This is an intermediate to advanced level course. Prior to taking this course, and in addition to the prerequisites and requirements outlined for the Machine Learning Engineer Nanodegree program, you should possess the following experience and skills:
Minimum 2 years of programming experience (preferably in Python) Git and GitHub experience (assignment code is in a GitHub repo) Basic machine learning knowledge (especially supervised learning) Basic statistics knowledge (mean, variance, standard deviation, etc.) Linear algebra (vectors, matrices, etc.) Calculus (differentiation, integration, partial derivatives, etc.) See the Technology Requirements for using Udacity.⬐ staredIt's a course assuming some background in ML.
For _probability_ vectors, there is a distance measure called cross-entropy. It's a standard error measure in classification problems. It has some important properties different from the Euclidean distance (especially for low probabilities) and there is information-theoretic interpretation.
More on it:⬐ j2kunI understand it, I just disagree with the presentation. If you're sweeping complexity under the rug, say so and provide a link to further reading. I don't think cross-entropy is that common that someone in the target audience for this course would quickly and easily see the nuance.⬐ NoneNone