HN Theater @HNTheaterMonth

Neuroevolution of Augmenting Topologies is a classic genetic algorithm, as a teenager it was my first genetic algorithm I implemented after watching a Youtube video of an implementation of it - https://www.youtube.com/watch?v=qv6UVOQ0F44

This is very similar to Picbreeder [1], which allows you to evolve art in a very similar manner, with evolutionary algorithms. It runs on the same evolutionary algorithm behind MarI/O [2], called NEAT [3].

[1] http://www.picbreeder.org/

[2] https://www.youtube.com/watch?v=qv6UVOQ0F44

[3] https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...

I also remember seeing this video from SethBling from four years ago where he does something similar for Super Mario: https://www.youtube.com/watch?v=qv6UVOQ0F44

A slug doesnt need a supercomputer because a slug is working more like this thing https://www.youtube.com/watch?v=qv6UVOQ0F44

If you watch videos of AIs learning to run [1] or playing Mario [2], you see that there's definitely "try random shit until your reward function gives you positive feedback" as a method for training AIs.

The difficult part is developing the reward function. There's a lot of intelligence in our hormone-based incentive system - that we feel pain, that we feel sad, etc. Many of those emotions are pre-programmed. If we find a way to design reinforcement systems with the right goals, we can do it like mother nature did it for several billion years. But it's still a ton of computational power required.

If we look at nature as a massively parallelized computational system that defined "just live" as the reward (not because it "wants" it, but because it just emerged), it shows us how much power we would need if we would try to build a completely random process that gains intelligence.

I think your proposed method works ("just assemble random solutions and test them") because we've already seen it in nature. But it takes a lot of time and energy, regardless of our computational power. I think we're faster when we find the right boundaries and reward functions instead of randomly trying stuff.

[1]: https://youtu.be/gn4nRCC9TwQ [2]: https://youtu.be/qv6UVOQ0F44

⬐ s-shellfish

> If we look at nature as a massively parallelized computational system that defined "just live" as the reward (not because it "wants" it, but because it just emerged), it shows us how much power we would need if we would try to build a completely random process that gains intelligence.

With that definition I can't see how artificial life created would be any much different than the behavior of a computer virus.

> I think we're faster when we find the right boundaries and reward functions instead of randomly trying stuff.

Boundaries and reward functions in human society are part of the 'human social organism' that allows each individual human to function with autonomy in a fashion that is collaborative, aligned with our developed value systems, and allows us to live with stability, security, faith (be it in some sense of wonder, divinity, in each-other - doesn't matter), independence - and these things are base needs, no matter what variation they manifest with. Boundaries are redefined when there is conflict, and the less violent and destructive the conflict, the better the chances are for these boundaries and reward functions to continue functioning as they have been developed (rather than being obliterated in entirety, requiring them to built from scratch).

It makes us faster, but it's also us standing on the shoulders of giants. And I think it's important to question what things are worth applying random solutions to, and what things require deep contemplation. It seems very paradoxical, to try to define something that both is a function of the system it exists in, and also something that could potentially break the whole system. But, creation, destruction, clearly an oversimplification.

I do know that what looks random to one person is not necessarily random to another. This goes back to how the context is defined, how society is defined, boundaries and reward functions established a priori.

Emotions can be simple. The problem is that a computer already has them. Computer produces wrong solution, algorithm dies. Computer produces right solution, algorithm survives. We don't give the computer words to express itself about this because we never taught it to do that. What would happen if we did?

⬐ mockingbirdy

> try to define something that both is a function of the system it exists in, and also something that could potentially break the whole system

That is what's so damn hard about shaping societies (and the market). The recursive feedback loop and the self-interference.

It's also on individual levels: We're able to dynamically adapt our reinforcement system using meta-cognition (based in the prefrontal neocortex).

I understand the other parts of the answer, but I can't really see what you try to say with them (e.g. boundaries, society and deciding if we apply randomness or not).

⬐ s-shellfish

I tend to be of the mindset that over the long term, you are going to get a good perspective if you approach the problem 50/50. Apply randomness half the time, apply the knowledge you know the other half. Divide and conquer, sort of.

Randomness potentiates the space that you will both be able to identify error and consequently, find mistakes, find errors in reasoning (Monte Carlo simulations are traditionally used for this sort of thing). The other good thing with randomness as well, find ways to see errors as being 'not errors', e.g. a tool that can be developed, structured, a new way to see the problem, a creative approach. Some melding between the two seems to be something of significance for an AI. So you don't want it to be all random, because you need stability, structure, you need a 'language' or an 'awareness' you understand that isn't so chaotic and constantly changing that you can't even find a place to put your feet on the ground.

It doesn't have to be a perfect 50/50 balance, because that has it's own set of problems - divide and conquer all the divisions and you still wind up with 2^n newly defined problem spaces to interpret, possibly losing sight of the bigger picture or maintaining independent direction in one focused lineage of all those spaces. Just very generally, maintain balance, because the world is chaos sometimes.

It's like a stream of information. All the analysis to all of that space is meaningless if the context changes enough. So, adapt to a new context.

Honestly though, I don't know what I'm trying to say much of the time, aside, 'help', lol. Life is terrifying. :)

⬐ mockingbirdy

> Honestly though, I don't know what I'm trying to say much of the time, aside, 'help', lol. Life is terrifying. :)

from another comment made by you:

> Evolution does not care if an asteroid hits the earth and wipes out all life as it presently exists.

My advice: Stop worrying. Enjoy the randomness :) Maybe book a flight to Asia. Life is short. Embrace uncertainty. Sell luxurious sanitary pads to rich women in their 40s. Dress well and be funny. Then suddenly change your mood to sexy, ask a stranger for a kiss. I now write random love letters to my ex-girlfriends. Let's see where randomness will lead us to. See you on the other side of existence.

⬐ mockingbirdy

Found the boundaries pretty early. Wearing nothing but socks and singing "Why does it hurt when I pee?" while trying to cross a freeway is not considered "appropriate in the public". shmocks everywhere.

I'm in jail now. I'm free now. But a little cold.

⬐ s-shellfish

Meh.

⬐ None

None

OP, Togelius, is quite wrong in 1 point at least.

tl;dr Seth Bling directly references NEAT and his links[Show More [1]] link to 2 Togelius papers[13][14] among many others !

To wit, in OP, Togelius mentions Seth Bling's MarI/O[1] as lacking scholarship because he didn't mention Togelius:

" seen as research, it is atrocious because of the complete lack of scholarship. The guy didn't even know he was reinventing the wheel, and didn't care to look it up. "

Rather disdaignfully (sadly/ ironically/ poigniantly) Togelius refers to Seth not by name but only as " some guy ".

Yet Seth Bling[2] in his short video, mentions and explains NEAT and directly links Ken Stanley's and Risto Miikkulainen's foundational NEAT paper[10] at U.texas which Togelius builds on [hint: click SHOW MORE under Seth's video]

And much more, i.e.: ---- the innovative toolchain Seth uses: Bizhawk to script SNES emulators in Lua [12]!

---- Seth links the wikipedia page on Neuroevolution[11] which directly references 2 of Togelius's own papers[13][14]

Seth's [Show More] Links on the MarI/O video[1] are very very comprehensive and (IMHO) indeed a great example of scholarship !

Togelius is a top flight scholar, and publishes in the field of AI game players and level generators[15] so I hope he will welcome some random tinkerer pointing out his error.

[full disclosure: I am a Togelius fan]

To give Togelius his due he does at least link to Seth's MarI/O video.

Perhaps Togelius failed to click / read the Show More links ? The charitable benefit of doubt principle is accidental oversight.

Seth Bling is primarily a Twitch Streamer[2] and Speed Runner[3] who often popularises complex programming[4], math and academic ideas[5] to a primarily young audience using Mario and Minecraft[6] and takes requests via his subreddit[7]

Seth's very popular Youtube video MarI/O which uses NEAT (Neuro Evolution of Augmenting Topologies] to learn and play Super Mario World, has led to a huge number of streamers on twitch playing other MarI/O levels[16], using RNNs, LSTMs, and other SMW levels and SNES games, like RNN multiplayer x4 Mario Karts[17].

The OP article is great, academic rigor and bibliographic scholarship might sometimes differentiate the full time university scholar from the amateur researcher, enthusiast or tinkerer.

Seth and his many followers may not be 'scholars' as in academic research but it is an amateur community of mostly youngsters doing their own research by having fun and sharing ideas, mostly undocumented outside their own transient twitch streams.

Compare this to the well respected amateur astronomy community and perhaps a responsible academic approach is to try to reach out and tap in rather than adding to the distance.

By his own OP standards of Togelius is lacking in that hasn't discovered/ mentioned the work done by the Twitch/ Bizhawk community - it is not entirely undocumented [16].

Togelius your AI Mario competition[8] is legendary in academic AI circles but not necessarily widely known by youngsters playing Mario ( and to be fair AFAIK you don't publish in their forums like twitch) although the code Infinite Mario[9] you base your work on is by Marcus Persson[10].

Though Persson is better known as Notch [10] ( author of Minecraft ) who is more famous than perhaps even Alan Turing , Yann le Cun, Jurgen Schmidhuber, or Geoff Hinton to the under 16 crowd.

Perhaps Togelius could mashup some of his work and do a crossover episode with Seth or relaunch the Infinite Mario competition[8] / benchmarks to this new younger crowd evolving Neural Topologies to play Mario games - or even help document some of their fun.

As public servants funded by tax dollars it is perhaps the responsibility of the academic to reach out rather than blaming fun loving amateurs popularising and doing research / 'tinkering' for not putting in the due diligence of scrupulously mentioning every source.

[IMHO] I agree amateurs don't always do full and proper scholarship and links are food for hungry minds - but they aren't publicly funded researchers but amateurs working for love of the subject in their spare time, for free, or for streaming donations and youtube hits, using transitory, live media and whose audience must be constantly engaged and may have shorter attention spans than the average academic.

[full disclosure I am an amatuer neural net nerd]

[1]Seth Bling's MarI/O https://www.youtube.com/watch?v=qv6UVOQ0F44

[2]https://www.twitch.tv/sethbling

[3]https://www.youtube.com/watch?v=-spFoon7klA sub 1 min SMW via credits warp

[4]Bling's arranging shells in SMW to human inject Flappy Bird code into SNES https://www.twitch.tv/videos/57032858

[4]Bling's Atari 2600 emulator built in Minecraft https://www.youtube.com/watch?v=5nViIUfDMJg

[5]Seth Bling's RNN Mario Cart race https://www.youtube.com/watch?v=Ipi40cb_RsI

[6]Bling's Minecraft Redstone tutorials https://www.youtube.com/watch?v=DzSpuMDtyUU&list=PL2Qvl4gaBg...

[7]https://sethbling.reddit.com

[8]Togelius 2009 Infinite Mario AI competition http://julian.togelius.com/mariocompetition2009/

[9]https://web.archive.org/web/20080423023424/http://www.mojang...

[10]Stanley Miikkulainen NEAT paper http://nn.cs.utexas.edu/?stanley:ec02

[11] https://en.wikipedia.org/wiki/Neuroevolution

[12] Scripting Lua in Bizhawk Emulator http://tasvideos.org/Bizhawk/LuaFunctions.html

[13] Risi, Sebastian; Togelius, Julian (2017). "Neuroevolution in Games: State of the Art and Open Challenges" (PDF). IEEE Transactions on Computational Intelligence and AI in Games.https://arxiv.org/pdf/1410.7326v3.pdf

[14] Togelius, Julian; Schaul, Tom; Schmidhuber, Jurgen; Gomez, Faustino (2008), "Countering poisonous inputs with memetic neuroevolution" (PDF), Parallel Problem Solving from Nature https://www.academia.edu/download/30945872/poison.pdf

[15] Togelius' Game AI book http://gameaibook.org/

[16] https://clips.twitch.tv/CourteousEmpathicTruffleCeilingCat

[16] https://thenextweb.com/artificial-intelligence/2018/01/03/th...

[17] https://www.youtube.com/watch?v=S9Y_I9vY8Qw

⬐ 1wu

Thanks for the commentary togelius [1, 2] and deepnet [above].

Anyone interested in adding a soupçon of scholarship to Seth's project?

Words do mean different things to different people, in different contexts..

There is a recent body of literature that explores the modern "maker" [3] movement. However, "maker" as a term may not have been a good fit for the OP's argument [1], which contrasted (academic) researchers with so-called "tinkerers".

An alternative term for "tinkerer" might be "bricoleur", a loanword from French. (Roughly, it still means one who tinkers: https://en.wikipedia.org/wiki/Bricolage but has other meanings depending on the academic lens.)

Given that we are discussing AIs that play, in the context of education, we can also go back to Seymour Papert's work on https://en.wikipedia.org/wiki/Constructionism_(learning_theo... .

Originally known for work on _Perceptrons_ with Marvin Minsky, AI researcher Papert later adapted theories from education towards the vision of "learning-by-making" and the (young) bricoleur [4]. This can approach can be seen in the evolution from 1960s graphical [turtle] https://en.wikipedia.org/wiki/Logo_%28programming_language%2... to Lego https://en.wikipedia.org/wiki/Mindstorms_(book) to modern day efforts to encourage coding-for-kids [5,6,7].

One of Papert's later collaborators, Sherry Turkel, discusses bricolage as it applies to programming -- https://en.wikipedia.org/wiki/Bricolage#Internet .

When it comes to early education, Turkel argues for epistemological pluralism [8] and cites anthropologist Levi-Strauss in comparing analytic science with a "science of the concrete".

We can appreciate both Seth Bling's concreteness [9] and Togelius's original papers for academics. Almost a decade ago, Togelius introduced Super Mario Brothers as a benchmark for reinforcement learning [10] and, with Karakovskiy, for AI more generally [11].

deepnet what's your interest in neural nets?

[1] http://togelius.blogspot.com/2016/04/the-differences-between...

[2] https://news.ycombinator.com/item?id=16744694

[3] For example, https://scholar.google.com/scholar?&q=maker+movement

[4] http://www.papert.org/articles/SituatingConstructionism.html

[5] https://scratch.mit.edu/

[6] https://www.apple.com/swift/playgrounds/

[7] https://github.com/google/blockly-games/wiki

[8] http://www.papert.org/articles/EpistemologicalPluralism.html

[9] https://www.youtube.com/watch?v=qv6UVOQ0F44

[10] http://julian.togelius.com/Togelius2009Super.pdf

[11] http://julian.togelius.com/mariocompetition2009/

Also the video discusses previous work starting at 4:40, so I'm confused what OP is complaining about unless the MarI/O guy re-uploaded with this attribution.

https://youtu.be/qv6UVOQ0F44?t=4m40s

⬐ togelius

He mentions NEAT, which is good, but not any of the work on playing Mario, much less any work on evolving neural nets to play Mario. There's quite a few papers on the topic, not just mine.

Note that I don't say that SethBling is wrong. Seen as tinkering, this is perfectly fine, and he did a great job with the video. No hard feelings. I just use it as an example of how it is not research, because of the lack of scholarship.

⬐ frankmcsherry

Obvs he's complaining that the video didn't cite his work. :) One of the differences between researchers and tinkerers unmentioned in the article: vanity.

⬐ dyarosla

After seeing this, it completely discredits all of the 'scholarship' part of the author's post. Looks like the author should do his research first.

⬐ fwilliams

This is hardly a discussion of previous work. The author of the video states the paper he implemented.

In any semi-decent peer reviewed venue, you would cite a wide variety of papers that solve a similar or related problem or introduce a concept related to the method in the paper.

The related work section of a paper is one of the most important parts since it puts the research into context. By citing other work, the authors explain what has already been done, and what contribution their work makes.

A related work section should also illustrate the downsides, limitations, and differences of other cited research. Limitations of other works are often poorly understood since very few people have had the time to evaluate them beyond the initial experiments done before first publication.

Research is not simply about presenting new techniques but also understanding the trade-offs that arise when choosing different solutions to a problem.

⬐ traverseda

Replicating the results of existing papers is important too

⬐ lorenzhs

That is a different issue, though. The "Related Work" section is there to tell the reader about related approaches to similar problems, previous research on the same topic, etc. It's where you distinguish your work from what was previously done, and justify why this is a meaningful distinction. It's "what others have done and why we don't just use that".

Reproducibility is important but a completely different topic and I don't know why you're bringing it up here.

⬐ BlackFly

Many times the works cited occlude (as opposed to illuminate) relevant research and the context of the paper. For example, if you read a paper and find details lacking, will reading the works cited help you understand the paper or just waste your time? Many times you won't have an institute license for the journal, so what now? Request interlibrary access, piracy, call a colleague at another institute, assume the arxiv preprint is equal to the published version cited if available, give up? If you cite too broadly, the value of the citations just go down for the reader, both for an expert who is probably already aware of the other research and for the novice who doesn't have years to read the entire paper trail.

This of course comes up during the peer review process as well. The referee informs you of a paper that is tangentially relevant, but you couldn't find that during your literature search because it is paywalled. How relevant was it actually to the work you performed or to people reading your paper if it is not readily possible for you (and possibly your readers) to access the paper?

Citations in academia are often times more the currency of the trade than actual scholarship. There is a reason review articles exist, not every paper needs to be a review article in its own right.

connectionism is an approach to the philosophy of mind, the most successful approach as evidenced by recent deep learning hype

the only book in the OP I thought was even mildly relevant was the one by Dennet, and it's generally acknowledged that it's pretty bad.

A more interesting but flawed approach, that resonates as someone who loves Dawkins and Hitchens (atheists and advocates of evolution), is Gerald Edelman's Neural Darwinism (1978). There is even modern research (that is real, working code) at the University of Texas that could be considered a successor to Edelman: NeuroEvolution of Augmenting Topologies (NEAT)

https://www.youtube.com/watch?v=qv6UVOQ0F44

This is from the same guy who made the MarI/O video (https://www.youtube.com/watch?v=qv6UVOQ0F44 - using Neuroevolution). In the video he also compares MariFlow vs MarI/O.

Check out also MarI/O, very impressive: https://www.youtube.com/watch?v=qv6UVOQ0F44

This video of a Super Mario playing AI was pretty neat:

https://www.youtube.com/watch?v=qv6UVOQ0F44

It may, or may not be on a useful path to solving harder AI problems. I don't know. But it's a good way of demonstrating value (and publishing papers).

⬐ wodenokoto

The article doesn't really focus on algorithms, but rather reward functions. In this case, the evolutionary algorithm has a reward function that tells it which mutations are most useful and therefore should be used to generate new mutations.

This reward function is essentially the same no matter if you do normal reinforcement learning or evolutionary.

It's true that game AI is surprisingly under-developped compared to what you see in academia on other topics... but it is not always that basic.

What you describe in your football example is often called "Behavior Tree", and there are ways to learn those using machine learning (usually by reinforcement learning, using neural networks and/or genetic programming). There was a popular video showing this applied on Mario [1] ... and a paper published 7 years before that video doing the same thing [2] (more context about this in [3]). I remember seeing something on Gamasutra saying that similar methods are used in some AAA games.

[1] https://www.youtube.com/watch?v=qv6UVOQ0F44 [2] http://julian.togelius.com/Togelius2009Super.pdf [3] http://togelius.blogspot.co.uk/2016/04/the-differences-betwe...

You mean this script? Or machine learning in general? If the later then it has already been done: https://www.youtube.com/watch?v=qv6UVOQ0F44 (great educational video)

⬐ jeanlucas

looking at the author's github profile he did a lib to abstract the artificial intelligence and used in some other games.

Also, check out https://www.youtube.com/watch?v=qv6UVOQ0F44 for another cool ML-Mario implementation.

This is a similar project written in Lua, with a different Mario game: https://www.youtube.com/watch?v=qv6UVOQ0F44

⬐ qwertyuiop924

Referenced in the article as one of the two predecessors to this project.

If someone did this in the US, can they get in trouble?

When exactly does "hacking" a game become illegal? What about people doing it for research like the machine learning mario guy: https://www.youtube.com/watch?v=qv6UVOQ0F44. Or tool-assisted speedruns? Or the starcraft broodwar ai?

Is it only when you breach the TOS or copyrights? Just curious since it never even occurred to me that such things are punishable.

⬐ Aelius

At least in the US, I don't believe there is any precedent that establishes it is illegal to alter a product.

There is always push back by companies, but it seems that the general trend is always in favor of the consumer (see, rooting your phone is found legal, this video concerning the legality of modifying cars and tractors https://www.youtube.com/watch?v=Nps24EqiZjY )

⬐ 6nf

It might fall under the Digital Millennium Copyright Act? The DMCA criminalises the act of circumventing an access control mechanism.

⬐ wyldfire

> If someone did this in the US, can they get in trouble?

No, not really.*

> What about people doing it for research

§ 1201 has lots of exceptions including "Nonprofit library, archive and educational institution exception (section 1201(d))."

* Yes, technically the laws here in the US are broad enough that if you did something that the US DOJ or local LE became very concerned about, they could at the very least arrest, detain, and charge you with a crime if not convict you of it.

⬐ AntiRush

There are multiple cases in the United States regarding game hacks.

Some of the more famous involve Blizzard, with the biggest example being a suit against the creators of WowGlider, a bot for World of Warcraft (https://en.wikipedia.org/wiki/MDY_Industries,_LLC_v._Blizzar...)

⬐ jessespears

Fascinating: The Court of Appeals ruled that for a software licensee's violation of a contract to constitute copyright infringement, there must be a nexus between the license condition and the licensor’s exclusive rights of copyright. However ... that a finding of circumvention under the Digital Millennium Copyright Act does not require a nexus between circumvention and actual copyright infringement.

The DMCA's anti-circumvention section appears to be the only thing that preserved parts of the original finding. Perhaps the next time this law is up for review, some less onerous terms can be placed for reverse engineering for the purpose of interoperability.

He also did this: https://www.youtube.com/watch?v=qv6UVOQ0F44 (Also really impressive, I think this guy is a genius.)

⬐ harryf

Video conferencing with Minecraft is pretty cool too https://youtube.com/watch?v=sMH3wLuR9f0

⬐ pfista

That's awesome! Too bad they removed the content here http://verizoncraft.github.io

⬐ jags-v

@harator : Thanks for posting this link here. I would have never found it on my own.

⬐ rpgmaker

I just have a general knowledge of what neural networks are about so this video was actually more impressive to me than the one submitted. Thanks.

⬐ None

None

⬐ None

None

⬐ panic

This is cool, though (due to overfitting) it's really just "memorizing" the first level rather than learning how to play in general. It'd be interesting to see the same technique with separate test and training data sets.

⬐ minimaxir

The overfitting problem is addressed in the followup: https://www.youtube.com/watch?v=iakFfOmanJU

⬐ stormbrew

I wonder if there is a way it could learn to get over that tall pipe in this one. It might be sufficient for it to learn to deliberately attack any enemies it finds, combined with its tendency to move right all the time. But I suppose that might be too complex a set of actions for this.

MarI/O is one of my favorite.

https://m.youtube.com/watch?v=qv6UVOQ0F44

⬐ tomjakubowski

If you're interested in a "dumber" approach to machines "learning" to play classic video games (including Super Mario Bros), tom7's video [1] and paper [2] on Playfun are both fascinating.

[1] https://www.youtube.com/watch?v=xOCurBYI_gY

[2] https://www.cs.cmu.edu/~tom7/mario/mario.pdf

There is an objective, but it may not be exactly as you reason about it. There's a great video that made the rounds last year about building a neural net that plays Super Mario World, that may help visualize what's going on - https://www.youtube.com/watch?v=qv6UVOQ0F44

There's also a great snippet in the currently-ongoing AlphaGo videos that explains that when AlphaGo plays in ways that you may not expect, it's because it's strictly worried about _winning_ (even by the slimmest margin) with the greatest probability, and not necessarily by winning handily, like a human might.

⬐ qwertyuiop924

Yeah, I love sethbling. And the lua for MarI/O is only one file, and a relatively small one at that.

Jason Weston (Facebook) etc's work on Memory Nets[1]

Anything to do with transfer learning (the 2013 Zero-Shot Learning Through Cross-Modal Transfer[2] paper is a good place to start)

The increasing amount of demos around using NNs to generate "things" that look kinda-almost "intelligent". I can't point at a paper, but Andrej Karpathy demo of generating Shakespere-like writing, "Wikipedia" pages and "C" code in The Unreasonable Effectiveness of Recurrent Neural Networks[3] is the kind of thing I'm talking about.

The beginnings of work around goal-seeking. The (now Google) DeepMind Atari demo[4] and Marl/O[5]

Finally, the work being done on making this stuff usable by programmers (Torch etc).

[1] http://arxiv.org/abs/1410.3916

[2] http://arxiv.org/pdf/1301.3666.pdf

[3] http://karpathy.github.io/2015/05/21/rnn-effectiveness/

[4] https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf

[5] https://www.youtube.com/watch?v=qv6UVOQ0F44

Copied from youtube description: MarI/O is a program made of neural networks and genetic algorithms that kicks butt at Super Mario World. Source Code: http://pastebin.com/ZZmSNaHX "NEAT" Paper: http://nn.cs.utexas.edu/downloads/pap... Some relevant Wikipedia links: https://en.wikipedia.org/wiki/Neuroev... https://en.wikipedia.org/wiki/Evoluti... https://en.wikipedia.org/wiki/Artific... BizHawk Emulator: http://tasvideos.org/BizHawk.html

SethBling Twitter: http://twitter.com/sethbling SethBling Twitch: http://twitch.tv/sethbling SethBling Facebook: http://facebook.com/sethbling SethBling Website: http://sethbling.com SethBling Shirts: http://sethbling.spreadshirt.com Suggest Ideas: http://reddit.com/r/SethBlingSuggestions

⬐ iraldir

https://news.ycombinator.com/item?id=9713802

I believe it's been posted before, but sethbling (of vanilla-minecraft-hacks fame) pulled off the impossible and did the credit warp on an actual SNES, live[1]. As this video very-nicely explains, this required incredible amounts of pixel-perfect accuracy.

Lining up the line of koopa shells on exact x-offsets and very narrow heights mid jump (!) to write the landing-sled - using a controller, realtime - is utterly insane.

As a side note sethbling's latest project is teaching[2] (using neuro-evolution) his machine learning bot to play SMW[2], SMB1[3], and Mario Kart (original)[4]. While he machine learning is simple by today's standards, his brief videos give a really nice introduction to the topic that should be accessible to most people.

[1] https://www.youtube.com/watch?v=14wqBA5Q1yc

[2] https://www.youtube.com/watch?v=qv6UVOQ0F44

[3] https://www.youtube.com/watch?v=iakFfOmanJU

[4] https://www.youtube.com/watch?v=S9Y_I9vY8Qw

⬐ hayd

Someone apparently did a run in 42seconds ! https://www.youtube.com/watch?v=FkQdwUns7H8

Edit: Ah, that's "tool assisted".

⬐ yoha

Those videos are really great and worth watching!

⬐ Pewqazz

Note that this video is of SethBling pulling off the credits warp for the very first time, when the current setup for the warp was unknown. Here is a video of Seth performing the warp that was explained in the posted video, which is faster by over four minutes:

https://www.youtube.com/watch?v=KADhybyjOEo

⬐ aaggarwal

Thank you for sharing. The video explanation is really easy to follow.

For interested people, here is an interview with K. Stanley, the author of NEAT: http://aigamedev.com/open/interviews/galactic-arms-race/

I was wondering if this learning technique was applied to any other games. I came across this:

1. https://www.cs.utexas.edu/~mhauskn/papers/atari.pdf

2. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93....

⬐ deepnet

Bizhawk is a SNES emulator for Game Casters.

Seth, the video author is a Super Mario World Speedrunner and Twitchcaster. http://www.twitch.tv/sethbling

Bizhawk has Lua Scripting built in : http://tasvideos.org/Bizhawk/Features.html

Seth implemented Neuro-Evolution in Lua and http://pastebin.com/ZZmSNaHX

now machine learning casting is a thing...

⬐ scottfr

This is very cool, but it isn't really "playing" the game. Instead it's "memorizing" how to complete a specific level through trial and error.

If you took this neural network and tried it on another Mario level, it would fail miserably.

⬐ deepnet

The author has said that a net trained on one level can get about halfway through another level. http://www.reddit.com/r/videos/comments/39qel5/top_super_mar...

Super Mario World is very deterministic.

The Author is streaming various learning tasks on his twitch. MarI/O has been trained on 3 levels so far, for 24 hours each.

⬐ comex

Well, for some definition of fail miserably:

> Yeah, I think it wouldn't generalize super well, but I did try it on another easy level and it was able to get halfway through it.

- the author, at http://www.reddit.com/r/videos/comments/39qel5/top_super_mar...

⬐ synthmeat

I have reasonable suspicion that it wouldn't be nearly as easy to train without "spinning" move which basically gives invulnerability against all enemies sans their bullets, iirc.

⬐ babuskov

As far as I can tell, it would only require longer time (more iterations) for the network to learn, but it would still work.

The main problem I see is that the algorithm is learning from the same level (or set of levels). It would be interesting to see what happens if you give it some completely new level to play.

⬐ berberous

Wait, does it really give you invulnerability? In the video Mario never runs into a single enemy while spin jumping.

⬐ xahrepap

Not exactly. Some enemies that would kill you with a normal jump when you land on them, Mario will bounce off of with a spin jump.

⬐ M4v3R

A similar work was done for Super Mario on NES (along with other NES games): https://www.youtube.com/watch?v=xOCurBYI_gY

⬐ nip

I see this videoclip as a great introduction to neural networks and genetic algorithms (something I didn't know existed in ML) rather than a simple "computer completes videogame level". You're probably right nonetheless (I can't remember the mechanics of this mario game) :)

⬐ jeanlucas

Actually for Genetic Algorithms and/or Evolutionary computation it is not. :P

⬐ babuskov

TLDR: a neural network playing Super Mario.

A very cool demonstration showing how the network evolves.

⬐ dvirsky

This is a really cool and intuitive introduction to this sort of algorithms, and the example and visualization make it very accessible. Great work, I'm sure it will inspire some kids to get into ML and neural networks.

⬐ uxp100

I apologize, this is pedantic, but I think people often overstate the relationship between neural networks and the brain. I'm far from an expert, but saying neural networks are based on the brain is a stretch. They're inspired by a theory of how the brain works that is not wholly in line with what most scientists believe.

It's so tempting to say, this is based on the brain, when really it's more like, this was inspired by one psychologists model of the brain.

⬐ qtheninja

Awesomeness. How you explain machine learning principles/NEAT really shows a firm grasp. The fact it's done in lua...makes it even more eccentrically cool.

⬐ 140am

Veery nice visual introduction to the basics of neural networks

⬐ JonnieCache

Here's the author taking the world record (now superseded) at glitching mario world into the credits to "win" the game in 3:07:

https://www.youtube.com/watch?v=HxFh1CJOrTU

This is on a real console, not an emulator. They overflow the snes into the controller registers to achieve arbitrary code execution by hand. They do this by placing koopa shells at pixel-perfect locations to spell out ASM, overflow to the controller registers, and press a specific combination of buttons to jump to the sprite table and their literal shellcode.

At the start of this year there was a crazy few weeks where different speedrunners were competing to execute the "credits warp" in the quickest and most efficient way. It started at around 6 minutes, now it's down to around 2 I think, most of which is the game's intro. Fun stuff. SethBling was the first person to ever execute it on a real console.

Here's a video explaining it, from a different speedrunner: https://www.youtube.com/watch?v=vAHXK2wut_I

⬐ cheald

"literal shellcode" made me chortle. What a delightfully wonderful hack.

⬐ DadADadADA

You need Mario to teach this.

⬐ pdkl95

Sethbling (of Minecraft fame) gave very nice layman-accessible introduction to machine learning and neural networks. Specifically, he uses NEAT[1] to teach a neural network how to solve the first level of Super Mario World.

[1] https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...