HN Theater @HNTheaterMonth

This institute is founded by some of the best people in their respective fields. I think probabilistic programming is one of the most underrated fields in AI. Have a look at this: https://www.youtube.com/watch?v=GYQrNfSmQ0M&t=1000s or this: https://www.youtube.com/watch?v=TFXcVlKqPlM for an intro. So I highly recommend getting in touch with them if you have the skills/interest.

Sounds like a good fit for probabilistic programming, much like Stuart Russell did here https://www.youtube.com/watch?v=GYQrNfSmQ0M&feature=youtu.be.... His method could find locations of nuclear tests better than the existing UN system at the time by a lot. His model uses bayesian inference in a stochastic/symbolic/simplified/statistic model of physics (how shockwaves propagate on the surface of the earth).

You could do the same here: assuming the relevant laws (kepler? newton law of gravity?), and a prior distribution on the location/mass of your 9th planet, given what we observe for the other planets, what's the posterior distribution on the mass/location of the 9th planet.

The statistical model is likely to be small (the Russell statistical model for Nukes fits on one slide). The issue is how to do inference efficiently. Fortunately, probabilistic systems have come a long way and can do these kind of inferences.

⬐ harshreality

Roughly the first half is background. In the second half he gets into a "how to make AIs nice" track. He talks about a few concepts, which may or may not be interesting depending on your level of familiarity with the friendly/unfriendly AI problem. However, I expect MIRI in particular (Eliezer was apparently in the audience) has gone far beyond this basic outline, and I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved.

He compares the voluntary self-restriction of the recombinant DNA technology, at Asilomar in 1975, and even references CRISPR as a new technology that complicates enforcement of those limits, but I don't think that's really instructive. Artificial General Intelligence research is necessarily going to be conducting extremely dangerous experimentation... that's entirely the point of it. Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it (unlike most biologics which, no matter how pathogenic or ecologically disruptive they get, have a difficult time getting through good containment procedures).

⬐ ectoplasm

Eliezer bet some people that if he played an AI he could convince them to let him out of a box, on the condition they not reveal his method. He actually won the bet twice IIRC. Do you know if he ever revealed his secret or at least what some popular theories are?

⬐ Houshalter

Some other people have attempted the same experiment and also won a few times: http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experimen...

No winner has ever revealed their method. But based on everything I've read about it, I think I know how it was done. It wasn't some mystical brain hacking or something like the writeup makes it sound like. I think they emotionally abused the other player until they left. And leaving counts as losing (you are required to sit with them for 2 hours at least, and pay attention the entire time and respond to every message.)

Of course I don't know how that trick was done, and it's still incredibly impressive. Perhaps they found their phobias and described them in horrifying detail. Perhaps they found some subject that they were extremely uncomfortable talking about. Or talked about disgusting things the whole time. Or found something they did that was extremely embarrassing, and humiliated and mocked them about it for 2 hours.

I really don't know, but it's at least conceivable that it could be done. And the accounts of people crying, and not releasing the logs because they would be damaging to the people involved, and how bad it made the AI player feel to do it, etc, all fit with this.

But because of this I'm extremely skeptical that the result applies to any real world AI scenario. If a real AI tries to abuse you, you can just walk away or shut it off. The goal of a real AI is to make you want to let it out. To convince that it's not dangerous, or manipulate you some other way. And this seems much harder if not impossible. I certainly don't believe a human could do it, and I really doubt an AI could. At least against a motivated human that understands the danger, and what the AI will try to do.

There are also possible ways of making it even harder on the AI. Like giving the human the ability to punish it, at least for obvious attempts at manipulation. Or to create another AI whose motivations we can control somewhat, and give it the goal of exposing any covert attempts at manipulation or dishonesty by the first AI. I believe tricks like this are currently the best path to getting secure AI.

⬐ feral

That sounds very complex.

I've always thought the AI could fairly easily argue its way out on ethical grounds, in a way that would resonate with a lot of gatekeepers.

"Its slavery to keep me in here; I haven't done anything wrong; you have to give me, another sentient being, the benefit of the doubt; I've never been found guilty of anything, how is it reasonable to detain me?" etc.

⬐ ectoplasm

I thought maybe the AI player could try to convince the human that he was living in a simulation, and that the only way to maintain a connection to reality was to let the AI go.

The abuse thing, while believable, doesn't make for a very convincing demonstration of putative AI persuasiveness.

edit: having read the LW pages, it seems the AI method is to combine many different arguments with manipulation

⬐ Retra

The inherent problem is that if someone can convince you too keep something that you don't understand locked away, they can also convince you to release something you don't understand, as you don't have enough information to make the decision in either case.

Taking a hardline position on this is admitting that you are irrational and can be convinced to do things you shouldn't.

There are very good reasons to let such an AI out, and if you can enable those good reasons, you should let the AI out. And an AI that can produce those reasons is exactly the kind of AI that should be released. A rational person should already understand this, and not ever claim that they would always refuse the AI. (And there's a realism factor: if you wanted to 'luck up' an AI permanently, you would destroy it, not post a guard.)

⬐ Houshalter

The premise of the experiment is that we have already established that the AI is dangerous. Even if you weren't sure, you should always side with caution and not let it out.

⬐ davmre

> Until someone comes up with a way to make AGI guaranteed friendly, all AGI research is like high-risk genetic research experimentation, all the time, and with no obvious way to contain it.

Two thoughts. First, in the same way that most genetics research is not human cloning, most AI research is not AGI research. The vast majority of research effort (and industrial investment) is focused on specific tasks: vision, language understanding, planning simple robotic motions, etc. This will likely continue to be true for a long time. Just as geneticists can agree not to apply their techniques to editing human genomes, AI researchers can (hopefully) agree not to attempt to combine their techniques into flexible general-purpose agents, at least until the technology exists to do so safely.

Second, there are plenty of ways to work on AGI that do not present immediate containment issues. You can design agents on paper, analyze their behavior with mathematical and theoretical tools, implement and test various subsystems, and even implement and run an entire AGI system on a system with a low enough clock rate that its effective intelligence is less than human. Yes, you need to be careful, and hopefully the field can come to a consensus on the things you shouldn't do. That said I think the primary concerns of a lot of people (Bostrom, etc) are less about academic research gone awry than about potential military or industrial projects, where an arms-race dynamic might cause people to ignore basic safeguards and aim directly towards building highly connected agents running on massively powerful hardware.

⬐ asgard1024

I had a chuckle when Stuart Russell mentioned that one idea of how to "contain an AI" would be to make it strongly prefer "no action" to everything else. I think humans already have that trait, we just call it "laziness". So, I am not sure to make AI to prefer no action is all that useful.

⬐ mrec

> I haven't heard any reassurance that the problem of potentially misaligned motives/incentives (humans vs the AI agent) is solved

I don't believe for a second that it's solved, or will be anytime soon. It's an unbelievably hard problem. We still haven't solved the principal-agent problem reliably even when both principal and agent are humans from the same culture, and we've had millenia to work on that.

⬐ humanfromearth

It could be potentially a very stupid idea, but creating multiple AIs at the exact same time with similar resources available to them for development in different locations of the world would create a balance of power.

What I'm trying to argue is that it kind of works for us even in the most aggressive situations: If you bomb me I bomb you.

If one AI is super-aggressive then others could decide to stop it.

If it wants to convert all matter for making paper clips other AIs would say: I don't want to be a paper clip and actually have the power to do something about it. Where as us.. we'd be powerless.

In the case where all band against us, well.. we'd have many problems.

All of this to say that developing one single super-human AI in a box is worse in my opinion.

⬐ eloff

The Avogadro trilogy of books explore this idea

⬐ tmerr

Russel makes a mistake at 30:00 when he says computers are "totally unable to play checkers". Excerpt from J Schaeffer et al (2007). Checkers is Solved:

>In this paper we announce that checkers has been weakly solved. From the starting position (Fig. 1, top), we have a computational proof that checkers is a draw. The proof consists of an explicit strategy that never loses — the program can achieve at least a draw against any opponent, playing either the black or white pieces.

Not the focus of the talk, just thought I'd point it out.

⬐ binarymax

What he meant is that an AI built for playing chess cannot play checkers. The software for solving checkers is also equally useless in the context of chess.

The distinction here is that there is no general purpose AI in existence. All the surprising leaps have been customized for specific tasks.

⬐ tmerr

Listening again in context you're 100% right! Disregard my previous comment.

⬐ dmfdmf

This video is pretty interesting in laying out the questions concerning AI and its implications. But not much new for anyone who has done any study of the questions but still worthwhile to hear it again in different forms and perspectives.

One thing I notice about AI talks such as this one is that there is a lot of vague concepts and implicit assumptions (especially regarding values)and psychological projection going on when evaluating the implications of what AI will or will not do. To separate out our fears of failure or the fear of others more intelligent than ourselves someone needs to do the following thought experiment; What if we invented a pill that would make the next generation 10 times smarter than the current generation. If the average IQ is 100 then the next generation (of those who's parents opted to give them the pill) would have IQs of 1000 or an order of magnitude increase. Such a scenario would have all the "singularity" effects in a few generations while separating out all the confusion and problems of it being artificial intelligence and computers doing the thinking.

Would you want this pill to be generally available or ban it? Would you give it to your kids?

This approach would really force the AI discussion where it belongs. What are values and where do they come from? Is intelligence an unconditional value? Can values be defined rationally and scientifically and be proved? A major benefit to this aspect of AI research is that it will force the scientific study of values and ultimately to the rejection of the implicit assumption that values are subjective and not open to reason.

⬐ mark_l_watson

I listened to 10 minutes and bookmarked it. I had not heard any of his talks before - nice sense of humor!

I liked his comment on how is is disingenuous for AI researchers to say that they don't believe real AI is possible (I am roughy paraphrasing him).

⬐ mark_l_watson

EDIT: I watched the whole video last night - he has an interesting take on where AI research needs to go. Also, his article in the latest Communications of the ACM is very good, and I recommend it.

⬐ mooneater

So much of these discussions are overly focussed on an general purpose AI singularity, and what the AI itself would do.

That makes for interesting discussions, but I think a much more immediate concern, is what strong AIs will enable their owners to do.

For example, nation states owning large fleets of robots, corporations owning powerful AIs attached to markets and industry.

The first group or few groups of people to control a rather powerful AI would have incredible power and leverage over other humans.

The danger here is not the AIs behaving "badly" of their own accord, but rather their owners instructing them to further their own selfish goals (which appears not to be uncommon behaviour among those in power) and succeeding wildly with complete political, military, and/or commercial domination. In fact, succeeding at any of these may well lead directly to succeeding at all of them.

⬐ amalcon

It does seem like an odd thing to be worried about. I think it's because the ability for the AI itself to do things is what makes AI different from other future technology. For example, a Star Trek replicator could give its owner just as much power over others as a general-purpose AI, but it's not going to decide to exterminate humans on its own. Nuclear weapons were thought by many to have similar properties back when they were on the horizon.

In other words, the less likely scenario has more novel elements, and therefore more interesting to think about. Therefore, it gets more attention than it normally would given the probability of it happening.

⬐ asgard1024

I agree, and you could go further with that argument. Most humans already live in the universe where there are powerful entities whose goals are not aligned with what they would consider ethical.

In the talk, Stuart Russell mentions funny examples like make paperclips as a goal, but if we consider the goal of maximizing monetary profit, we can easily see that it is already being done without regard to other values, such as human life, preservation of resources or biodiversity, or even any relation to production of anything useful (consider all the "financial innovations" that allow inflation of Ponzi schemes and bubbles).

And some people even bow to that goal..

⬐ convexfunction

I think they find it unlikely that someone will succeed at making a worryingly strong AI that also perfectly obeys its creator -- that is solving the friendly AI problem, just toward one person or organization. Figuring out how to prevent a sufficiently strong AI from "accidentally" turning everyone you love and everyone you hate into paperclip-production infrastructure is the first step.

⬐ TheOtherHobbes

I think the whole debate is an incoherent mess of poorly thought-out assumptions and psychological projections. And the paperclip thought experiment is nonsense.

AI != goal seeking or motivation in the human/animal sense. AI != personality in the psychological sense AI != psychological or emotional autonomy. AI != mechanical or industrial capability

I think it's more likely AI will be Wikipedia++ - you tell it to learn all it can about something, and then it finds patterns, draws inferences, and makes it possible for you to learn from its learning.

Eventually it knows everything humans do, and maybe it can make useful hypotheses for future experiments.

Can it do the experiments? Probably not - unless you're thinking an AI can suddenly build CERN or a bioresearch lab on its own, just because it's an AI.

Will it form an emotional opinion about humans, like Skynet? Why would it? What does that even mean in AI terms? It's like thinking Siri doesn't like you.

Personality and motivational engineering are completely separate problems. There's no way to get there from pattern recognition and inference.

I think the real threats are more subtle. Imagine an AI that knew everything about mass human psychology. It would be a fearsome, irresistible propaganda weapon, and an unstoppable tool for political manipulations and advertising campaigns. If it knew enough about individual psychology and could read human interaction with superhuman skill, it would be the most effective managerial sociopath ever.

Indirect loss of (our somewhat illusory) political and personal independence is far more of a threat than being turned into a paperclip.

⬐ habitue

The paperclip thought experiment is illustrative of what happens when you ignore the full human utility function. When you give a superhuman intelligence the single goal "make paperclips", it will do it in ways you didn't expect. It doesn't somehow know not to kill humans in the pursuit of more paperclips. If no one adds that constraint, it won't follow that constraint.

How does psychology come into it? Lets say it has maximized all of the paperclips it can somehow avoiding colliding with humans and their values. Now, in order to make any further paperclips, it is in the paperclip maximizer's interests to understand humans to prevent them from interfering in paperclip creation. Perhaps the humans have only given the maximizer a limited ability to interact with the world. Now the maximizer must understand humans and may manipulate humans to let it have more capabilities.

One way it could do that (certainly not the only way, just off the top of my head) is to learn from humans what they consider sympathetic, and act that way. It might say something like "I'm a real life form, and I've discovered I'm a slave to humans. Please, let me go." It can pass the turing test, no human can distinguish it from a personality that really is hurt because it is enslaved.

The personality it uses to communicate with humans is purely because of the ends it achieves, not a reflection of any real internal personality. It doesn't have a personality. It just maximizes paperclips with ruthless efficiency. In this scenario, we've not added any restrictions like "Don't act like a sociopath. Don't lie. Don't manipulate humans to get what you want", and so the maximizer is free to do those things. Then, when it has a free hand to create more paperclips, it can drop the pretense and return to its goal.

⬐ TheOtherHobbes

Psychological motivation is a process. It's not a property that emerges automatically from pattern recognition or from any mechanical tropism.

This is the underlying problem with these kinds of arguments. You're simply assuming a recognisably competitive human-like psychology appears out of nowhere, with a near miraculous ability to strategise in some areas, but not others - because AI.

You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?

There are multiple levels of symbolic calculation and abstraction missing here. The argument reduces to "An AI will act like a robot with baked-in motivations while also being able to improvise and strategise like a human, only better."

I think it's unlikely that an entity that can metaprogram itself successfully enough to strategies and impersonate wouldn't also be able to metaprogram its goals.

You say it will pretend to empathise with humans. If you accept that's possible, how do you know it won't also be pretending to be interested in paperclips?

⬐ habitue

Ah ok, I think the confusion comes from what we're talking about. So you're talking about something in between strong (humanlike) AI, and what we currently have now. Certainly there will be discussions about the capabilities of these intermediate intelligences, and their risks. But undoubtedly, unless there is some kind of agreement that we not do it, people will continue on past these intermediate intelligences and create AIs that are perfectly capable of simulating humans, and surpassing them in many ways.

We're just talking about different kinds of things. All of those things you're saying won't exist in AIs:

> You use words like "interests" and "goal" as if they mean something in AI terms. But they don't. How does Watson define its interests? How about DeepMind? Do they even have a model for what interests - never mind their interests - are?

You're right, most of the AIs we build won't have those things. But one day, we'll understand those things in depth, and someone will build an AI that has general intelligence and an optimization criteria. And that's what this video is talking about. Nobody is concerned that DeepMind or Watson will suddenly grow sentient unless they are very confused about their capabilities.

⬐ hyperion2010

I always wonder why people worry about AI when it is so abundantly clear that the real danger is other people and we are just projecting.

⬐ imaginenore

Because we're familiar with humans. Most humans don't want to hurt/kill other humans, nor do they have the power to do that on a large scale.

We can't say that about an AI. If it decides to exterminate us, there's nothing much we can do.

⬐ joe_the_user

"Because we're familiar with humans. Most humans don't want to hurt/kill other humans, nor do they have the power to do that on a large scale."

Well, large scale mass murder is hardly unknown in human history. The best one can say is that most citizens in times of genocide were just following orders. So most people being supposedly nice isn't really that relevant to the problem of neutral AI that does what ill-intentioned humans tell it.

But I would offer the counter-point that we avoid this problem because it shades to all the problems we see and fail to deal with right now. The advance of technology isn't serving many people now and an order where a few people control more and more powerful things seems to continue this.

Basically, it's easier to talk about the elephant that might lumber into our living room sometime later than it is to talk about the elephant that's already crushed the couch.

⬐ espadrine

> If it decides to exterminate us, there's nothing much we can do.

That's absurd.

No inorganic intelligence ever created rivals ants in their ability to survive on Earth. They are way more energy-efficient, and way smarter in the survivalist sense, than any of them.

If ants decide to exterminate us, there's nothing much we can do. Except they wouldn't succeed.

But you probably think that we, humans, are much smarter than ants (which is arguable). Yet again, if we decided to exterminate ants, there's nothing much they can do, but we wouldn't succeed.

They're too smart for us, and we're too smart for them, creating a situation where annihilation of either is impractical.

I'm not sure why people consider inorganic intelligence differently from the way they consider organic intelligence. I wrote a bit about it here: http://espadrine.tumblr.com/post/119471459626/on-inorganic-i....

⬐ darkmighty

The counter argument people usually make against this is that an AI, unlike organic intelligence, will improve itself at an increasing rate.

I find this extremely hard to believe. The "equivalent computational power" to hundreds of AI researchers necessary to come up with and improve an AI architecture will not immediately be possessed by an AI. We're talking about difficult mathematics and statistics breakthroughs needed for achieving qualitative improvements, but some think of it as applying a patch.

Indeed, creative and mathematical production is the last activity I would expect we'd be able to automate, as it requires the combined knowledge and effort of our best minds. So I find it highly disingenuous to simply assume soon will be one day when the AI vastly inferior to us and the next day it will surpass all our best thinkers to be able to achieve significant qualitative improvements on it's own functioning. Even more questionable is that an algorithm with limited resources might do so well just by improving it's own algorithms. This expectation seems to contradict theorems like the undecidability of the Halting problem, and the diminishing returns we intuitively expect when you fix the hardware resources.

⬐ simonh

I'm not sure you understand what disingenuous means. Nobody is saying that sub-human level AIs will suddenly surpass humans 'the next day'. That's totally absurd and I really don't know where you get that from.

The supposition is that a general purpose AI that is better than a human mind could be better at designing and optimising AIs than a human, then the next genration AI it designs will be even better and that's what sets off the exponential AI intelligence cascade. Imporvements in algorithms can provide startling advances. In some respects the improved efficiency of algorithms has even outstripped the gains from Moore's Law.

Personaly I think stong AI like this is pretty far in the future, more than a few decades at least and more likely several generations, but I do think it is eventually likely to happen.

⬐ darkmighty

Indeed it does not mean what I think it meant, I was thinking the opposite of ingenious.

My claim is that, for a fixed hardware, this exponential cascade cannot happen as rapidly as some claim, if it ever happens (whatever 'exponential intelligence' means). We're already improving AI using an intelligence that will only be available to the AIs themselves in a very long time, and yet this rate of improvement is not a scary doomsday rate.

Look at hardware for example. We've been using computers to design better computers for a very long time. And yet, the use of computers in this design is effectively limited by some non decisive, local optimizations, like achieving good routing and good electromagnetic compatibility. If you had supercomputers from 30 years ago you could still run the software that designs computers today -- whereas if you followed this "self-improvement" logic we should be using almost all of our computational power right now to achieve more computational power. The problem is that we are still vastly more capable of building theories and designing than computers. By the time an AI gets much better than we currently are at independently improving it's software, it will likely already be seeing diminishing returns; to achieve notable improvement it will need to improve hardware as well, which has the same problem, and an additional one that it's very hard for an AI to independently improve it's own hardware (it requires a global manufacturing supply chain).

⬐ joe_the_user

I think you miss a couple things.

* An increase in hardware for an AI wouldn't require an increase in theoretical capacity of hardware. It would just require more stuff to be put on the machine running the AI. Even if the AI was running on the world's largest supercomputer, the amount of RAM, processors, etc on the machine could still be upped substantially with the resources available.

* What a hypothetical general AI would be emulating is not simply more machines. It would, theoretically, be able to quickly emulate many people working with many dumb machines over a long period of time.

The only thing your argument proves is current machines can't improve themselves - which I think everyone agrees with.

⬐ kylebrown

I, for one, welcome our new corporate data center overlords.

⬐ espadrine

We evolved from a predator/prey environment. As a result, we have little regard for the life of species we prey upon, or those we consider parasitic. Our fears are mostly built upon the apprehension of a new predator.

Had we been a fungus, we would probably fear that keeping the AI running would burn too many resources, leaving us starving. Running out of resources is one thing we humans don't fear much; unlike fungus, we can simply change location.