HN Books @HNBooksMonth

> What area of machine learning do you feel is closer to how natural cognition works?

None. The prevalent ideas in ML are a) "training" a model via supervised learning b) optimizing model parameters via function minimization/backpropagation/delta rule.

There is no evidence for trial & error iterative optimization in natural cognition. If you'd try to map it to cognition research the closest thing would be behaviorist theories by B.F. Skinner from 1930s. These theories of 'reward and punishment' as a primary mechanism of learning have been long discredited in cognitive psychology. It's a black-box, backwards looking view disregarding the complexity of the problem (the most thorough and influential critique of this approach was by Chomsky back in the 50s)

The ANN model that goes back to Mcculloch & Pitts paper is based on neurophysiological evidence available in 1943. The ML community largely ignores fundamental neuroscience findings discovered since (for a good overview see https://www.amazon.com/Brain-Computations-Edmund-T-Rolls/dp/... )

I don't know if it has to do with arrogance or ignorance (or both) but the way "AI" is currently developed is by inventing arbitrary model contraptions with complete disregard for constraints and inner workings of living intelligent systems, basically throwing things at the wall until something sticks, instead of learning from nature, like say physics. Saying "but we don't know much about the brain" is just being lazy.

The best description of biological constraints from computer science perspective is in Leslie Valiant work on "neuroidal model" and his book "circuits of the mind" (He is also the author of PAC learning theory influential in ML theorist circles) https://web.stanford.edu/class/cs379c/archive/2012/suggested... , https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...

If you're really interested in intelligence I'd suggest starting with representation of time and space in the hippocampus via place cells, grid cells and time cells, which form sort of a coordinate system for navigation, in both real and abstract/conceptual spaces. This likely will have the same importance for actual AI as Cartesian coordinate system in other hard sciences. See https://www.biorxiv.org/content/10.1101/2021.02.25.432776v1 and https://www.sciencedirect.com/science/article/abs/pii/S00068...

Also see research on temporal synchronization via "phase precession", as a hint on how lower level computational primitives work in the brain https://www.sciencedirect.com/science/article/abs/pii/S00928...

And generally look into memory research in cogsci and neuro, learning & memory are highly intertwined in natural cognition and you can't really talk about learning before understanding lower level memory organization, formation and representational "data structures". Here are a few good memory labs to seed your firehose

https://twitter.com/MemoryLab

https://twitter.com/WiringTheBrain

https://twitter.com/TexasMemory

https://twitter.com/ptoncompmemlab

https://twitter.com/doellerlab

https://twitter.com/behrenstimb

https://twitter.com/neurojosh

https://twitter.com/MillerLabMIT

⬐ KKKKkkkk1

> I don't know if it has to do with arrogance or ignorance (or both) but the way "AI" is currently developed is by inventing arbitrary model contraptions

Deep learning is incredibly successful in solving certain real-world problems such as detecting and recognizing faces in photos, transcribing speech, and translating text. It's true that some trolls claim that gradient descent is how the brain works [1]. But if you open almost any machine learning textbook, you'll see on one of the first pages an acknowledgement that the methods do not agree with modern neuroscience (while still being incredibly useful).

[1] https://twitter.com/ylecun/status/1202013026272063488

⬐ sillysaurusx

For what it’s worth, I agree with this take. But I think RL isn’t completely orthogonal to the ideas here.

The missing component is memory. Once models have memory at runtime — once we get rid of the training/inference separation - they’ll be much more useful.

⬐ andyxor

not sure about RL, but ANN even in their current brute force form, can be used as pre-processing/dimensionality reduction/autoencoder layer in content-addressable memory model, such as SDM by Kanerva, which does have some biological plausibility https://en.wikipedia.org/wiki/Sparse_distributed_memory

Also, the 'neocognitron' by Fukushima, which is the basis of CNNs, was inspired by actual neuroscience findings from visual cortex in the 70s (and speculatively, may be that's why it works so well in computer vision). So deep learning might have some complementary value as a representation of lower level sensory processing modules e.g. in V1, what's missing is a computational model of hippocampus and "the rest of the f..g owl"

⬐ bobberkarl

just to say this is the kind of answer that makes HN an oasis on the internet.

⬐ unishark

The place/grid/etc cells fall generally under the topic of cognitive mapping. And people have certainly tried to use it in A.I. over the decades, including recently when the neuroscience won the Nobel prize. But in the niches where it's an obvious thing to try, if you can't even beat ancient ideas like Kalman and particle filters, people give up and move on. Jobs where you make models that don't do better at anything except to show interesting behavior are computational neuroscience jobs, not machine learning, and are probably just as rare as any other theoretical science research position.

There is a niche of people trying to combine cognitive mapping with RL, or indeed arguing that old RL methods are actually implemented in the brain. But it looks like they don't much benefit to show in applications for it. They seem to have no shortage of labor or collaborators at their disposal to attempt and test models. It certainly must be immensely simpler than rat experiments.

Having said that, yes I do believe that progress can come considering how nature accomplish the solution and what major components we are still missing. But common-sense-driven tacking them on there has certainly been tried.

The back-prop learning algorithm requires information non-local to the synapse to be propagated from output of the network backwards to affect neurons deep in the network.

There is simply no evidence for this global feedback loop, or global error correction, or delta rule training in neurophysiological data collected in the last 80 years of intensive research. [1]

As for "why", biological learning it is primarily shaped by evolution driven by energy expenditures constraints and survival of the most efficient adaptation engines. One can speculate that iterative optimization akin to the one run by GPUs in ANNs is way too energy inefficient to be sustainable in a living organism.

Good discussion on biological constraints of learning (from CompSci perspective) can be found in Leslie Valiant book [2]. Prof. Valiant is the author of PAC [3] one of the few theoretically sound models of modern ML, so he's worth listening to.

[1] https://news.ycombinator.com/item?id=26700536

[2] https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...

[3] https://en.wikipedia.org/wiki/Probably_approximately_correct...

⬐ hctaw

I think there's a significant difference worth illustrating that "there is no feedback path in the brain" is not at all equivalent to "learning by feedback is not possible in the brain."

It's well known in dynamics that feed-forward networks are no longer feed-forward when outputs are coupled to inputs, an example of which would be a hypothetically feed-forward network of neurons in an animal and environmental conditioning teaching it the consequences of actions.

I'm very curious on the biological constraints, but I'd reiterate my point above that feedback is a mathematical or logical abstraction for analyzing the behavior of the things we call networks - which are also abstractions. There's a distinction between the physical behavior of the things we see and the mathematical models we construct to describe them, like electromechanical systems where physically no such coupling from output-to-input appears to exist, yet its existence is crucially important analytically.

exactly, besides ignoring the innate structures, heuristics and biases hardcoded via evolution, the whole notion of "learning" became highly intertwined with reinforcement kind of learning, i.e trial & error, stimulus and response behaviorist terms popularized by Pavlov and Skinner a century ago, which is just one type in a large repertoire of adaptation mechanisms.

Memory in these models is used as afterfact, or some side utility for complex iterative routines based on calculus of function optimization. While in living organisms memory and its "hardcoded" shortcuts allow to cut through the search space quickly as in a large database index.

Speaking in database terms we have something like "materialized views" on acquired and genetically inherited knowledge, built from compressed and hierarchically organized sensory data and prior related actions and associations, including causal links. Causality is just a way to associate items in the memory graph.

Error correction doesn't play as much role in storing and retrieving information and pattern recognition, as current machine learning models may lead you to believe.

Instead, something akin to self-organized clustering is going on, with new info embedded in the existing "concept" graph via associations and generalizations, through simple LINK and JOIN mechanisms on massive scale.[1] The formation of this graph in long term memory is tightly coupled with sleep cycles and memory consolidation, while short term memory serves as a kind of cache.

Knowledge is organized hierarchically starting from principal components [2] of sensory data from e.g. visual receptive fields, and increasing in level of abstraction via "chunking", connecting objects A and B to form a new object C via JOIN mechanism, or associating objects A and B via LINK mechanism. Both LINK and JOIN outputs are "persisted" to memory via Hebbian plasticity.

All knowledge including causal links are expressed via this simple mechanism. Generating a prediction given a new sensory signal is just LINKing the signal with existing cluster by similarity.

Navigation in this abstract space is facilitated via coordinate system similar or perhaps identical to the role hippocampal place & grid cells play in spatial navigation. Similarity between objects is determined as similarity between their "embeddings" in this abstract concept space.

It's possible that innate structures are genetically pre-wired in this graph which represent high level "schemas", such as innate language grammar which distinguishes e.g. verb from noun, visual object grammar which distinguishes "up" from "down", etc. It is also possible these are embodied, i.e. connected to some representation of motor and sensory embeddings. And serve to bootstrap the graph structure for subsequent knowledge acquisition. I.e. no blank slate.

The information is passed, stored and retrieved via several (analogue) means both in point-to-point and broadcast communication, with electromagnetic oscillations playing primary role in synchronization in neural assemblies, facilitating e.g. speech segmentation (or boundary detection in general), and coupling an input signal "embedding" to existing knowledge embeddings in short term memory; while neural plasticity/LTP/STDP as storage mechanisms on single neuron level.

[1] See Leslie Valiant "neuroidal" model and his book https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...

[2] See Oja Rule http://www.scholarpedia.org/article/Oja_learning_rule

and Olshausen & Field classic work on sparse coding http://www.scholarpedia.org/article/Sparse_coding

Deep learning is mostly irrelevant for AGI but the best part of the article is bringing up the "recursive process called Merge”.

This Merge [0] is called "chunking" in cognitive psychology [1, 2], first mentioned in classic paper "The Magical Number Seven" by George A. Miller [3].

In the original Chomsky work[0] it is buried so deep in linguistics jargon it's easy to miss the centrality of this concept, which is the essence of generalization capability in biological mind.

It's the JOIN in Leslie Valiant LINK/JOIN model [4, 5]:

"The first basic function, JOIN, implements memory formation of a new item in terms of two established items: If two items A and B are already represented in the neural system, the task of JOIN is to modify the circuit so that at subsequent times there is the representation of a new item C that will fire if and only if the representations of both A and B are firing."

Papadimitriou & Vempala [6] extend it to "predictive join" (PJOIN) model.

Edit: As I think about it deep learning might be useful in implementing this "Merge" by doing nonlinear PCA (Principal Component Analysis) via stacked sparse autoencoders, kind of like in that "Cat face detection" paper by Quoc Le [7]. The only thing missing is hierarchical memory representation for those principal components, where NEW objects emerge by joining most similar existing objects.

[0] https://en.wikipedia.org/wiki/Merge_(linguistics)

[1] https://en.wikipedia.org/wiki/Chunking_(psychology)

[2] http://www.columbia.edu/~nvg1/Wickelgren/papers/1979cWAW.pdf

[3] https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus...

[4] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.208...

[5] https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...

[6] https://arxiv.org/pdf/1412.7955.pdf

[7] https://ieeexplore.ieee.org/abstract/document/6639343

⬐ mcswell

"the task of JOIN is to modify the circuit so that at subsequent times there is the representation of a new item C that will fire if and only if the representations of both A and B are firing." What that describes sounds like what in linguistic (syntactic) terms one would call conjunction. I don't recall whether Chomsky's recent theories (which come under the rubrik of "minimalism") count conjunction as a type of Merge, but what he's more concerned with is subordination, which is quite a different thing from conjunction, given that subordination--but not conjunction--results in hierarchical structures.

And for the record, I don't think Merge in the linguistic sense has anything to do with "chunking", much less the "magical number seven." But then I don't know much about chunking.

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent.

Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient model of learning as reward and punishment, iterative trial and error process.

This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently.

Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization.

This is the case where computational efficiency [1] comes first and sets the constraints by which cognitive apparatus needs to be developed.

As for actual cognition models a good place to start is not ML/AI field (which tends to getting stuck in local minima as a whole), but state-of-the-art cognitive psychology, and may be looking at research in "distributional semantics", "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience.

You'd be surprised how much knowledge we gained about the mind since those RL & ANN models developed in the 1940s.

[1] https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...

⬐ GregarianChild

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

   cognitive psychology

You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience?

[1] https://arxiv.org/abs/1505.00521

⬐ bra-ket

>Where is .. AI

it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.

I mean you can't just throw things at the wall and hope they stick, but it's literally the state of the art, if you follow ML conferences and their world-beating toy benchmarks results, with a lot of pseudo-rigorous handwaving for theory.

The reason physics has been so successful is that their theory closely followed empirical data and constraints imposed by nature.

I think the only hope to achieve common sense in AI is to align it with hard constraints living organisms have, using those constraints as a guide.

A few terms I mentioned are coming from that POV, if you dig a bit deeper they all have direct physical manifestation in natural learning systems.

⬐ YeGoblynQueenne

>> it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.

A small correction: that's deep learning researches, not AI researchers and not all machine learning researchers even. To be charitable, it's not even all deep learning researchers. It's just that the field of deep learning research has been inundated with new entrants who are sufficiently skilled to grok the practicalities but lack understanding of AI scholarship and produce unfortunately shoddy work that does not advance the field (any field, any of the aforementioned ones).

As a personal example, my current PhD studies are in Inductive Logic Programming which is, in short, machine-learning of logic programs (you know, Prolog etc). I would not be able to publish any papers without a theoretical section with actual theoretical results (i.e. theorems and their proofs - and it better be a theorem other than "more parameters beget better accuracy", which is not really a theorem). Reviewers would just reject such a paper without second thought, regardless of how many leaderboards I beat in my empirical results section.

And of course there are all the other fields of AI were work continues - search, classical planning, constraint satisfaction, automated theorem proving, knowledge engineering and so on and so forth.

Bottom line- the shoddy scholarship you flag up does not characterise the field of AI research, as a whole, it only afflicts a majority of modern deep learning research.

⬐ eli_gottlieb

>Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

Brainfuck is also Turing complete, so logically if we just do, for instance, Markov chain Monte Carlo for Bayesian program learning in Brainfuck, we can realize AGI that way.

"Everything is possible, but nothing is easy." The Turing tarpit.

⬐ ForHackernews

> Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

This seems like overstating your point. Nobody has been able to rigorously define "AI" yet, so there's no way of saying whether it's possible with a Turing machine architecture. The human brain, at least, doesn't seem that similar to a Turing architecture. Neurons don't carry out anything like discrete operations.

Maybe it's possible to run AGI on a Turing machine, maybe it's not, but there are more options than simply "possible with a Turing machine" or "completely impossible".

⬐ joe_the_user

Maybe it's possible to run AGI on a Turing machine, maybe it's not,

Arguing from ignorance, of course nothing is knowable for certain. However there has been a lot of work on the universality of Turing machines, showing that a Turing machine can simulate any conceivable concept of finite computation and can approximate any conventional physical system.

I think a more useful way to express your intuition is to note that if human-built AGI comes into existence, it might be runnable on a Turing machine but quite possibly not efficiently so.

⬐ GregarianChild

While I agree that we don't have an agreed upon definition of AI, the problem is firmly in the "I" part of AI! The "A" part is taken to mean implementable by a computer, i.e. a Turing machine. This is the content of the Church–Turing thesis [1].

[1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis

⬐ ForHackernews

> It states that a function on the natural numbers can be calculated by an effective method if and only if it is computable by a Turing machine.

The article at the top of this thread is specifically about properties ("The Generalized Archimedean Property") that real numbers do not possess.

There's also a little bit of slipperiness around the use of "AI" vs. "AGI" - you could easily argue (and people do!) that we've already achieved "AI" for many specialized domains. It's the General bit that seems to be the sticking point, and that this article focuses on.

⬐ bra-ket

>Where is the world-beating AI

it's about 5 years away.

⬐ fsuuttg

Why does it matter? Turing completeness is a low bar. Both exp(exp(N)) and log(log(N)) algorithms solve the AI problem in a finite number of steps, but one of them is a useless abstraction, while the other one really works.

⬐ YeGoblynQueenne

Note that Turing complete means undecidable. An algorithm that can learn any program that can be computed by a Universal Turing Machine must, in the worst case, search an infinite program space. So, even if a Neural Turing Machine can learn arbitrary programs (I haven't read the paper so I can't say) it might need to consume infinite resources before learning any particular progarm.

In short- Turing completeness is no guaranteed path to AGI. Assuming an "AGI program" exists, it is hidden away in an infinity of almost identical, but not quite, programs.

⬐ gnulinux

> The idea of using "rewards" as a learning mechanism and a path to actual cognition is just wrong, full stop.

I'm a layman (just a software engineer) but am curious, I train my cat only with rewards (never punishment because apparently doesn't work on cats) and the kitty learned how to high-five me, sit, jump, follow me etc. It seems to work really well for us. Basically, ever time he does something desirable, I click my pen and give him his favorite treats. Is this ineffective?

⬐ kypro

Also a layman, but I think OPs point wasn't that it isn't possible, but that's it's not effective or analogous to how humans or other species learn.

For example, your cat's brain isn't just a randomly initialised neural net. Your cat comes pre-wired in such a way that it understands certain things about its environment and has certain innate biases that allow you to train it to do simple tricks with relative ease through a reward mechanism.

A more analogous example would be building a cat-like robot with four legs and a neural processor then switching it on and expecting to be able to train it with treats. Without a useful initial neural state (founded with an understanding of cognitive psychology and neuroscience) it would be almost totally useless.

⬐ benchaney

Your cat was already capable cognition before you started training it. GP is talking about generating a cognition where it did not previously exist.

⬐ gnulinux

I see, I understand the distinction. Thank you for clarifying.

⬐ joe_the_user

Why couldn't a mechanism (say A Neural Turing or whatever) that you train be "cognition capable" when you start and then be trained to actual behavior after that?

⬐ benchaney

You would need something that is "cognition capable" first and that has not been invented yet.

⬐ joe_the_user

It's hard to know. Maybe something "cognition cable" exists, it's just the proper train routine hasn't been provided to it.

But regardless, the broader point is yeah, combine something akin cognition capability and the proper training routine and there you go, AGI from "reinforcement learning", broadly defined.

⬐ anotherProd

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm.

Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" is a pretty accurate description with how far we've come when it comes to consciousness, and I would imagine even less for the newer field of AI/AGI (consciousness has been around for a while P: and our theories have mostly sidestepped this real world phenomenon).

> "The idea of using "rewards" as a learning mechanism and a path to actual cognition is just wrong, full stop."

This to me is a huge red flag (mostly of ego/hubris). I think if we rephrased the goal to not talk about "AGI" and maybe around quantitative things like the things you've listed ("computational efficiency", likelihood of being stuck in local minimimas, etc) then I'd happily concede that we should be looking at "X" and not "Y" but unless I've missed something, again likely, when we're talking about AGI, we're talking about consciousness (epiphenomenon that come about through physical/deterministic interactions). A quick way to gut check myself here is twisting what you state is not a good place to start "ML/AI field... gets stuck in local minima" and ask myself is it possible that local minima (which we consider "bad" for current/traditional tasks) could be necessary for consciousness ? I think the widely accepted answer to this is currently "We don't know".

If I think that achieving AGI is going to be similar to what the algorithms and architecture we currently use (where the likelihood of being stuck in a local minima is something we can look at) then sure, your opinions stand. But that is just a guess and unless I'm mistaken AGI hasn't been achieved because we don't know how to do it.

This isn't to say that we should have 100% of the data before making strong judgements like this about a subject. It's just that the subject of "consciousness" is a big one (I'd say THE big one) so making such strong statements about something we know we don't know much about is interesting. <- this is where I get flashbacks to SE world where a missing piece of data can really throw you off or leads to wrong assumptions and when I think about consciousness we know we don't know a lot.

⬐ bra-ket

My point was that RL/DL is being used like some kind of massive hammer to hit all the nails. Cognition requires different, specialized, energy-efficient tools.

> consciousness

All talk about this is premature and "pre-science", before we figure out more basic, fundamental things like object storage and recall from memory, object recognition from sensory input, concept representation and formation, the exact mechanism of "chunking" [1], "translational invariance" [2], generalization along concept hierarchy and different scales, representation of causal structures, proximity search and heuristics, innate coordinate system, innate "grammar".

Even having a working, biologically-plausible model of navigation in 3d spaces by mice, without spending a ton of energy training the model, would be a good first step. In fact there is evidence that navigational capacity [3] is the basis of more abstract forms of thinking.

On all of these things we have decades worth of research and widely published, fundamental, Nobel-winning discoveries which are almost completely ignored by the AI field stuck in its comfort zone. Saying "we have no idea" is just being lazy.

Edit: As for OP's actual paper I think something like complex-valued RL [4] might bypass his main claims entirely. But my point is that RL itself is a dead end, trivializing the problem at hand.

[1] https://en.wikipedia.org/wiki/Chunking_(psychology)

[2] http://www.moreisdifferent.com/2017/09/hinton-whats-wrong-wi...

[3] http://www.scholarpedia.org/article/Grid_cells

[4] https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22c...

⬐ anotherProd

> My point was that RL/DL is being used like some kind of massive hammer to hit all the nails. Cognition requires different, specialized, energy-efficient tools.

My point is that you don't know that. We don't know that.

> All talk about consciousness is premature and "pre-science", before we figure out more basic, fundamental things like object storage and recall from memory...

And those are prerequisites to consciousness ? What I'm trying to point out is that you're making assumptions about what it takes to create AGI/consciousness, full stop.

You believe that those things are required before you can achieve consciousness.

> Even having a working, biologically-plausible model of navigation in 3d spaces by mice... would be a good first step

Same as above, do you know something that's not widely known about consciousness ? I'd like to see this , obvious to you, plan where we know the dead ends and we know good first steps.

If we're not talking about consciousness but very-efficient-task-doers (not-consciousness) then sure let's talk about this like we talk about our PC's or compartmentalized components.

> Saying "we have no idea" is just being lazy.

I agree, I do think we have some idea but I believe that we don't have enough of that idea to be able to rule out anything, full stop. Anything. Unless you know of some breakthrough that lead to that closed-minded way of thinking about something we haven't achieved. It might be unlikely (article title) but to see such absolutes and assumptions about consciousness...

⬐ nagaa

I have learned to play piano and drive a car. IMO both of these took two completely different sets of systems and algorithms in order to accomplish the learning task. Nothing I learned from piano applies to driving and vice versa. The only thing in common is my brain. We want a computer though to apply those algorithms I learned driving and playing piano to golf and have it work. We will then have "AGI". Obviously that fails. Obviously.

⬐ xvilka

I think it will require at least a few more centuries to build AGI.

⬐ thomasfromcdnjs

Stumbled upon this the other day, seems interesting ¯\_(ツ)_/¯

https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind

"Penrose argues that human consciousness is non-algorithmic, and thus is not capable of being modeled by a conventional Turing machine, which includes a digital computer. Penrose hypothesizes that quantum mechanics plays an essential role in the understanding of human consciousness. The collapse of the quantum wavefunction is seen as playing an important role in brain function."

⬐ thechao

Penrose is a superlatively brilliant physicist, but his opinion on AI is the worst sort of woo. It’s little better than reading Chopra. His argument is most salient as an example of attempting to use authority in one field to garner authority in another.

⬐ mannykannot

I don't know whether RL is the wrong level of abstraction, but I am pretty sure that the individual organism is.

Individual humans are born with a neural architecture, and a mechanism for growing it further, developed over half a billion years through a form of reinforecement learning called evolution. We are not blank slates at birth, nor anything like it, and the capabilities of our intelligence is not constrained to merely that which we could learn in our own infancy and adolescence.

⬐ eli_gottlieb

>state-of-the-art cognitive psychology, and may be looking at research in "distributional semantics", "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience.

Look, uh, I've read Gardenfors too, but are those really the state of the art? I don't remember there being anything about them at CogSci this past summer. Maybe I wasn't paying close-enough attention?

⬐ bra-ket

In RL/DL context any CogSci developments after 1943 is the state of the art.

Some interesting recent work [1] related to Gardenfors ideas was combining them with discovery of place & grid cells, and extending the "cognitive maps" and spatial navigation machinery into concept spaces, treating the innate coordinate system as foundation for abstraction and generalization facilities.

And they actually found empirical data to prove it in [1] and related papers, so Gardenfors was right.

I believe it gotta be the starting point for anyone seriously considering an AI, kind of like Cartesian foundation. It also aligns nicely with rich "distributional semantics" work and popular vector space models.

[1] https://pubmed.ncbi.nlm.nih.gov/27313047/

⬐ odnes

> The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent.

A lot of research in RL is focused on intrinsic motivation and the question of whether we can bootstrap our own 'rewards' from our ability to predict and control the future according to some self-defined goals/hypotheses.

⬐ cmrx64

do you know about the dopamine reward error hypothesis? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6721851/ is it so wrong? what does cognitive psychology have to say about how these neurons work? this is a lot more recent than the 40s and behaviorism.

⬐ bra-ket

dopamine rewards operate on a different time scale vs. that required by these error correction models. I don't remember the exact paper, will need to look it up, but it was orders of magnitude difference in response times.

Edit: for authoritative reference on biologically-plausible learning see anything by Edmund Rolls [1]. He explicitly stated in his recent book [2] that something like back-propagation, or similar error correction mechanisms have no supporting evidence in experimental data collected so far

[1] https://www.oxcns.org/profile.html

[2] https://www.amazon.com/Cerebral-Cortex-Principles-Edmund-Rol...

⬐ cmrx64

thanks for the link, it's been a lovely rabbit hole :)

⬐ postnihilism

> "This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently."

> "Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization."

But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured.

⬐ qyv

> But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured.

This is true in the context of the universe as a whole, not by the organism itself.

⬐ renjimen

Except no organism is born a blank slate. Parent is correct in that our prior was massively expensive to construct

⬐ AlphaSite

I think the point huh is being mad is that individual people (or models) dot learn that way. It’s not like models training models, all the way down.

⬐ the8472

Individual people are not trained from scratch. ML models often have to be (modulo fine-tuning) since the field is still young.

⬐ mannykannot

That's already changing. That we have only relatively recently moved beyond always starting from scratch might indicate that we are still in the Cambrian of AI, however...

⬐ dkersten

So we can expect our ANN’s to yield AGI in a few million or billion years? That doesn’t sound like a good place to put our current efforts then.

⬐ mannykannot

That does not necessarily follow, as I imagine you well know.

⬐ YeGoblynQueenne

Why wouldn't it follow? Human intelligence evolved in the real world with all its vast information content. Deep learning systems are only trained on a few terrabytes of data of a single type (images, text, sound etc). Even if they can be trained faster than the rate at which animals evolved, their training data is so poor, compared to the "data" that "trained" animal intelligence that we'll be lucky if we can arrive at anything comparable to animal intelligence by deep learning in a billion years.

Or unlucky, as the case may be.

⬐ mannykannot

You elided the "necessarily".

One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

⬐ YeGoblynQueenne

>> You elided the "necessarily".

Well, if it follows, then it follows necessarily. But maybe that's just a deformation professionelle? I spend a lot of time working with automated theorem proving where there's no ifs and buts about conclusions following from premises.

⬐ mannykannot

If I am not mistaken, it does not necessarily follow unless it turns out to be a sound argument in every possible world.

⬐ YeGoblynQueenne

Ah, so you are making a formal argument? In that case you should stick to formal language. And probably publish it in a different venue :)

⬐ mannykannot

No, I am simply responding to your rather formal point, in kind. Unless you are aguing for it being an established fact that the time evolution took to produce intelligent life rules out any form of reinforcement learning producing AI in any remotely reasonable period of time, then that original point of yours does not seem to be going anywhere.

In your work on theorem proving, am I right in guessing that there are no 'ifs' or 'buts' because the truth of premises is not an issue? In the "evolution argument", the premises/lemmas are not just that evolution took a long time, but also something along the lines of significant speedup not being possible.

You might notice that in another comment, I suggested that we might still be in the AI Cambrian. I'm not being inconsistent, as no-one knows for sure one way or the other.

⬐ YeGoblynQueenne

I didn't make a formal point- my comment is a comment on an internet message board, where it's very unlikely to find formal arguments being made. But perhaps we do not agree on what constitutes a "(rather) formal point"? I made a point in informal language and in a casual manner and as part of an informal discussion ... on Hacker News. We are not going to prove or disprove any theorems here.

But, to be sure, as is common when this kind of informal conversation suddendly sprouts semi-formal language, like "argument", "claim", "proof", "necessarily follows" etc, I am not even sure what exactly it is we are arguing about, anymore. What exactly is your disagreement with my comment? Could you please explain?

⬐ mannykannot

"Necessarily" has general usage as well, you know... why would you read it otherwise, especially given the reasonable observation you make about this site? And my original point is not actually wrong, either: whether reinforcement learning will proceed at the pace of evolution is a topic of speculation - it is possible that it will, and possible that it will not.

Insofar is I have an issue with your comment, it is that it is not going anywhere, as I explained in my previous post.

⬐ YeGoblynQueenne

>> Insofar is I have an issue with your comment, it is that it is not going anywhere, as I explained in my previous post.

I see this god-moding of my comment as a pretend-polite way to tell me I'm takling nonsense, that seems to be designed to avoid criticism for being rude to one's interlocutor on a site that has strong norms against that sort of thing, but without really trying to understand why those norms exist, i.e. because they make for more productive conversations and less wasting of everyone's time.

You made a comment to say that unless I claim that X (which you came up with), then my comment is not going anywhere. The intellectually corteous and honest response to a comment with which one does not agree is to try and understand the reasoning of the comment. Not to claim that there is only one possible explanation and therefore the comment must be wrong. That is just a straw man in sheep's clothing.

And this is not surprising given that it comes at the heels of nitpicking about supposedly important terminology (necessarily!). This is how discussions like this one go, very often. And that's why they should be avoided, because they just waste everyone's time.

⬐ mannykannot

"Necessarily", when read according to your own expectations for this forum, made an important difference to my original post (without it, I would have been insisting that the issue is settled already), so it was reasonable for me to point out its removal. The nitpicking over it began with your response to me doing so, and you have kept it going by taking the worst possible reading of what I write. This is, indeed, how things sometimes go.

Meanwhile, in a branching thread, I had a short discussion with the author of the post I originally replied to, in which I agreed with the points he made there. Both of us, I think, clarified our positions and reached common ground. That is how it is supposed to go.

I did not set out to pick a fight with you, and if I had anticipated how you would take my words, I would have phrased things more clearly.

⬐ dkersten

Not grounds for stopping work[1], but perhaps grounds to explore other avenues[2] to see if something else might yield faster results.

I’m no expert, but my personal opinion is that AGI will probably be some hybrid approach that uses some reinforcement learning mixed with other techniques. At the very least, I think an AGI will need to exist in an interactive environment rather than just trained on preset datasets. Prior context or not, a child doesn’t learn by being shown a lot of images, it learns by being able to poke at the world to see what happens. I think an AGI will likely require some aspect of that (and apply reinforcement learning that way).

But like I said, I’m no expert and that’s just my layperson opinion.

[1] if the goal is AGI, if it’s not then of course there’s no reason to stop

[2] some people are doing just that, of course

⬐ mannykannot

Fair enough, though I do not think the evidence from evolution moves the needle much with respect to the timeline. For one thing, evolution was not dedicated to the achievement of intelligence.

⬐ dkersten

Sounds reasonable.