HN Theater @HNTheaterMonth

The best talks and videos of Hacker News.

Hacker News Comments on
Free Energy Principle — Karl Friston

Serious Science · Youtube · 5 HN comments
HN Theater has aggregated all Hacker News stories and comments that mention Serious Science's video "Free Energy Principle — Karl Friston".
Youtube Summary
Serious Science - http://serious-science.org
Neuroscientist Karl Friston on the Markov blanket, Bayesian model evidence, and different global brain theories
http://serious-science.org/free-energy-principle-7602
HN Theater Rankings

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this video.
Friston on the Free Energy Principle and Markov Blankets:

https://youtu.be/NIu_dJGyIQI

Ok, so less about the flow and nature of information and more about the semantics.

> Seeing the computation as a physical process, processing information "about" details in its environment, highlights this.

Are you familiar with Karl Friston's work on the Free Energy Principle? [0] He touches on how the nature of a living being (computational agent) is to model its external environment and act upon that model. That making decisions based on internal logic requires external stimulus and semantics being analyzed and augmented in sort of a feedback loop.

> The key to all this is appreciating that, once you see semantics and information processing as a matter of physical processes, you can see that the correct semantics can be necessary for producing a physical outcome.

I think I understand what you're saying and it touches on some of my own inquiries. A more concrete example, if you have one, might better help align me with where your thoughts are at regarding this.

[0] https://www.youtube.com/watch?v=NIu_dJGyIQI

jamesrcole
> Ok, so less about the flow and nature of information and more about the semantics.

I'm working on explaining the fundamental nature of information, and arguing that it is fundamentally semantic. I think there's a lot of confusion about what information theory actually tells us. I don't think it actually tells us about the fundamental nature of information. This isn't to detract from its importance, or to say the theory itself is wrong -- I'm saying that the usual interpretation of what it means, regarding information, is flawed.

> Are you familiar with Karl Friston's work on the Free Energy Principle?

Not terribly familiar with it. I watched the video.

I don't think it gets down to the a precise understanding of the fundamentals. I don't think there is a good understanding of the concepts like information and modelling, that it uses. It doesn't seem to provide specific mechanistic explanations of how the phenomena work and how they have their apparent properties.

As a couple of examples, what explanation does it have of how a system may have an understanding of the character of some entity? Where that entity might be something apparently "abstract" or "imaginary"? If the semantics are about mathematical details, what exactly are they about? What are those mathematical details?

But I don't think I can really hope to explain my position here. It requires a lot of explanation, and I'm still working on the explanation.

>> The key to all this is appreciating that, once you see semantics and information processing as a matter of physical processes, you can see that the correct semantics can be necessary for producing a physical outcome.

> I think I understand what you're saying and it touches on some of my own inquiries. A more concrete example, if you have one, might better help align me with where your thoughts are at regarding this.

Imagine a robotic arm that, when a electrical current is sent to it, reaches out and grabs at an area in front of it. If that current happens when there's an object in that position, then the robotic arm will end up picking up the object. If the current happens when there isn't an object in that position, the robotic arm will have picked up nothing. Thus, the semantics of the electrical current has a necessary causal role in the arm picking up the object. And thus we can analyse the causal details to get a concrete understanding of the semantics and its role.

This ties in quite nicely with Karl Friston's Free Energy Principle: https://www.youtube.com/watch?v=NIu_dJGyIQI

There are so many strands of neuroscience that show predictive coding now. There's a really nice example in the use of common vs unusual words in sentences. If you're using fMRI to scan someone while they read these two sentences:

The jam on the motorway made everything slow.

The jam on the motorway made everything sticky.

You'll see a flash of activity when people read the word "sticky" rather than the more common word in that context "slow". The theory is that when everything in the sentence proceeds as predicted there's very little need to update prior expectations, but when you come across something that goes against previous predictions it's a signal that something needs to be actively thought about.

fwip
If anyone is interested in more sentences like these, check out garden path sentences.

https://en.m.wikipedia.org/wiki/Garden-path_sentence

dtech
> The theory is that when everything in the sentence proceeds as predicted there's very little need to update prior expectations

Throwing exceptions is expensive

nvrspyx
In psychology/neuroscience, many refer to this generally as heuristics. It’s astonishing how much the mind relies on these heuristics from aspects like language (as you’ve pointed out) to visual perception. Many think the main purpose is to speed up processing tasks and require less “resources” so that cognitive load is freed up for other things. One of my favorite examples of this is how car drivers seem to fall into a sort of “auto-pilot” mode freeing up resources to think about random things like what you’re going to eat for dinner once you get home.

I think the ELI5 way to look at it is pattern recognition and then using those patterns for not only prediction, much like ML models, but also in other things like physical actions, emotional response, etc.

Many theorize it is that these heuristics are trained and the “path more taken” is the default. It’s when a stimulus doesn’t fit that path that the brain has to decide whether:

1. Is something wrong with this?

2. Is this a new path I need to start paving to speed up future processing?

Whether or not that second choice is made is dependent on lots of things co-processing at the time: Emotion, attention, attitude, etc. How much that path is paved is also dependent on those things.

This idea is fundamental to areas of study like symbolic systems at Stanford [1]

[1]: https://en.m.wikipedia.org/wiki/Symbolic_Systems

blattimwind
> You'll see a flash of activity when people read the word "sticky"

I noticed that my gaze actually jumped back to "jam" when I read "sticky".

stubish
And that flash of activity is also the basis of humor. We find unexpected things like this funny, the reward that helps our brains to analyze and learn from the unexpected situation.
frereubu
Not all humour, but Steven Wright certainly made an entire career out of sentences like this.
aportnoy
Also related: https://en.m.wikipedia.org/wiki/N400_(neuroscience)
wazer5
I'm not surprised neuroscientists can measure it, I about fell over upon reading that sentence.
ta1234567890
Cool! This sounds similar to koans: "a paradoxical anecdote or riddle, used in Zen Buddhism to demonstrate the inadequacy of logical reasoning and to provoke enlightenment."

If you explore the meaning of language enough you'll realize it doesn't really make sense.

Everything is based on context and assumptions that we take for granted when communicating. But if you dig deep enough (and it's really not that deep anyway), you'll notice that pretty much anything you say or write can be interpreted in multiple different ways, which means there's no one absolute meaning to any expression of language.

Meaning comes from interpretation, which is subjective and unique to the person doing it.

Interesting exercise: for a day or a week, make an effort to think how anything that someone else says to you is actually true - asume they are always right and think about how to justify it. It's really amazing, at first it's really hard, but then you discover you can always just interpret what they say in a way that fits your views.

soulofmischief
I think you can make unambiguous sentences: "I walked one mile today." Unless context specifically indicates the very rare possibility that walked is personal shorthand for another activity, it's a pretty unambiguous statement.
chacham15
Even statements like this can become ambiguous depending on the speaker. E.g. is the speaker speaking literally or metaphorically? Maybe the speaker feels like they walked a lot and so they chose a distance which represented a lot to them. Also, is one mile a lot or a little? Maybe its the opposite and they feel like they barely walked. Science makes every attempt to be unambiguous, people do not.
p1necone
I can think of two different ways to interpret that sentence.

1: They went for a walk as a single activity on top of their regular daily walking, and that walk was a mile long.

2. They walked a total of 1 mile throughout the entire day, including regular walking around the house/office etc.

I do agree with the sentiment though, there definitely are plenty of unambiguous sentences you can say in everyday conversation.

Perhaps "my refrigerator is currently powered on" is an arbitrary example that works.

Rhinobird
What if you walked an indeterminate length for an Airline mile?
soulofmischief
That's pretty good. But then knowing that ambiguity, I could craft a statement like, "I walked a mile on the treadmill today."
ta1234567890
Exactly, and if you keep going, you'd notice you need each time a longer, more precise description, which at the same time, paradoxically, opens the door for more ambiguity.

In that example, the first context is English language, then knowing who I is, then what a mile is, and a treadmill and today. All of those are contextual. Even mile. How long is a mile? Like any physical measure, it actually varies, because it is defined relative to something else which varies. The meaning of English language has also varied over time, that sentence might not make any sense to someone reading it in 20 years. And then when is today? May 7, 2019 - which is also a relative measure of time and not even that precise.

And then finally, what if you were lying? Did you really walk a mile today? How can I be sure? And how can you be sure I understand the same meaning from the words you say or write?

soulofmischief
I understand where you are coming from, but a lot of what you just presented isn't actually relevant to the topic of ambiguity from lack of context in language. Some of it just doesn't apply. Allow me to explain...

> the first context is English language

Let's ignore this context. If you don't understand English then you shouldn't be expected to understand an English statement regardless of context.

> How long is a mile?

That depends on what time and space you are in. But that's ok. All such measurements are relative and defined by non-natural constructs. But we can be realistic and assume a mile is whatever our government has decided.

When I tell you "I ran a mile", I'm not telling someone in Australia I ran a mile. I"m telling you that. If we are speaking over the internet it might complicate things, but we only have to resolve the meaning of mile once for our relationship, and all proceeding discussions will use this meaning.

I would argue that this makes the word mile unambiguous in the sense that you are saying. If we had to assert each time what I meant by a mile, then that would be ambiguous.

> that sentence might not make any sense to someone reading it in 20 years

Evolution of language is a separate phenomenon. We don't need to consider that. Only what the language means today right now as I use it. That's the utility of language. For older English texts, we have literature which matches our common vernacular to the vernacular of the time. Foreign translations are out of scope for the same reason listed above. Since we have mappings between today's English and yesterday's English, I would again argue that this is not a source of ambiguity.

> And then when is today? May 7, 2019

That doesn't have anything to do with language. That is a chronospacial coordinate. It represents a certain degree of completion of our orbit around the sun. It isn't a measurement of time at all. It's only relative in the sense that all time and space measurements are relative to the spacetime coordinate of the Big Bang.

Time is inherently a relative concept. We don't have time unless we are comparing (relating) two different frames. We define one in terms relative to the other (motion, state, etc). In a single, non-relative frame of reference time just doesn't exist.

> And then finally, what if you were lying?

Again, this has nothing to do with language. You aren't trying to guess what I'm thinking, you're trying to understand what I'm saying. That's what language parsing is about. Discussion of motivations, misdirection, etc. is irrelevant.

ta1234567890
Interesting. All of the above rebuttals are basically: ignore this because it's context and should be obvious.

Well, that's the problem. It might seem obvious to you now, but it is not obvious to everyone all the time, which is what makes any statement ambiguous.

Additionally, I don't know the purpose of what you are saying. For example, you might say you ran a mile because you wanted to express you were tired, or because you wanted to imply you are healthy, or who knows why, it is definitely not clear from the statement alone what you wanted to convey, which is the main problem.

We feel something internally and then we try to express that in words, hoping that those words will mean the same to the person perceiving them. That process is highly suceptible to noise in many forms, like the other person not hearing you correctly, being distracted, being stressed out, or maybe they speak a slightly different version of your language and some of the words or expressions mean something a bit different to them than to you.

In the end, any statement doesn't have meaning by itself, it only acquires meaning when someone interprets it and understands it in a certain way. That is always a subjective process.

soulofmischief
Honestly I feel like we could have a very deep conversation about this because I certainly agree with a lot of what you're saying.

Language is certainly subjective, but I would argue that we can make certain concessions on what context truly is "obvious" and what isn't. And I think the matter of what level of precision is acceptable enough to consider a statement or idea "understood" is up for debate and is also contextual.

At some point, any extra information is just that, extra information. The core idea may have been expressed fully, even if there are certain small ambiguities or if the listener wishes to draw insight from the statement beyond the basic sentiment expressed.

ta1234567890
Completely agree with you.

Language is an incredibly useful tool and it works very well for most of our daily lives and many important things without thinking too much about it.

It's just very interesting to me to see how something that seems so solid, in some ways is much more fluid and fuzzy.

Thank you for keeping this thread going, really enjoyed discussing this.

Karl Friston's free energy principle says something similar - https://www.youtube.com/watch?v=NIu_dJGyIQI
I found this interview with Karl Friston helpful to understand free energy principle from a high level:

https://www.youtube.com/watch?v=NIu_dJGyIQI

HN Theater is an independent project and is not operated by Y Combinator or any of the video hosting platforms linked to on this site.
~ yaj@
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.