HN Theater @HNTheaterMonth

It just so happens I was thinking about spatial org just yesterday, that it would be so nice if I can have 'villages' of tabs on a (geographical) Map. Amazing co-incidence that I woke up to bonsai browser notification!

You might find The Humane Representation of Thought[1] by Bret Victor[2] useful, where he demonstrates by example, how existing tools/software (generally speaking) doesn't utilize all the 'superpowers' that we acquired via evolution.

---

[1]: https://www.youtube.com/watch?v=agOdP2Bmieg

[2]: http://worrydream.com

Edit - grammatical fixes

I'll be contrarian and throw my opinion on top of the pile, why not.

I've never found most of the arguments about the UNIX philosophy or plaintext super compelling. Some people love to wax poetic about "composing together small programs" and "being able to use one toolset to accomplish many things." In my experience this works really great for systems administration (the activity the paradigm was designed around - big surprise) but not very well for other things. Sometimes I want to take a picture. Sometimes I want to query a sqlite database. Sometimes I want Powershell objects. Life is not systems administration, and neither is computing.

Don't get me wrong, I have emacs open all day every day at work. But when I hear people talk about using plaintext for everything, I think about Bret Victor's rant about the inhumanity of keyboards and screens:

https://www.youtube.com/watch?v=agOdP2Bmieg

The first 20 mins of Bret's talk (https://www.youtube.com/watch?v=agOdP2Bmieg) discusses representations quite well in a very similar context / framing.

The challenge with roman numerals was definitely computing with them and really "thinking with them". Abacuses helped, but it was still a poor UX in many ways and it took lots of training & practice to get familiar.

Algebraic notation also requires training & practice but has a much quicker learning curve. Almost every literate adult uses equations in some form or another on a weekly basis. Both the medium (pencil & paper) as well as the representation (algebraic notation) were critical to this (as well as other things like education system / literacy, etc).

The benefits & disadvantages of different representations is massively understudied and misunderstood. Data visualization, semiotics, and some other fields focus a bit on this but still fall short in many ways.

⬐ Koshkin

> Algebraic notation ... has a much quicker learning curve

Not true - many people still struggle with algebraic notation.

One of the goals of Dynamicland is to foster the creation of new representations that take advantage of + live across more sensory modalities. Bret calls them modes of representation, but there are similar phrases in learning & psychology literature.

Most representations are haphazardly created (math notation, periodic table of elements, etc) by (often) singular experimentation. Representations are poorly researched and understood (there's no branch of science dedicated to them). If representations were studied more closely, we could get much better at deliberately creating representations that are more humane. When the scientific method is applied, we can then apply the engineering method to create new things from those principles. Bret's talk is a great, accessible overview of these ideas from his context (https://www.youtube.com/watch?v=agOdP2Bmieg).

To vastly oversimplify things, one way to improve accessibility is to parallelize representations that live in just 1 or 2 sensory modalities to all of our sensory modalities. This way, math notation could be made accessible to & equally powerful to the blind (as it is to those who can see).

I empathize (a little, it would be naive for me to assume I can 100% empathize) with your reaction along the axis of accessibility. What Dynamicland looks like right now isn't what it's going to look like later. Right now, you may see Dynamicland as text boxes but now distributed in a room. In 50 or 100 years, there may be no text AT ALL and you can use the senses available to you to interact with information.

To paint another perspective, I would argue that current computing environments are VERY in-accessible for many people. Laptops and desktops only tap into our visual and symbolic channels, with some tactile feedback (but quite minimal, we know where "X" is on the keyboard b/c of spatial reasoning not b/c the X key gives us specific/unique tactile feedback). Humans who thrive in spatial and kinesthetic channels are debilitated on a modern computer.

⬐ miki123211

Actually, if the tools become real tools (instead of a page representing a layer, some 3d printet thing with a specific look and feel) this could, theoretically, become accessible for the blind. People with moving impairments will have it much harder, though.

⬐ mwcampbell

> Humans who thrive in spatial and kinesthetic channels are debilitated on a modern computer.

Interesting; this isn't something I know anything about. Is it not feasible to develop alternative hardware and software that adapt our symbolic notations for these people, while still enabling those of us who practically require symbolic notations to collaborate?

⬐ skadamat

Possibly! That def could be interesting.

In general, we're just not great at processing symbolic representations. We haven't evolved to do so (unlike spatial, visual, auditory, etc senses for matching representations) and have only manipulated symbols for a few hundred years. I mean true abstract symbols, not hieroglyphics or other old symbols that still mapped to language / were still quite visual.

I believe it could be more compelling to figure out how to re-represent things instead of just scaling up symbolic representations. That helps make it more powerful and accessible, for everyone.

A good example here that Bret mentions in his talk is Roman numerals -> Arabic numerals. Or even typing paragraphs of text to Arabic notation for equations (y = x^2 + 1 used to be PAGES of written text). People used to think only super educated elite mathematicians could grasp algebraic ideas. But after they were re-represented, we discovered almost all 8 year olds could grasp the ideas!

⬐ None

None

⬐ miki123211

Such work has been done, see Braille or the ASTER system http://www.cs.cornell.edu/home/raman/aster/aster-toplevel.ht...). ps. Sorry for the previous comment (already deleted), I've accidentally pasted the wrong thing from my clipboard.

I had the pleasure of visiting Dynamicland and hanging out with Peter Norvig, Nicky Case and many others. There's a few things that stuck out for me:

- The community! Most people I interacted with were pretty inter-disciplinary. Some came from an education heavy background, others from a more programming language background. Some from interface design, others from physical installation or video game background (and everything in between). Everyone had their own degree of skepticism but were excited by the discussions that were happening (a sign of a good research culture!).

- The representations: What excites me the most about Dynamicland is that it's an environment for fostering unique representations of ideas. The cultural forces & ideas (social programming, remixing, visible state / code at all times, 3d environment, etc) baked into the space encourage the experimentation & creation of new ways of representing complex ideas. Some scattered examples here - https://twitter.com/dynamicland1?lang=en

- Removal of artificial barriers: When programming on a computer these days, there's so many barriers to doing simple things. The amount of code that exists to do virtual actions in a virtual world is MASSIVE, and acts as a huge barrier. This is something Bret talks about at the end of his interview on Postlight (https://postlight.com/trackchanges/podcast/computing-is-ever...). Because code / programs in Dynamicland are embodied in the room, you don't need code to move a dialog box or a slider around. You just moved it yourself. This is super powerful and it means your code can focus on the actual computation itself, not on virtual UI movement. Eventually, Dynamicland will have robotics to automate moving of objects, but this already is a great start.

- Moving around: Moving around, even if it's just around a table, is AWESOME. We're so used to sitting (or standing) at a desk and staying still that we don't get to take advantage of embodied intelligence. We think in SO many different ways, and our "thinking" doesn't just happen in the head. Our arms, legs, stomach, and feet all contribute to the thinking process. Combine that with multi-sensory representations that live across all of these channels and you can explore a thought & idea space SO quickly and uniquely. This is so hard to describe and under-rated. This link (http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi...) and this talk (https://www.youtube.com/watch?v=agOdP2Bmieg) attempt to do these ideas justice but it's quite hard to transfer this context!

⬐ picometer

Is it possible to visit Dynamicland as a random interested person? I'm on their mailing list and couldn't make the open house last spring - maybe they'll have another one at some point.

⬐ skadamat

I made a small donation to Dynamicland and then requested to come by as part of that.

⬐ copperx

How big does that donation have to be, if I may ask?

⬐ skadamat

I have no idea to be honest. I just gave a nominal amount, think it was like a hundred dollars. I would happily give a LOT more if I had stronger financial means / if I thought it would actually have an impact.

It seems like they're mostly seeking funding from larger institutions, as crowd-funding such a large research effort isn't really feasible or sustainable.

on one level, yes

on another, they're down right scary and overwhelming, or at least dumbfounding and perplexing

..thinking specifically about The Humane Representation of Thought https://www.youtube.com/watch?v=agOdP2Bmieg I had to stop it half way through.. :-/

see also Seeing Spaces https://www.youtube.com/watch?v=klTjiXjqHrQ and Media for Thinking the Unthinkable https://www.youtube.com/watch?v=oUaOucZRlmE

Really excited to go through all the other talks listed here I'm not familiar with!

Three talks that I tend to reference a lot and have had a huge impact on the way I think about software:

The Soul of Software: Avdi Grimm - https://youtu.be/IgbHzFb1hGw

The Humane Representation of Thought: Bret Victor - https://youtu.be/agOdP2Bmieg

How to Program Independent Games: Jonathan Blow - https://youtu.be/JjDsP5n2kSM

Highly recommend Bret's 2 talks for context:

[1] Seeing Spaces: https://www.youtube.com/watch?v=klTjiXjqHrQ [2] Humane Representation of Thought: https://www.youtube.com/watch?v=agOdP2Bmieg&t=1913s

⬐ westoncb

I saw them both a while back, and reviewed the poster for Seeing Spaces just now. However, I haven't been unable to find anything that addresses the issue I brought up.

I agree very much with most of the general principles Victor talks about (you can see my own extensive work on making program behavior visible here[0] for example)—but when it comes to the space concept, I have difficulty finding something concrete about how the space would be used beneficially. I get the flavor of this idea that your workspace should be 'extension of your body', but at this point, years after becoming familiar with it, I'd like to get more concrete on the subject. Dynamicland/Realtalk certainly are more concrete—but my thoughts on it are reflected in my first comment.

[0] http://symbolflux.com/projects/avd

⬐ DaniFong

visit and donate. they need help.

it's foolish to speculate on the possibilities of a new medium on this extraordinarily limited medium. i can't even embed images here.

⬐ westoncb

I'm pursuing similar lines myself, and am only able to work in spare moments due to lack of funds, so I'm in no position to donate.

> it's foolish to speculate on the possibilities of a new medium on this extraordinarily limited medium

I'll be happy to be shown otherwise, but I haven't yet found any counter-example to the idea that the 'verbal medium' is sufficient to describe any principle. So my original question still remains: is there some principle which would allow them to achieve similar generality to traditional computing without using some kind of re-combinable primitives?

Edit: to be more specific: without that, how could this ever compete with AR? People could build their own spatially interesting, room-scale workspaces without having to find physical materials to build every application out of. The fact that it can deal with actual physical things has an appeal to it, but I can't see any way of escaping the tradeoff of a massive loss in generality.

⬐ Glench

> how could this ever compete with AR?

"How could the written word ever compete with pictograms?" It's a different medium with real benefits and tradeoffs. Right now for you it's all theoretical, but one day hopefully you'll be able to see for yourself if the medium has the time to grow, breathe, expand, and develop. Funding Dynamicland means that we as a species have the option of seeing how that medium might work if it was allowed to flourish.

⬐ westoncb

We'll, I wish you guys luck and sincerely hope this removes some desk time from my life :)

It's going to be entirely different I feel. Instead of manipulating graphics on a flat screen, focus will probably be on manipulating physical objects on a table / in a room.

Watch his 2 talks that highlight the thinking and inspiration behind the RealTalk project:

https://www.youtube.com/watch?v=klTjiXjqHrQ https://www.youtube.com/watch?v=agOdP2Bmieg