HN Theater @HNTheaterMonth

I've been working on VR and AR stuff for about 8 years now. I started doing it full time about 4 years ago. A lot of what I know has been from just being in the zeitgeist as things develop, so unfortunately I don't have a "reading list" to get people started.

I have been trying to start a reading list, but it's woefully incomplete. I'll copy the content here (I don't have it publicly online yet). I'm primarily centered on VR, but I've also done a lot of AR work. I think good application design is very similar in both. or rather, all the "bad" apps I talked about out there are similarly bad in the failure to take the immersiveness of the experience into account. But I do think there is a lot of overlap in terms of needing to take less of a traditional, compartmentalized application mindset and start thinking about immersive software as more akin to clothing, overlays on top of the world.

Kent Bye's "Voices of VR" Podcast (Site: https://voicesofvr.com/, Twitter: https://twitter.com/kentbye?s=20). Kent Bye has been a singular voice in the VR and AR community for the entirety of the contemporary VR movement. He brings a philosophy and social impact perspective. I think a lot of application design--immersive or otherwise--doesn't take human factors into account often enough.

Road to VR (Site: https://www.roadtovr.com/, Twitter: https://twitter.com/RtoVR?s=20) is also a very long-standing blog on all things AR and VR. They are more focused on gaming, but they also cover industry trends, new hardware, and companies

John Palmer has a few blog posts covering spatial interfaces that are very insightful (https://darkblueheaven.com/) https://darkblueheaven.com/spatialinterfaces/ https://darkblueheaven.com/spatialsoftware/

This Medium post by Douglas Rushkoff talks about some of the problems with digital media, which I believe VR can help solve (https://medium.com/team-human/digital-media-still-isnt-very-...)

Jaron Lanier is one of the "fathers" of VR, part of the "first-wave" of VR work in the late-80s/early-90s. The Verge did an excellent interview with him (https://www.theverge.com/2017/12/8/16751596/jaron-lanier-daw...). You can get to his books from his website (http://www.jaronlanier.com/), which are all excellent treatises on humanity's relationship to technology.

Incidentally, here's Kent Bye interviewing Jaron Lanier (http://voicesofvr.com/600-jaron-laniers-journey-into-vr-dawn...)

Liv Erickson has an excellent blog on technology that covers a lot of issues in VR, accessibility, and machine learning (https://livierickson.com/blog/). In particular, "6 Questions to Ask Before Diving Into VR Development" is great primer on VR concepts (https://livierickson.com/blog/6-questions-to-ask-before-divi...)

Tom Forsyth (Twitter: https://twitter.com/tom_forsyth) has an excellent blog post about different technical aspects of the optics in VR systems (http://tomforsyth1000.github.io/blog.wiki.html#%5B%5BVR%20op...)

Jesse Schell's article "Making Great VR: Six Lessons Learned From I Expect You To Die" is a little old but still excellent (https://www.gamasutra.com/blogs/JesseSchell/20150626/247113/)

This is an excellent article on the importance of audio in immersive applications (https://arinsider.co/2019/10/02/sound-ars-unsung-modality/)

This is an interesting video made by a man who spent a whole week in VR, eating, sleeping, working, and living with a VR headset on 24/7 (https://www.youtube.com/watch?v=BGRY14znFxY)

⬐ yomly

This is fantastic - thank you.

How did you find yourself in this field?

⬐ moron4hire

Part 2:

That was right about when Google Cardboard hit. That was 2014? I saw it during a Google I/O livestream and just so happened to have a few lenses left over from some still-photography experiments involving lasers and... never mind, another thing that went nowhere, I just had some lenses around. I quit watching the Google I/O stream and immediately hacked together a new cardboard box viewer with the lenses.

Saw Versailles, which I have never been to, but I have been to Linderhof Palace in Bavaria, which is modelled after Versailles. Saw the Galapagos, where my wife and I had just spent our honeymoon about a year or so before. I immediately saw the experience was closer on the spectrum to really being somewhere than it was to seeing it on TV.

So I'm sitting there, this cardboard box in my hand, this powerful smartphone in my hand, thinking to myself "I have to do this, I have to make these things". But all I knew was C# and JavaScript. Unity was garbage at the time. It's still garbage, but also back then. I had tried my hand at native app development and just didn't have the patience for Xcode or Android Studio. So I started hacking with Three.js and JavaScript. I ended up making the first JavaScript API for VR, releasing the first version about a month before the first announcements of WebVR.

Then I saw RiftSketch (https://www.youtube.com/watch?v=db-7J5OaSag). It blew my mind. Started chatting with people. Brian Pieris talked about wishing he could get syntax highlighting into the app, and complained about how he had to use CSS 3D transforms to position the box on top of his WebGL view. At the time, I recognized how early everything was and how primitive the tools were. So I thought, if I could make developer-oriented tools that made making VR easier, I could make something out of that. I thought Brian could be my first user. So I made a RiftSketch clone, added it into my WebVR framework, and that became what was eventually called Primrose.

I got a small amount of internet fame out of Primrose. People started recognizing me at conferences. A "startup" hired me to be their head of VR. That turned out to be a different kind of hell. It crashed and burned after about a year. We had a kid and another one due any day, and we were completely out of money. I thought he VR dream was finally over. Started applying to jobs back in web and DB work.

In the mean time, I had just started working in Unity at the startup, I had all this time on my hands, and Unity was offering full-access to their learning materials for the first month for new customers. It was clearly designed to go through in 3 months, but I churned through all of it in a month. Then one of the folks that I had hired on at the startup to work on VR stuff made a connection for me at a gigantic, multinational consultancy, for their Unity dev team. He thought I was pretty good even before I learned Unity properly, so I breezed through the interview. It was also the most money I'd ever been paid. Seemed like a huge win!

Then the reality of giganto-consultingware companies set in. You've not seen office politics until you've worked in an organization that runs under a partnership model. It wasn't exactly the worst job experience I've ever had, but it definitely ranks up there. But it basically got me a ton of Unity development experience. They mismanaged the hell out of the team and eventually had to layoff half of everyone. I also ended up meeting a few folks along the way who got me an interview at my current place.

I'm now the head of VR at a foreign language instruction company. Our main client is the military's foreign language school, the very same place where my parents met and then shortly thereafter had me. Things are going well. The company is stable. It's on its second owner, who brought it back from bankruptcy in the early 2000s. I report directly to the president of the company. He thinks the world of me and lets me do whatever I think is best. We have weekly meetings where we geek out about video games and VR. I just got to hire my first employee to work on the project with me. On a weekly, sometimes even daily basis, the company does something that proves "our employees are our biggest asset" isn't just a platitude for them. It's amazing. I've never worked anywhere this nice before.

⬐ yomly

Sorry I never saw this. This both was captivating and inspiring!

Thank you for sharing your story, it was quite the journey and it is uplifting that you were able to eventually culminate all the dotted things you've worked on out of passion into something that you're now sought after for!

⬐ moron4hire

Part 1:

That's a funny wording, because it was actually going through the process of "finding myself" that I ended up in immersive software. It simultaneous feels like it came out of nowhere, but also that my entire life prepared me for it.

I'd always had a fascination with 3D imaging as a kid, both stereo graphics and holographics. I don't know what was going on in the late 90s, but there were a lot of cyan-magenta anaglyph comic books at the time, and a lot of Marvel comics were doing holographic overlays for special edition covers. I read every book I could get out of the local library on optics and holographics.

Through college, I studied a lot on computer graphics. It was fascinating to see how much of the optics stuff I learned as a kid applied to my studies in college. But also learned I hated 3D modelling. This was also around the time the first stories about how terribly exploitative of employees the game development industry started coming out. And it was also around the end of the dotCom bust, though as I was graduating we didn't know it was the end yet. So I ended up going into web and database development, because one of the other things I had done as a kid was learn HTML and JavaScript.

My first job was in Geospatial Information Systems. I did a lot of work in map rendering and web-based rendering. A lot of study of projection systems. This was pre-HTML5, so I made a lot of filthy hax in JavaScript to make DIV tags look like pixels. Bresenham's Line Algorithm, which everyone in else in college thought was a waste of time to study, was my savior. I ended up making a JavaScript library that was essentially Java's Graphics2D API, rendered in DIV tags.

I had a lot of trouble with my relationship to work. My first job wanted to move me out to the middle of nowhere, gave me very little notice on the decision. It was terrible, so I also quit without notice. My next job was OK, but the work was mind-numbingly boring and I wanted to move away from home. Found a job near Philadelphia, near some friends. That job was at a company that was basically a cult, though there was one highlight where code I wrote helped catch a criminal by proving there was no way they were delivering the products they said they were because the timestamps between stops were too short for the distances they had to travel. My next job, for a company that makes used car database websites (so already soul-sucking to start), had all the anti-corporate, pro-developer surface details I thought were the problem with my previous jobs. I hated it even more, the constant Nerf gun battles and the lack of focus, at the same time as being constantly brow-beaten for not getting anything done in that environment. I quit after 3 months.

I thought the problem was consulting. I wanted to get out of consulting and into product development. A friend got me an interview at a company working in home-automation devices. It was a huge paycut, but I was supposedly "getting in on the ground floor" of a "hot startup". Turned out the "small startup" was actually "a poorly managed company that couldn't find a market fit and did some shady deals to rebrand every 3 years to escape their reputation". I ended up right back in web and database consulting work there. The systems were terrible. Most of my work was manual data entry and fixing stupid timing bugs in the device configuration tool. I fixed everything, made tools to automate the data entry, made simulators of devices to speed up testing of the configuration tools, fixed all the stupid code that made bad assumptions that led to comms race conditions. I got fired because I refused to work overtime. I refused to work overtime (unpaid, mind you!) because there was no overtime work to do. I got told that I needed to log 60 hours a week no matter what. I told my boss that if she wanted to lie about the work I was doing, she could fudge the invoices to the client herself and leave me out of it.

I thought maybe I hated programming. I eventually learned that I didn't hate programming, I just hated the people I was working for. I decided I was not going to look for a "job" and I was going to stick to being independent for as long as possible. I had joined a hackerspace when I first moved to Philly, started using it as my office. I tried starting a t-shirt printing business. I tried selling photography prints. I built a couple of museum installations on contract, these Arduino-based things. I tried making music-teaching toys. I did pyrotechnics on an indie film. It was all over the place.

One of those things I tried to build at the hackerspace was basically Google Cardboard, 4 years before it was a thing. When smartphones first came out and started to show promise with 3D rendering and high performance motion sensing, I built my first "phone in a cardboard box" stereo viewer. It... worked... for values of "worked" that include massive headaches. I didn't have lenses, the box was relatively long to be able to focus and fuse the images, but seeing stereo animation of my own making for the first time was amazing. This was around 2010 or so.

At the same time, I also started playing with different camera-oriented apps that took motion sensing into account. Built a few different apps that could help you take stereoscopic photo pairs and render them in side-by-side or color anaglyph. Tried to build a turn-by-turn direction app and a point-of-interest discovering app using Google Maps data. That sort of stuff. Discovered sensor data and rendering were still not quite good enough to make a good experience.

Eventually, a friend from the hackerspace got me an interview at a company that makes tilt sensors. I thought the job sounded boring, but I needed the money. They hired me on a 3-month contract-to-hire, and when the 3 months were up, I asked them if they would be willing to keep me as a freelancer. They agreed. I spent 3 years there, the longest I had spent anywhere up to that point. I met a girl. She lived in DC, so I started pushing to let me work remote so I could travel to see her. They agreed. I eventually moved to DC and became 100% remote, which they were cool with. I even got to where I had hired a few people part-time on my own to help out with the work. They thought it was great. I actually loved it, for a time. Someone in the company got a bug up their ass about me having my own employees. I guess someone parallel to the guy I reported to got the owner convinced that it was a "security risk" or something. I don't know what, all I know is that this other guy took over and I got slowly squeezed until there was only enough work for myself. I was right back into having a bad boss again, so I was on the lookout for an exit.

⬐ moron4hire

I just tried to reply but HN complained that my comment was too long. Never seen that one before. I will need to reply in two parts.

⬐ buboard

Cant watch the vid. It s been 2 hours. Can anyone tell me what happened?

⬐ llbowers

He basically spends a week in VR, though allowing himself a passthrough - which is, from what I gather, a video stream from a camera within(?) the headset that allows him to "see" his room.

He does various games and activities and interacts with people through various chats. He has his peaceful moments and energetic ones, such as working out, but grows bored and lonely by the end of it.

Interesting to think if there were more immersive/interesting things to do within VR would he have felt the same way? Or at least not as quickly?