HN Theater @HNTheaterMonth

The guy mentioned: https://www.youtube.com/watch?v=qSKBtEBRWi4

⬐ kitotik

Good lord.

I guess I knew this was possible, but seeing it in practice is something else entirely.

I wonder if it’s output unipolar 0-10v or bipolar -5/+5 and how that “feels” to the user.

⬐ elihu

Most CV signals can be positive or negative. For instance, there's a standard for pitch CV that's 1 volt per octave. 0 volts might be middle C (whether it is would vary depending on the synth and/or position of tuning knobs), and -1 volt would be an octave below middle C. (There's another hz/volt standard used most notably by the Korg MS-20 where 0 volts == 0 hertz, but it hasn't caught on as widely.)

If the prosthetic adapter only outputs positive voltage, you could just turn the knob corresponding to the CV being controlled to its lowest setting (assuming there is a knob and it is summed with rather than bypassed by the jack).

⬐ kitotik

Right. I guess my question is more around the actual sensation and expressiveness when using it.

Is it simply “the more I tense up the muscle the higher the voltage”?

Is it able to be more or less expressive than say a Cv joystick or a high quality ribbon controller.

⬐ gcatalfamo

This is so fucking amazing. The “mind control” DJ is definitely a concert I would attend. I wonder how much extendable is the prototype.

⬐ bnegreve

I know I'm not fun, but everything is "mind control". When you turn the knobs on a synthesizer you control the music with your mind (not your hands). You can also mind control the music with other devices such as an xy-pad https://en.wikipedia.org/wiki/Kaoss_Pad, if you've ever tried it, the result is about the same as what you can ear in the video.

So this is "mind control" in the same way that a trackpad can be used to mind control the pointer on the screen. It's a device that allows you to control two continuous variables with (almost) no physical effort. And yes it's connected to the nerves, but what can you do with this that you can't do with a joystick or a pad?

⬐ lolc

Haha yes! The funny thing is that with fingers, one can still observe the nerve signals getting turned into CV. With this guy, it's not possible.

⬐ sevencolors

Oh. Thanks for, "explaining it". And also way to miss the point completely.

⬐ bnegreve

Oh come on it is not that obvious. At least it took me a while to realize that this was not a superpower.

But you're right, I know I'm missing the point. I don't care, I can't help, I need to know: can this guy do something that I cannot do with a trackpad?

⬐ gcatalfamo

It’s obvious we didn’t truly believe about “magic” mind control. He’s a handicapped person that can do something truly amazing thanks to tech skills.

Even though we all possess the same electric signals let me tell you. Unless you can chop your arm off you will never be able to do the same thing with your bloody trackpad.

⬐ bnegreve

When I saw the video, I thought "wow there are so many crazy things that I could do with this". Now I realize: probably not so much that I cannot already do with a bloody trackpad. This is just something I wanted to share.

Of course this guy's story is inspiring.

⬐ pdkl95

Wow, that's a really cool project! I love the modular design of the prosthesis that allowed the hand to be quickly exchanged with the CV output module. I wonder how many independent signals you could learn to control using this type of interface? I suspect at least ~5 per arm (>=1 per finger), similar to the fingering used for piano/guitar/etc?

While it isn't remotely as cool as controlling CV with you mind... someday I should convert my Nintendo Power Glove into denkitribe's SynthGlove[1] MIDI controller.

[1] https://www.youtube.com/watch?v=qpLkX7X37D4

⬐ twomoretime

I was thinking, I'd consider hacking off an arm if I could control a machine with the stub, and then I realized, one likely does not need to remove any appendages to interact with such a mind body interface. The future is coming.

⬐ type_enthusiast

It's happening! I'd say we're no more than five years away from a computer interface that integrates with your existing fingers.

⬐ peteretep

> a computer interface that integrates with your existing fingers

Like ... a keyboard?

⬐ lozf

Just wait 'til you guys see the Theremin from the 1920s!

⬐ type_enthusiast

Yeah, that's the joke.

⬐ twomoretime

Imagine how much faster you could type if you interacted through muscle twitches or directly through brain impulses. No latency from pressing/releasing keys.

This would be a step beyond a keyboard. Imagine letting kids learn it while they're young. Your WPM is limited by the speed of your mind!

⬐ klmadfejno

> Your WPM is limited by the speed of your mind!

I mean... it already is. This tech is highly unlikely to surpass keyboards for a few reasons:

1) It needs to be customized for each user.

2) It needs really special hardware to read an accurate signal, especially if there's no open channels to connect to via a missing limb.

3) The accuracy / speed tradeoff for muscle twitches vs. touch typing can't possible be that good. The signal to noise ratio of brain impulses is bad.

4) The limiting factor on wpm typed for any original work of non trivial length is going to be your thinking of the idea, not your typing of it.

5) Imagining the tech worked perfectly, such an interface would probably be very mentally draining to use. Typing out a passage is easy to imagine doing, but editing existing text? I feel exhausted just thinking about it.

⬐ bnegreve

I know I'm not fun, but I think this is "mind control" in a the same way that a trackpad can be used to control the pointer on the screen. It allows you to control two continuous variables with (almost) no physical effort.

So yes, this is mind control, but you can also mind control the music with other devices such as an xy-pad https://en.wikipedia.org/wiki/Kaoss_Pad, if you've ever tried it, it does about the same as what you can ear in the video.

Don't get me wrong it is a very cool project, and it is useful project for the author of the video, because his prosthetic hand is not precise enough to control the knobs on the synthesizers, or a pad. But I don't think we should see it as a new power for humanity.

⬐ nwsm

Awesome. I wonder how much control he has over the output. I assume it's easy for him to hit min or max, but can he be accurate in the middle?

⬐ koheripbal

I imagine each of the sensors are binary - either picking up a signal or not - so it probably depends on what "speed" he has set the "knob turning" to be, and he sets it in the middle by holding on a sensor for a mid-range amount of time.

Most fine precision motor skills are made with a orchestra of opposing muscle stimuli.

It'll be interesting one day to see fully-integrated humans with these digitally controlled devices. I imagine the human brain will be easy to train to operate, say a cell phone, especially if it's integrated at a very young age.

⬐ lilSebastian

https://news.ycombinator.com/item?id=22342314

⬐ kylek

This is so rad- I love that the prosthesis designers are so excited to work with him on this. Really need to ditch those cables for bluetooth though.

⬐ zadler

I wonder how many channels he can get up to.

⬐ alexgodin

This is exactly the type of content I come to HN for.

⬐ sevencolors

That is fucking rad.

⬐ floatingatoll

Previously on HN (46 comments, 2 months ago):

https://news.ycombinator.com/item?id=22342314

⬐ lostlogin

The below comment there from horentis got me thinking.

> This is incredible. I think the next step from here - technology wise I mean - is figuring out how devices can tap into this without needing to lose a limb.

Accessibility mode, but not as I have seen it before.

⬐ sean2

From what I got out of his explanation, is that his prosthesis listens to the nerve signals that would control the muscles of a left hand (which he doesn't have). So, I think you could hook up a similar detector to your own arm for similar effects, its just that your arm would twitch along with your commands.

I suspect the reason we don't see these built for able-bodied people is that its easy to contact the fingers to buttons and knobs and read the muscle activations through these cheap, well-understood pieces of hardware. Might be slower than his input method though.

⬐ Thorentis

This is incredible. I think the next step from here - technology wise I mean - is figuring out how devices can tap into this without needing to lose a limb. Implants that can read the signals from a limb to control something else could be the bridge in between what we have now, and brain-computer interfaces.

You know those devices that severely disabled people like the late Stephen Hawking used to interact with his computer? They require only a very small amount of muscle movement to control a switch etc. What if an implant could be put directly into a muscle or a group of muscles (or a a nerve ending somewhere more suitable) so that a tiny movement could trigger some action? It would essentially give humans a set of re-programmable multi-function buttons. Connect it to your phone over bluetooth, and you can setup an action to be triggered by it. How would you filter out regular/involuntary input vs. intentional input? Not sure! But this avenue of development is exciting.

⬐ wefarrell

Already exists: https://spectrum.ieee.org/the-human-os/biomedical/bionics/mi...

⬐ pmoriarty

Another cool device, which is not an arm and is for input rather than control, is one that lets people see through their tongues.[1]

It's currently targeted at the blind, but I'd imagine seeing people could use it just as well if it picked up, say, infrared, ultraviolet, x-rays, radio waves, ionizing radiation or virtually anything else normally undetectable to human senses.

The potential of human sensory enhancement is truly staggering.

[1] - https://www.newyorker.com/magazine/2017/05/15/seeing-with-yo...

⬐ emiliobumachar

If it was cheap and easy enough, I'd seriously try it as a sighted person, to read while resting my eyes, which routinely get strained by the end of the day. Possibly decoupled from the camera and getting the signal from an adapter which itself gets text.

https://www.wicab.com/brainport-vision-pro

"Ready to buy or try a BrainPort Vision Pro? Fill out the patient survey here to determine if you are a good candidate. A Certified Trainer will contact you to discuss the next steps!"

Oh, never mind... I got from the quoted text that it would probably be prohibitively expensive, and that if I tried to find out more I'd probably divert professionals from helping actual blind people in need of this tech .

⬐ pmoriarty

It costs $10,000.

I wonder how hard it'd be to engineer something similar for a lot less money, though.

It just looks like a tiny camera mounted in regular sunglasses and connected to a computer which then translates the pixels from the camera to electric impulses on a grid of electrodes on a "lollipop" paddle that goes in to the mouth.

The camera, computer, and software translating the signals seems like the easy part. The only tricky part would be the electrodes, as those would have to be spaced pretty closely and create enough of a signal to be felt while not interfering with each other and at the same time being safe to put in to the mouth and not be affected by the saliva in the mouth, not taste bad, resist being accidentally bitten on, and probably some other engineering challenges I can't think of right now off the top of my head.

It's definitely an interesting challenge.

⬐ kregasaurusrex

It's great that the representatives from the company sent him the relevant parts of the schematic to help him connect his arm to the interface board. Products aren't often built with accessibility with mind and seeing DIY projects like this inspire better solutions for these groups.

⬐ pmoriarty

Not only that, but it must be incredibly rewarding to work on something that actually helps people. In addition, they got to work on something that's really cutting edge or even futuristic in a way that most technology work isn't.

It's a huge win all around, and the world be a much better place if more people in tech did stuff like this than figuring out more effective ways of getting people to click ads or spy on them.

⬐ caymanjim

The thing that caught me the most about this video has nothing to do with the hack (which is awesome). What impressed me was how natural the prosthesis is, and how naturally he moves with it. Specifically, he grasps his non-prosthetic wrist with his prosthetic hand, casually, while he's talking. It shows how he's able to use the prosthesis without thinking about it. That and the fact that the prosthesis has the same dimensions as a natural limb, and doesn't require any bulky power attachment. I'm guessing this is standard now, or at least not experimental. I didn't realize prosthetics had advanced this far.

⬐ holri

"the fact that the prosthesis has the same dimensions as a natural limb, and doesn't require any bulky power attachment."

This is standard in arm prothesis at least since the 1960ies.

⬐ caymanjim

For powered ones, vs purely manual? I guess it doesn't need a lot of power.

⬐ holri

Powered: https://eduard.horvath-vienna.at/ot2003.140-141.pdf

⬐ rad_gruchalski

This is incredible, indeed. And must be well understood technology by now. There is for example this man who build prosthetics with lego: https://www.reuters.com/article/us-spain-lego-prosthetic-arm....

⬐ pmoriarty

Another interesting video along these lines: [1], of a girl with two proshetic arms.

[1] - https://www.youtube.com/watch?v=4eMQuhxNDJM

⬐ pmoriarty

In case anyone's wondering what that cool synthesizer that he's using is, it's a Eurorack[1] modular synthesizer[2].

If you are at all interested in this, I'd strongly encourage you to check out Muffwiggler[3], a huge forum for modular synthesizer enthusiasts.

Also useful is the Modular Grid[4], a gigantic database of thousands of amazing modules (and which also lets you design your own modular setup using these modules).

VCV Rack[4] is also great for playing around with software versions of some of these modules, which can be pretty expensive (the guy in the video has likely invested thousands and thousands of dollars in to his Eurorack setup, while VCV Rack is free). On the other hand, you can also build your own Eurorack modules (see the DIY forum on Muffwiggler for that), which can be a lot cheaper.

Something else I was thinking about was that if the main problem the guy in the video was trying to solve was that the knobs and switches on his modules were too difficult to manipulate with his prosthesis, then he could have tried the HackMe Vectr[5][6], which would have let him simply wave his whole hand around above the module to send out three voltages from his hand movements in three dimensional space. Mind control is, of course, way cooler, and doesn't require movement at all.

Finally, I was also thinking about how his modular interface might have been designed with opto-isolators, to minimize the risk of voltage inadvertently being sent back from the modular in to his arm, should he plug his cable in to an output rather than an input. Someone with more of an electronics background than me can probably comment on whether this would be a good idea.

[1] - https://en.wikipedia.org/wiki/Eurorack

[2] - https://en.wikipedia.org/wiki/Modular_synthesizer

[3] - https://muffwiggler.com/forum/

[4] - https://vcvrack.com/

[5] - https://www.youtube.com/watch?v=HSoVBz-1eQ0

[6] - http://hackmeopen.com/Vectr/

⬐ mxmilkb

See also the JACK audio and LV2 plugin form of CV - https://llllllll.co/t/jack-lv2-cv-ports/29428

⬐ tomduncalf

Very cool, I didn’t know this existed! Is it Linux only?

⬐ mxmilkb

Both JACK and LV2 are multiplatform, but there are Windows build problems for JACK atm. I'm unaware as to the current state of things regarding macOS. However, Carla is multiplatform can do it all internally.

There are a limited number of binary LV2 bundles available for Windows n macOS. That number has been slowly increasing, though just a fraction of LV2 use CV ports so far, but it is early days yet.

https://kx.studio/Applications:Carla https://github.com/falkTX/Carla

https://github.com/lv2/lv2/wiki

⬐ msds

Yeah, some degree of isolation is probably a good idea. The issue wouldn't be an accident due to plugging into an output, but issues with bad grounding or a short to mains. It's unlikely, as it would be a failure condition for the modular synth too, but better safe than sorry. I'd personally like 2-5k kV of galvanic isolation between me (and my $$$ prosthesis!) and the wall.

Plugging into an output would merely make some op-amps unhappy. Maybe a little magic smoke, but nothing lethal...

(Not a real EE and I'm definitely not confident enough to design interfaces between fleshy bits and mains-powered hardware.)

⬐ Ididntdothis

I think there is a big advantage in having some engineering/hacking minded people being the users of medical devices like this arm prosthesis. I work in medical devices and I always think how good it would be to have test patients who have a deep understanding of our devices. I am sure progress could be accelerated a lot if you had patients who can give better feedback. I know it’s not FDA approved but if I had device myself I certainly would play with the programming. Obviously the level of experimentation depends on the device. If a malfunction can do serious harm you probably don’t want to experiment much. But there are a lot of devices that won’t kill you if something goes wrong.

⬐ liveoneggs

real user feedback? do you live in la-la land or something? We're going to need an outside agency, a giant game of telephone, and a bunch of contrived questionnaires to spend that kind of money.

⬐ cronix

This is really fascinating to me. I've often wondered what exactly makes an artist/musician. Is it the person who can create it in their head, or the person who can actually write it down and/or actually play it on an instrument? Up until now, it's always been the latter. Not everybody can read/write music or play an actual instrument, but most can create it in their head. Like, I'm sure most of you can create a simple tune and hum it, and don't know how to read/write music or play an instrument. This kind of thing would allow you to. I wonder how many hidden artists throughout time have created brilliant masterpieces, but just could never express it.

⬐ pmoriarty

I'd encourage you to read Brian Eno's "Composers as Gardeners"[1], which talks about a different way of looking at composition, which is pretty different from the usual way people think about it.

Many non-musicians are under the impression that composers think up music in their head and then write it down, and while some do work that way to some extent (with they myth of Mozart working like that in the movie Amadeus being probably the most well known and most extreme case of composition being completely like that), for most composers (including Mozart) this is much more of an iterative, experimental process.

In recent decades, and especially with the advent of electronic music and computers, a lot more randomness and elements not fully or directly under the composer's control have entered in (though, again, this goes back at least back to the age of Mozart, with his dice game[2]).

There's a whole field of algorithmic composition[3] where music-generation becomes either semi-automated or even completely automated, and the composer's role is one of writing or choosing the algorithms, and of choosing various other aspects of the music (such as which instruments or sounds get used) but without pre-determining the final work in totality or in advance.

Brian Eno talks about this much more eloquently and deeply than I can, so I again encourage you to read his talk.

[1] - https://www.edge.org/conversation/composers-as-gardeners

[2] - https://en.wikipedia.org/wiki/Musikalisches_W%C3%BCrfelspiel

[3] - https://en.wikipedia.org/wiki/Algorithmic_composition

⬐ econcon

I've always wondered why we've learn a specific music instrument and I say this after learning flute (simple bamboo one I bought from India)

Even without music theory or knowledge of specific instrument we are able to hum or vocally replicate a piece of music - why is it not possible to directly transform it into some instrument?

Is it not possible to hum > grab the pattern > generate instrument music with that?

I mean i can articulate smth best with my voice which is with me since I've learned to generate voice on my own so why we don't exploit it with help of some software?

⬐ wheels

You can to some extent. There are a number of tools that attempt to convert from audio to note information (MIDI).

As someone who's done some experiments with those, there are some major caveats:

- Most people can't sing a melody as well as they think they can. Trying to figure out what they were trying to do is where most of the effort goes. Even I'm mediocre at such, and I'm at a professional level at one instrument, and competent at a few others.

- You have to deal with overtones. Almost all naturally produced notes are a stack of waveforms, and figuring out which one caries the intended signal isn't always trivial (and can be even more complicated in polyphonic harmony).

⬐ esistgut

I had a similar thought until I actually started learning an instrument and my SO, flutist graduated from an Italian conservatory, made me start singing with the right intonation (singing the same notes you are playing on the instrument) and rhythm (division and subdivision alongside a metronome). Long story short: singing is hard and requires real effort. There is a good chance that what you think you can articulate is probably far away from minimal quality on both pitch and time.

⬐ ddingus

While that's true enough for many people, adding a feedback step and some practice would improve matters quickly.

There is also a good chance someone can learn to encode quite a bit into vocalizations.

⬐ holri

"some practice would improve matters quickly"

Anyone who seriously learned an instrument or to sing, knows that it takes very hard practice for years to improve.

⬐ ddingus

I have done that myself. Both vocal and instrument.

Yes, but vocalizations may well improve more quickly.

Frankly, I have toyed with this kind of thing. Software could do a lot, in particular, allow the user to adapt their expressions in various ways.

⬐ enisberk

It is really cool, indeed. Bravo for the creativity and the effort. Facebook recently acquired a company called ctrl-labs that develops a neural interface. Their vision is making computer interface more natural. Bertolt, in the video says “The thing is for me, that is such a natural thing to do, I do not really have to think about it. I just do it, it is zero effort.”(6.23) I love that, I hope we can have such computer interfaces in near future.

⬐ 4NDR10D

Great concept and interesting demo, however the title is pretty misleading. This isn't what I would call "thought controlled" more than the old Thalmic Myo or any EMG [0] is thought controlled.

This is reading muscle signals with electrodes and translating those muscle activations into a variable voltage signal. One could argue that because you're not reading mechanical action it could be considered "thought control" but I'll leave that up to the individual to gauge for themselves.

[0] https://techcrunch.com/2018/12/10/ctrl-labs-first-dev-kit-is...

⬐ goldenkey

Most cheaper EEG headbands are misadvertised and actually measure skull muscle activation instead of actual brain signals, so there's that..

⬐ yayitswei

If anyone owns a Neurosky (brain wave reader that fits on your head) and wants to try out something similar, you can try a simple demo that I wrote: https://github.com/yayitswei/hairtime

⬐ rntksi

Did this remind anybody of Exapunks?

It's a game that has a defined assembly-like language, you hack into things to do other things. The catch is some of those things you hack into are actually your nerves, so that you can still function properly.

Pretty cool game by Zachtronics

⬐ berryg

Indeed, very cool. I kept thinking that with an Arduino he probably could have build a first prototype with first building a custom circuit. Just nitpicking.

⬐ kristianov

Initially I was wondering why his prosthesis has ARM processor...

⬐ ggggtez

Very cool, skip to the end to hear it in action.

⬐ caymanjim

Don't skip to the end. It's not about the final product.

⬐ allovernow

That's awesome. I figure its only a matter of time before I'll be hard pressed not to make the difficult decision of giving up my arms for the eventual advantage of a prosthetic, and the seamless way he is able to control the synth brings that horizon much closer.

Combine this with a little ML and you could probably have a general purpose interface trainable in a few seconds/minutes to interact with any suitable output based on your personal signals! Imagine piloting a vehicle without moving a muscle, or playing a video game, or interacting with someone remotely...

Hell, now that I think of it, there's no reason you have to cut off your arms to do so. Is anyone aware of any open source prosthetic tech/code that one could conceivably hook up to functioning appendages?

⬐ daenz

brb cutting off my arm

⬐ sachdevap

I know this is meant to be humor, but realistically, as I understand it, you can just tap into the muscular signals with electrodes - that does not need any such dramatic step.

⬐ aryik

Honestly one of the coolest things I’ve seen on HN. It’s a great display of a ton of things that have just recently (10-20 years) become accessible to the masses. 3D printing for the enclosure, the internet for the schematics, and advanced prosthetics to translate the muscle signal into voltage combined with tried and true technologies like relatively simple circuit design and modular synths. Thanks for sharing!

⬐ holri

"advanced prosthetics to translate the muscle signal into voltage". This is not new. Myoelectric controlled arm protheses were developed and in production already in the 1960ies. https://eduard.horvath-vienna.at/ot2003.140-141.pdf

⬐ mhh__

The thing that's really struck me (which is potentially saying a lot - or nothing - because I'm still young) is how cheap PCB Fabrication (and Assembly!) at a hobbyist level has gotten recently e.g. An 8-layer ENIG, double silkscreen, a few gold fingers for $40 to $50 shipped is pretty damn cheap

Think of all the projects to never get round to making!

⬐ takeda

I don't think making PCB was ever expensive, perhaps making a professionally looking one.

In old times you had a piece of plastic with a copper layer. You put wax on it, then you could use a needle to remove wax in right places (you basically print the inverse of the circuit)

Place that PCB in acid, and after the copper gets dissolved you clean it up, and have printed PCB.

There are many techniques people have, and modern ones involves printers, but you kind of always could do one at low cost. It might just not look that nice.

⬐ kube-system

Doing a complex 8 layer board would have been traditionally expensive... or nonexistent, depending on how long ago we’re talking. DIY accessible stuff is usually single layer or dual layer through-hole stuff.

⬐ takeda

my bad, now after reading your comment I noticed the 8 layer. Yeah I meant only simple PCB.

⬐ Zenst

Now this is true hacking, great lateral thinking and brilliant adaptation as potentially now have better control than you could do with your hand as literally tapping into the muscle/brain signal.

⬐ DyslexicAtheist

him saying "I'm not an expert" , "I don't know anything about electronics", "I'm actually a psychologist", gave me Impostor Syndrome.

⬐ DyslexicAtheist

> Together with Chrisi from KOMA Elektronik and my husband Daniel, I am in the process of building a device (the "SynLimb") that attaches to my arm prosthesis instead of the prosthetic hand. The SynLimb converts the electrode signals that my prosthesis picks up from my residual limb into control voltages (CV) for controlling my modular synthesizer. The SynLimb thus allows me to plug my prosthesis directly into my snythesizer so that I can control its parameters with the signals from my body that normally control the hand. For me, this feels like controlling the synth with my thoughts. I show the prototype(s), explain how we put it together and how it works, and do a little demo.