HN Theater @HNTheaterMonth

Loosely related, MIDI standard can be used to generate all kinds of sounds, even human speech on a real life piano. Here's a demonstration from 2009[1] and a recent article with more context [2].

[1]: https://youtu.be/muCPjK4nGY4?t=8

[2]: https://ledgernote.com/blog/interesting/human-speech-piano/

⬐ lioeters

How wonderful, that gave me the feelies. From the artist's website:

> The piano imitates the human voice and at the same time operates as an alienated recording and reproducing device. It has thus been replaced as traditional musical instrument: no artist operates it in order to play music. It becomes an oversized phonograph which is not used for the production of previously composed music but for the reproduction of the human voice. The sudden comprehensibility of single words, whenever the piano becomes the faithful representation of language, equally has the effect of a phantom’s abrupt appearance: the close up reality of the voice is a ghostly apparition – as though the “forbidden” border between dream (music) and reality (language) had been crossed. The “talking” piano represents a mimetic machine which is capable of producing the mimesis of a mimesis: it absorbs, it imitates what has already previously been imitated, namely the recording of sound.

> From: Chico Mello “Mimesis und musikalische Konstruktion”, Shaker Verlag, Aachen 2010

https://ablinger.mur.at/docu11.html#qu3

I love this demo. It’s a good example of how much our perception is affected by our priors.

This is a other great example, where speech is synthesized with an acoustic piano (though the keys are controlled digitally, the sound is just from the hammers and strings). Without the subtitles you probably couldn’t understand, but it sounds pretty clear when you know what it’s saying already.

https://youtu.be/muCPjK4nGY4

⬐ hprotagonist

you hit it on the nose here. This is a stellar example of priming.

Have you also done your fair share of psychophysics? :-D

⬐ jarmitage

See also this version of "All I want for Christmas is you" converted to MIDI.

It's a really great illusion I think.

https://www.youtube.com/watch?v=psIdan9GSRM

⬐ adrianN

It's like with those "Play X backwards to hear the Devil" type videos. You understand it perfectly with subtitles, but close your eyes and it's gibberish.

⬐ scrollaway

This article talks about that effect (with a similar example):

http://www.theatlantic.com/technology/archive/2014/06/sounds...

⬐ th-ai

great link thanks! it's like the McGurk Effect https://youtu.be/G-lN8vWm3m0, and why I time Vocal Text to appear just before you hear it https://youtu.be/b-jrV7LyN7E

⬐ Frenchgeek

( Strangely enough, that's also how I 'hear' most politicians.... )

⬐ userbinator

I remember when this first came out there was some discussion (possibly here on HN?) that even the sound in the video was post-processed by overlaying some of the original voice. Although I can't seem to find it now, there's a recording of the piano without this editing, and it's far less comprehensible. In other words, they're cheating a little.

⬐ afandian

I always though this was amazing, and a great disappointment that there weren't more videos and documentation about the project. It clearly took a lot of effort, and it's a shame that there's nothing more than a news report to show for it.

⬐ bencollier49

The possibilities are intriguing!

This reminded me of the speech synthesis using pipes which William Gibson described in Neuromancer, I think.

I'd like to see this tried with a church organ. Or a set of flutes.

⬐ codeshaman

Very interesting !

I guess it's possible to use a MIDI synthesizer instead of the physical piano to achieve the same result ?

All that's needed is the conversion from the voice spectrum to notes/chords...

The added advantage is that it would be possible to use the pitch wheel to achieve intermediate notes for higher fidelity..

The output can then be processed again and it's pitch changed and so we can create music from synthesized speech. Does this make any sense :) ?

⬐ None

None

⬐ goldenkey

It's just quantization at that point.. Aka pixie sound filter (also known as bit breaker.)

⬐ TheOtherHobbes

https://www.youtube.com/watch?v=ecm5PKTiVhw

⬐ afandian

Thank you for bringing Paul Lansky to my attention! A radically new soundscape (for me at least). Any more recommendations?

⬐ JoeDaDude

I don't have a specific recommendation, but Lansky sounds a lot like the Tape Musician's of the 50's. Said musicians would slice and glue together countless snippets of sound recordings of real objects or spoken voice to create their compositions. Today, the same can be accomplished digitally. Here is a recent example (though a little pop-music like): https://www.youtube.com/watch?v=pAwR6w2TgxY

⬐ ohitsdom

I'm really curious about the build of the bot. Are the "fingers" that press the keys pneumatic? I'm about to start an unrelated project but it does need fast responding "pressers", which might be classified as actuators. What are the benefits of using air powered vs electric? Does a bot like this have a huge can of compressed air or need to be hooked up to a pressurized line?

⬐ jackhack

Seeing those banks of enormous capacitors, I'd say simple solenoids.

Pneumatic is one approach. Simple and inexpensive but with a lack of control over actuation speed.

It would be expensive, but linear actuators would give incredible speed and precise control over position and velocity.

⬐ ohitsdom

Thanks for the info, much appreciated for a hardware noob like me.

⬐ ogig

Consider the low resolution of this piano speech. Pay attention at how many notes are used to create that low resolution speech, then imagine how many notes/frequencies are really used in natural speech.

As an amateur singer it's amazing and frightening how big is the range to sing out of tune.

⬐ callinyouin

Assuming it's not all homegrown, can anyone determine what software he was using besides MPlayer?

⬐ Tistel

great. now my nightmares have a voice. :)

I do love the fact that someone had a non-trivial odd-ball idea and followed through.

⬐ po

Not mechanical, so much lower resolution... but you might have recently seen the History of Japan video being linked around... the guy who did that, Bill Wurtz also did this:

https://www.youtube.com/watch?v=YBI-nZTUgf8

There's an entire subreddit of this stuff as I guess it's really good practice for playing by ear:

https://www.reddit.com/r/zappafied

It includes this awesome Palin one:

https://www.youtube.com/watch?v=YBI-nZTUgf8

P.S. In case you missed it this is the History of Japan one again: https://www.youtube.com/watch?v=Mh5LY4Mz15o If you watch it again, you can hear him using that technique of harmonizing with his own spoken voice in the background music of that video.

⬐ ardiem

This technique is similar to one Steve Reich's incorporated in a lot of his music, most notably Different Trains (https://www.youtube.com/watch?v=LsRpfbTI1EA)

⬐ cousin_it

I'm 5 minutes into /r/zappafied and it's already my new favorite thing. Thank you!

⬐ agumonkey

Ha... there's a guy named Chassol who does the same live. Lovely to find more.

⬐ ino

That reminds me of the metal band Spastic Ink playing voices, sounds and music from Bambi.

original: https://youtu.be/X5SWc_k_Hmk?t=16m59s

with Bambi: https://www.youtube.com/watch?v=UhP4lg28fs0

Something awesome happened to me. I've known the album well but I didn't know the track was from Bambi. It was a weird track. Some time later I decided to watch Bambi because I was into animation classics and it was awesome all the time thinking "where have I heard this before?" Thank you for making me remember this awesome experience I've had.

I guess the "speaking piano" could also classify as black MIDI:

https://www.youtube.com/watch?v=muCPjK4nGY4

⬐ 0xdeadbeefbabe

Seems like black MIDI could also speak.

Even more notes:

https://www.youtube.com/watch?v=muCPjK4nGY4

Here's a similar method using a piano: https://www.youtube.com/watch?v=muCPjK4nGY4

A loudspeaker is nothing but a coil, a magnet and a piece of cardboard and it too is capable of incredible tonal quality and a huge range of polyphonics.

This one really got me:

https://www.youtube.com/watch?v=muCPjK4nGY4

⬐ userbinator

Apparently he somewhat cheated in that video by mixing in a bit of the original audio.

Here's what it sounds like on its own: https://vimeo.com/1483630

⬐ mg1982

Thanks for that. I am literally more disappointed than when I found out there was no Santa Claus.

Now I'm too sad to work, instead of just being too lazy.

⬐ nealabq

It'd be interesting to hear this done with other (midi) voices: drums, violins, whistles, birds, dolphins, a full orchestra. You should be able to take an arbitrary sound-track, and recreate it by mixing a bunch of samples of ... just about anything.

⬐ hmsimha

Perhaps it's possible, but it would be harder to chain an actual sound with a part spectral print of another sound. I think here they are cropping out the part of the vocal recording that falls outside of the range played by a piano, and then pressing the appropriate piano keys to emulate that recording. A piano key when pressed doesn't even really create a simple tone, but a sound wave with its own structure that sounds like an exact note, even though it contains a combination of other frequencies that mature and decay with their own pattern over time. My guess is, in order to determine which keys to press and when, they assumed the keys were exact notes. It would be impossible to do this with anything producing true white noise (many percussions), and hard to do so with most natural sounds.

This is pretty interesting. Reminds me of the "speaking piano" a little bit: http://www.youtube.com/watch?v=muCPjK4nGY4

We do know that more interesting things are certainly possible with many many notes.

https://www.youtube.com/watch?v=muCPjK4nGY4

⬐ pavel_lishin

I read a fair bit of scifi, and wonder what an alien life form would sound like, trying to pronounce English. This is pretty damn close.

hey this reminds me of the talking piano (http://www.youtube.com/watch?v=muCPjK4nGY4)

MIDIs can do that too right?

And then there's Peter Ablinger: http://www.youtube.com/watch?v=muCPjK4nGY4

⬐ lutusp

It's an ingenious application of the Fourier transform: make a recording of someone reciting text, convert the time-domain recording into a frequency-domain spectrum, then apply the resulting spectral lines to individual piano strings by way of mechanical actuators.

This would be a great movie special effect ... a femme fatale is tiptoeing through a dark room, when a nearby piano suddenly speaks up, saying "You aren't safe here", in a spooky voice that is very clearly a normal piano being forced to utter speech sounds.

talk about complex to play:

http://www.youtube.com/watch?v=muCPjK4nGY4

eerie!

You're correct about the 'reproducing pianos', I didn't know the correct English term.