HN Theater @HNTheaterMonth

Very cool.

Your results look much better than the washed out AI colorization that I've seen in the past.

I think you could charge money for this service.

A suggestion for something fun to do / marketing tool: recolorize this video frame-by-frame https://www.youtube.com/watch?v=hZ1OgQL9_Cw

Feature request: colorize B&W comic books. I really want to create a full color book of Calvin & Hobbes comics. (Not for publication)

⬐ emilwallner

Thanks! yeah, a few people have used the tool to colorize videos, frame by frame. For example Lord of the flies (1963): https://www.dailymotion.com/video/x8eiho4

Although, I'd recommend colorizing a few key frames and then use https://github.com/zhangmozhe/Deep-Exemplar-based-Video-Colo...

Cool, yeah, my next model will be better for comic books. You can also use the 'Surprise Me' button in the editor and you'll get some decent results.

Much better ones:

https://www.youtube.com/watch?v=hZ1OgQL9_Cw

https://www.youtube.com/watch?v=fo_eZuOTBNc

https://www.youtube.com/watch?v=EQs5VxNPhzk

⬐ mlindner

I really hate these types of videos that ruin a perfectly good video and fill it with AI-generated fake frames and colors. Colorization especially is not something that an AI can do well yet. It generally turns it into a rainbowy mess of constantly shifting colors.

Also even the framerate is often wrong as these old movies didn't have consistent framerates. They were hand cranked so the framerate varied quite a bit.

As an African-American it was actually useful to watch some of these colorized videos and get a better since for how African-Americans integrated into the society at that time.

Having it in color with sound made the video seem much more real and relatable.

For me this just ads another facet to develop a better-rounded grasp of the era.

Inserting the extra frames seems to slow the footage down and allow more time for digestion.

I find it often difficult to track what's going on in old videos because of the off-speed of the recordings.

But for the historians who spend a lot of time with these artifacts, I'm sure this feels like a bastardization of history.

But as a lay person, it makes me want to devour more of these videos to get a fascinating and easier-to-digest look at life back in those times.

Btw, this is the video I watched: https://youtu.be/hZ1OgQL9_Cw

⬐ roywiggins

The dance scene from Hellzapoppin' is my favorite use of the colorization algorithms so far- it's so much easier to follow the dancers when the color pops them out of the background. Color contrast helps a lot. In that case the whole setup- the costumes, the lighting, etc, are all artificial in the first place, adding some fake color on top doesn't seem like a historical sin.

https://www.youtube.com/watch?v=qzc7vY9VTnk

Personally, I don't find frame interpolation to be as effective as colorization; the artifacts it causes are often more distracting than just leaving the framerate alone. It can create a sort of artificial floating feeling to the motion- a problem that people often complain about with native 60fps, and is much worse with interpolated 60fps. Algorithms keep improving though, eventually I might find them more satisfying.

⬐ dnautics

I love the hellzapoppin' colorization but it does show that we have a long way to go -- you can see the colorization struggle with keeping the skin tone of the dancers consistent. obviously, this has a lot to do with training data bias.

⬐ webmaven

> I love the hellzapoppin' colorization but it does show that we have a long way to go -- you can see the colorization struggle with keeping the skin tone of the dancers consistent. obviously, this has a lot to do with training data bias.

The colorization is struggling to keep colors consistent on the costumes, walls, curtains... basically everywhere the lighting keeps changing a bit.

Many things in the scene are constantly shifting minutely between pinkish-beige and bluish-grey, often making them look somewhat iridescent.

I've noticed this sort of effect in many colorization attempts. This one does better than most.

Training data bias is obviously part of the problem, but not the whole story. It might not even be the biggest cause. Image segmentation is one way to help the colorization stabilize.

⬐ ijidak

Wow! Just watched it. Agree. It is easier to track the dancers!

I wonder if it has to do with the fact that our vision is color.

Is there color data that the brain is using to process fast moving images that we lose in black and white?

Maybe color moving over a different color is easier for the brain to process than two similar colors moving with relation to each other.

I just tried watching one of the upscaled and interpolated videos, and then comparing it to the source. Please consider doing this before commenting, because in my opinion it really illuminates how silly this all is.

4K: https://youtu.be/hZ1OgQL9_Cw

Original: https://youtu.be/aohXOpKtns0

The colorization, of course, is lies. The upscaling and interpolation, however, is harder to argue about. Sure, if you were to freeze a frame and zoom in, you are looking at data that may not directly correspond to what was recorded. But it didn’t come out of thin air either - its upscaled and interpolated. I have read the arguments and some of the comments on this page and I still cannot figure out how this isn’t just rationalizing elitism.

Of course we need to keep the source material, but nobody is suggesting to not do that.

⬐ aidenn0

The thing that struck me most about the colorized video was how every single person in it was wearing a hat. I did not notice this in the B&W video for some reason.

⬐ underwater

The ironic thing is that your "original" has completely fabricated sound added to it, which involves much more subjectivity than interpolating frames.

⬐ jchw

This is true. I would’ve linked a different source, but that’s of course the source used by the upscaled version. It is certainly worth mentioning.

⬐ paxys

Even the "fake" colors are a lot more representative of reality than black and white.

⬐ the_pwner224

The fake colors are completely made up by the computer. The people could have been wearing drab green or bright red or hot pink suits and it would show up as the same sepia/navy blue that the computer makes all the clothes look like in the colorized video.

⬐ int_19h

I don't know about that particular video, but they're not always made up. There are videos and photos of objects that we know were colored in a certain way (from contemporary descriptions) - flags are very handy in that regard, for example. These can be used to calibrate the algorithm. Of course, a B&W image will still be ambiguous, and the algorithm has to make guesses. But those guesses aren't completely random.

⬐ the_pwner224

It's the computer trying to copy the color from the training image set onto the video. My point about a guy in a bright red or hot pink suit in the B&W original being represented as sepia/navy in the colorization is still the same. It's impossible to turn B&W image into a color image without guessing what the colors are. The average color of a guy's suit in the colorized copy might be right if it's trained on a realistic dataset, but ultimately it's still guessing and making up information for each individual suit.

⬐ mrtksn

Wow, I played those in sync and for me the 4K version is significantly more relatable. Even though the colours are obviously fake, the smoothness of the movements of the people in the video is what makes it great because I can actually relate and feel the emotions through the body language. It also feels more immersive because the nature's physical behaviour looks more accurate.

⬐ shard

After reading your comments, I was curious so watched the two clips as well. Although the 4K version was a bit nicer to watch (which I watched first), I didn't feel a significant difference compared with the original. Perhaps I am biased, as I do enjoy watching old (or new) B&W movies.

Somewhat related, there have been some impressive ML-powered video remasterings recently (colorized and upscaled to 4K at 60 fps):

- Footage Of The First Flying Machines 1890-1910: https://www.youtube.com/watch?v=a_G1YbItY9o

- A Trip Through New York City in 1911: https://www.youtube.com/watch?v=hZ1OgQL9_Cw

- Views of Tokyo, Japan, 1913-1915: https://www.youtube.com/watch?v=MQAmZ_kR8S8

⬐ kubanczyk

The New York 1911 is amazingly well done.

⬐ ThJ

I think all those trains, cars, carriages and people would’ve made a lot more noise than the audio mix suggests. The trains and cars especially should be drowning out everything else as they pass near the camera. This sounds more like the audio track of a feature film than anything captured with a microphone in the field.

⬐ lakkal

At 00:05 there's a credit for 'Ambience sound by guy jones', so I think as someone else said, the sound was added later.

⬐ Piskvorrr

Of course it wasn't captured, it's a modern audio track applied to a colorized+upscaled, originally silent film.

Silent films (which meant that literally no audio track was recorded) did depend on the cinema creating one, either through people literally banging on cans, through an orchestra, to an outright Rube Goldberg contraption: https://psmag.com/environment/sound-effects-in-the-silent-mo...

⬐ russfink

Maybe there were fewer people in general, but did you notice how few private vehicles there were, and many people used transit or simply walked? How many people walk across the Brooklyn bridge today vs. driving in a car?

⬐ Piskvorrr

Yup, this is before the car revolution took off.

⬐ jiggawatts

It's amazing how "properly dressed" everyone is compared to current standards. Almost everyone, including young adults, are dressed in a suit and tie with a hat.

⬐ russfink

I wonder if the hats were necessary fashion, e.g., lots of fires burning for heat, industry, etc, producing soot that got in your hair.