HN Theater @HNTheaterMonth

Please check this video presentation for a simple overview of Laplace Transform [1]. Basically Fourier Transform (FT) is a special case (subset) of Laplace Transform (LT) where the signal waveform revolves around a unit circle (real power of exponents) . Similarly, LT is basically a generalization (superset) of FT that the signal waveforms revolve off the unit circle (complex power of exponents).

The discrete FT or DFT, however, as the name clearly implied, is the discrete version of FT and similarly discrete Laplace Transform (DLT) is the discrete version of LT. The main difference is that DFT covers finite sum but DLT covers infinite sum.

The faster version of DFT (without compromising the resolution accuracy) is called FFT and it is probably the most useful and important algorithm in the 21st century! The inverse FFT is called IFFT and it was discovered around the same time of FFT. The faster version of DLT is interestingly called Chirp-Z Transform (CZT) and somehow its inverse (ICZT) discovery is at a much later date as has been reported recently [2] and also featured in HN [3]. This much later date of discovery is mainly due to the complexity of complex power exponents (pardon the pun but cannot resist).

Fun fact, CT was discovered by Lawrence Rebinar who was working at AT&T's speech processing lab (SPL) [4]. The lab is so well funded that Kernighan and Ritchie who were belong to the other lab has to scrap by the older computer of the SDL (the infamous PDP-7) where Unix was originally developed on when Multics project got canceled.

[1]https://youtu.be/n2y7n6jw5d0

[2]https://www.electronicsweekly.com/news/research-news/dsp-inv...

[3]https://news.ycombinator.com/item?id=21230757

[4]https://ieeexplore.ieee.org/document/1276120

If you are familiar with Fourier transform, then

Fourier transform: sinusoidals

Laplace transform: sinusoidals + exponentials

Here is nice video explaining it: https://www.youtube.com/watch?v=n2y7n6jw5d0

⬐ londons_explore

I actually used this math at home! - one of my LED lamps was flickering in cold weather, and upon opening it up I saw a primary side controlled flyback transformer to act as a constant current power supply (for ~20 LED's in series).

The flicker was due to oscillation, due to a pole too close to the right half plane. I was about to stick a bigger capacitor in the feedback compensation circuit, but a bit of maths told me that that would make the problem worse (and probably blowing up my LED's) - so instead I used a smaller resistor value, and yay - perfect flicker free light!

⬐ peter_d_sherman

A bizarre effect that I've noticed on LED flashlights that I own is as follows: The flashlights contain multiple LED elements, and as the battery depletes, some of the LEDs will flicker, but not all of the LEDs will flicker at the same rate! That is, LED A might flicker at half/twice/? the rate of LED B...

This is probably not related to Laplace Transforms, but I thought it was an interesting phenomena...

⬐ londons_explore

It probably is related to laplace transforms... Pretty much anything oscillating can be analyzed with a laplace transform.

I'd guess in your case it's because each LED has it's own constant current power supply, and when the battery voltage gets low, the power supply becomes unstable, but the exact flicker rate depends on small variations in the components it's made from.

⬐ mkl

What was the link to cold weather?

⬐ londons_explore

When I did the stability analysis, it was marginally stable (ie. poles on the axis). I would guess whoever designed this lamp fitting used the wrong formula or didn't understand stability analysis.

By chance, it was mostly stable when warm (presumably because some resistance or capacitance slightly changed to push the pole to the left half plane), but when cold it flashed on and off at about 2Hz.

⬐ klodolph

Aluminum electrolytic capacitors have higher ESR at colder temperatures, and lower ESR at higher temperatures. That is my guess at what went wrong. They generally perform better at higher temperatures (within reason), although a particular circuit may require the ESR to be within a certain range (don’t go randomly swapping out your capacitors for low-ESR capacitors, it may make the circuit unstable or destroy something).

⬐ londons_explore

Yes - the circuit had electrolytics on both the input and output side, and for my analysis I assumed they were both perfect, zero ESR capacitors, which is clearly an over-simplification...

⬐ drhodes

I noticed outdoor LED lamps flickering too, but only at dusk, and for what I thought was a feedback issue. These lamps are solar powered and turn on at night.

The cool part is that at twilight, when the light turned on, the little bit of reflected light bounced back to the photovoltaics, adding just enough light to turn the light off, which brought the system full circle. Amazingly, this behavior only occured for about 1 minute each day.

⬐ dreamcompiler

Thermostats would "flicker" this too if they didn't incorporate hysteresis. Hysteresis means the threshold at which the system transitions from OFF to ON is different from the threshold from ON to OFF.

⬐ londons_explore

The normal solution to this is hysteresis. In most cases, it doesn't cost any extra to add hysteresis.

⬐ Enginerrrd

It does if you pay for a competent engineer in the first place who would know to do things like debounce switches and add hysteresis.

⬐ m23khan

I really appreciate people (amateurs and professionals) who take time to make explanation of mathematics concepts simpler and easier to understand -- even for layman just familiar with basic primary school maths.

Because of such people efforts, I personally consider this to be a golden age for mathematics and an untold opportunity for younger folks (i.e. students) to finally understand the maths, get involved in it and perhaps use it in ways previously only math wizz would have used.

I wish these were available back when I was taking countless Calculus and Algebra courses at University -- all we had were Professors who couldn't explain in a simple way and books which nobody had time to read during course of semester. The end result was simply treating mathematical phenomenons as black-boxes and/or perform rote memorization to clear the course.

⬐ targonca

I prefer this one:

Where the Laplace Transform comes from (Arthur Mattuck, MIT) https://www.youtube.com/watch?v=hqOboV2jgVo

⬐ gtycomb

A piece of jewel. Reminds me of chalk board talks of Gilbert Strang and Robert Gallager (also on youtube, mit video lectures) that celebrates this gift for teaching us something.

⬐ rusk

This is tangential, and veering very much into the off-topic .. but the Laplace transform has a special place in my heart, because after banging my head against the wall of advanced calculus and other engineering mathematics topics for 4 years, with varying degrees of success, when presented with laplace I could "just do it".

From what I recall, the transformations were so well defined and the solution space so self contained that I could solve the assignments with little difficulty.

I remember in one of the other modules using laplace to solve a problem and the lecturer remarking that it was "an interesting approach". I had no idea what it meant, (though had some slight intuition about how it worked, based on previous exposure to Fourier transforms), but to this day I can't understand how I could take to it so fluently, while floundering at everything else.

Could just have been that in 4th year I gave up part time work and drinking to focus on my finals, and this improved my abilities.

⬐ ta0xdeadbeef

The whole point of Laplace is to make problems easier to solve by mapping linear differential equations to algebraic equations. To quote my dynamical systems professor, "you learned how to solve this in high school."

There is a bit of hand holding in engineering courses with it however. A lot of looking at transform tables, and maybe a few problems where you need to exploit the properties of the transform or remember its definition to solve a problem. That can trip people up, and it happens in the real world more often than you'd like.

⬐ lonelappde

It's the multivariate calculus version of drawing a graph at logarithmic scale?

⬐ dreamcompiler

This was my experience too. I learned LTs after my differential equations course, and I was a little mad at my DE instructor for forcing us to solve the things the hard way. In retrospect that was probably the right approach. Students should learn manual transmission before automatic.

⬐ zozbot234

The LaPlace transform is also isomorphic to the net present value problem in finance. The Wiki article on net present value even links to a paper pointing this out from a finance POV. (The problem of how the net present value of a flow varies with changes in the discount rate is also of some theoretical interest in finance, since it clarifies the conditions under which the so-called 'roundaboutness' of e.g. an investment is well defined.)

⬐ pgt

Can you expound on this? I recently applied some classic signal processing techniques to help reconcile my personal tax accounting by tracking coherence. This sorta works if you consider double-entry accounting as reverb in a linear time-invariant chamber (or echoes with unknown delay). I would love hear about any non-traditional applications of DSP.

⬐ londons_explore

I learned all this at university, over many hours of classes, exams and practice questions.

I'd be interested if people with a science/tech/math background, but no specific training on laplace transforms, managed to understand this video. If they did, it might be a good time to replace all those university classes with this video!

⬐ fapjacks

I am an autodidact with programming and math with no specific training on Laplace Transforms and as usual with videos of this nature, I do a lot of pausing and rewinding, but I did come out the other side understanding the concepts explained in the video. I have no idea if I could satisfactorily answer test questions about Laplace Transforms, but I feel like I understand the intuitions he was trying to convey.

⬐ yummypaint

I'm a practicing physicist and I though this video was perfect. Undergrad classes were roughly a decade ago for me and I never took complex analysis. If I did this in grad school classes I don't remember and it was maybe 1 or 2 homework problems total.

I have a feeling I'll retain the ideas better than I would have with a traditional lecture. There is something about how the visuals are presented and transition into each other that really helps make things intuitive. I don't expect to use this directly anytime soon, but it's nice to keep the concepts in the back pocket.

⬐ eyeundersand

Will test and report back! I have an academic background but the only thing I remember about Laplace right now is his correction for the speed of sound, way back in high school.

⬐ aliceryhl

I'm studying pure maths at my second year and while we've seen Fourier series, we haven't seen Fourier or Laplace transforms. I found the video rather easy to follow.

⬐ lordnacho

Well I also did Fourier and Laplace at uni, but I'm certain it would have been easier to learn if the current explosion of videos on tech topics had happened before I'd gotten there. That 3D of the sums makes a lot of sense, because you need to motivate taking that integral somehow. In my mind I did that, but it's nice to see the video.

Quite a lot of these topics are the kind of thing where you needed to find the explanation that made sense _for you_, and despite universities having a lot of books chances were there were only a handful on any topic such as this.

⬐ peter_d_sherman

Apparently (and I am no expert), The Laplacian Transform is a superset of the Fourier Transform, and the Fourier Transform is a subset of the Laplacian.

Also, apparently the Laplacian Transform can be used to convert Calculus derivative equations into algebraic ones...

Again, I am no expert; what I've noted is based solely on the claims in this video...

But, some fascinating stuff...

⬐ TeMPOraL

> convert Calculus derivative equations into algebraic ones...

This trick is - AFAIR from my classes - the basis of control theory math. You convert a control system to algebra, do your work there, and in the end, you convert back to differential equations.

⬐ ta0xdeadbeef

You can also draw a parallel to category theory/abstract algebra, where Laplace is a functor between algebras.

⬐ mikorym

It should say Laplacian in OP's post, not Laplacian Transform (if I am not mistaken). The Laplacian is a matrix of (partial) derivitives and is used for the equational conversion.

⬐ Jugurtha

> It should say Laplacian in OP's post, not Laplacian Transform (if I am not mistaken).

Neither. It's "Laplace Transform".

> The Laplacian is a matrix of (partial) derivitives and is used for the equational conversion.

Bar the name, this is called Operational Calculus[0].

[0]: https://en.wikipedia.org/wiki/Operational_calculus

⬐ mkl

It should be "Laplace transform". This video has nothing about Laplacians at all.

Are you thinking of the Jacobian matrix? I'm not sure what you mean by equational conversion. The Laplacian matrix is from graph theory and doesn't involve derivatives. The Laplacian operator involves differentiation, but is not a matrix.

⬐ mikorym

One more point to sort this out. Is Laplacian always that modified adjacency matrix in graph theory, or does it mean something else as well?

⬐ mkl

That's the "Laplacian matrix". "Laplacian" as a noun usually refers to the differential operator, and "Laplacian" as an adjective is attached to quite a few things as well as the natrix (mostly developed or worked on by Laplace, or based on such).

⬐ mikorym

Sorry, sorry. You are right, I am thinking of the Jacobian.

What does the comment mean then?

> "Also, apparently the Laplacian Transform can be used to convert Calculus derivative equations into algebraic ones..."

⬐ mkl

That's the primary reason to use the Laplace transform, as seen in the video. A derivative x'(t) gets transformed into a product (and an initial condition), s X(s) - x(0), and similar for higher derivatives, so a differential equation transforms into an algebraic equation, which can be solved by rearranging. This video assumed the initial conditions like x(0) = 0, and its notation was quite sloppy/confusing in places, as it didn't clearly distinguish the names of the two functions, x(t) and X(s).

⬐ b3orn

You usually don't convert back to differential equations, you convert back to algebraic equations.

⬐ JimiofEden

I remember taking Control theory classes and thinking 'Why the hell did it take me 4-5 years to get to this level of mathematical mastery.'

It was amazing to be able to just do a little bit of odd math here or there, and suddenly have the solution on how to fix something completely unrelated, like how to tilt an airplane safely or make a robot balance a pen upright or something, by modeling it. Granted, we did very simple models, but the sheer scalability and power of these techniques is what made me feel on top of the world.

⬐ leni536

In physics literature it is not uncommon to see Fourier transform with complex omega while Laplace transform is more common in engineering. It's the same thing at that point.

⬐ nicwilson

The Laplacian and the Laplace Transform are two completely different things.

The laplacian is the dot product of the gradient operator with itself, the Laplace transform is an integral transform with the kernel e^(-s*t).

The relation of the Fourier transform to the Laplacian is that FT diagonalises the Laplacian.

⬐ timonoko

Transform of nightmares. Computers were not yet invented in 1974 and we were trained to design analog electronics with massive laplace matrixes and sliderules and tables. Now I not remember anything about matrix calculations and know exactly nilch about any bloody transforms.

⬐ raxxorrax

If you know how to turn a dial on a pid-controller, you are probably already on par with 95% of the engineering work force.

I had a really good teacher on this topic, but without a formulary for time domain -> spectral domain I wouldn't come very far. And they are all pretty hard to memorize.

⬐ dreamcompiler

I'd love to see a follow-on that described the z-transform. The z-transform is the digital or discrete version of the Laplace transform. It's very useful for designing digital filters.

⬐ bainsfather

Previous video from the same author is on the z-transform.

⬐ dreamcompiler

This is a very good, quick video that describes both of the amazing properties of the Laplace transform (how it's the complex version of the Fourier transform and how it makes linear differential equations almost trivial to solve).

⬐ XnoiVeX

Also interesting to note that stock market returns map better to a Laplacian distribution. https://sixfigureinvesting.com/2016/03/modeling-stock-market...

⬐ pgt

CONTROL THEORY NERD ALERT! This is hands-down the best video on control theory I've ever seen, clearly depicting the relationship of the Laplace Transform to the Fourier Transform.

The Fourier Transform decomposes a signal into its sinusoidal frequency components. It is used in a bunch of everyday appliances like your guitar tuner, JPEG images (wavelet compression) and speech recognition on your smartphone or smart speaker.

The Laplace Transform, on the other hand, decomposes a signal into both its exponential factors (decaying or rising) AND its sinusoidal components. So the FT is just one slice of the Laplace Transform where the input signal has no exponential rise or decay.

In the electrical and mechanical domains, spring mass damper systems are super common, even your car's suspension! And to analyse and control them, engineers apply the Laplace Transform.

⬐ danharaj

> The Laplace Transform, on the other hand, decomposes a signal into both its exponential factors (decaying or rising) AND its sinusoidal components.

I want to clarify something a tiny bit misleading about this. In general complex exponentials are not orthogonal w.r.t. the relevant inner product. You can't really think of them as independent components that compose a function.

If you think of a function as the impulse response of a linear time invariant system, then the laplace transform of that function tells you the result of an experiment where you drive the system with an exponentially damped sinusoid. This is why the poles of the transformed impulse response tell you about the stability of the system: those are the inputs that cause the system to explode!

⬐ pgt

Aren't complex exponentials signal rotations? If you multiply a sinusoidal signal by e^(-iwt), you are essentially shifting its phase modulus, no?

⬐ danharaj

Those are purely imaginary exponentials. General complex exponentials also have a real part.

⬐ dreamcompiler

And that's how the FT differs from the LT. LT is complex while FT is purely imaginary, as the video explains.

⬐ raxxorrax

nitpick for people that like pictures: JPEG uses a 2D discrete cosine transformation (DCT) instead of a fourier transformation. Basically the same principle with a different set of base functions, but suitable for compression, because you generally net fewer real coefficients.

I think DCT is a good entry to understand these kind of transformations. The set of base functions for DCTs is displayed in the image on this site:

https://edoras.sdsu.edu/doc/matlab/toolbox/images/transfo6.h...

I think imaging makes understanding the motivation of frequency dissection plausible and you can imagine any image being a sum of these base images with a weight(R) and a phase/offset(IM). From that understanding LaPlace and FT is far easier. You can even kick one dimension in most applications like audio.

⬐ dreamcompiler

Yes he pretty much ignored the phase output of the FT, which is often very important. Like you said, a minor omission in the name of keeping the video short I expect.

⬐ esmi

I never thought of it that way. For me Laplace is all about region of convergence and representing things like the Dirac Delta function in a way that’s easy to manipulate.

Basically the video ignores the idea that the X(w) integral may not exist. It doesn’t for many x(t). I don’t want to type a book, but basically the X(s) integral exists (where it does is called the region of convergence) for many more x(t), so there are classes of functions which can now be analyzed where with just Fourier, they could not be.

The other thing is delta functions. With Fourier they are a pain but with Laplace they are a breeze. This is especially important if you want to mathematically model how the control system will respond to a jolt or impulse. I always figured that is why control engineers use Laplace.

⬐ achillesheels

What are your thoughts on time-frequency duality, in that time as a variable is able to be abstracted away for our analytical purposes (which works!)? And is the frequency domain a more qualified s-plane?

⬐ pgt

Consider a point on the edge of a rotating circle which is rotating counter-clockwise at a constant frequency. If you plot the point over time in three-dimensional space (starting from 0rad), you get a spiral or helix (like one half of a double-helix DNA strand).

If you view the spiral from the side, you see a sine wave.

If you view the spiral from the top, you see a cosine wave.

If you look down the barrel of the signal, you will see an infinitely bright circle with a radius equal to the amplitude of the signal. The brightness of the circle relates to the energy in the signal, since by looking down the barrel, we have collapsed time and relinquished all phase information.

Pure sinusoids are infinite, so real-world signals will be windowed (note: windowing always introduces artifacts).

Now, what happens if the signal is a composition of rotating circles all spinning at different frequencies and directions? Well, you'll see a bit of a blurry distribution of where the wavefront spends its time. If you think about this a bit, you can ask yourself what you have to do to "see" the frequency components of the signal :).

I have to go now, but will try to post more later.

⬐ achillesheels

I appreciate the insight!

>the brightness of the circle relates to the energy in the signal

What does this “energy” mechanically correspond to in the complex unit circle and how do these mechanics appear in the time-domain?

The ideal low pass filter’s mathematical application - of the frequency domain effects on natural signals - astonishes me and I can’t quite fathom the transformation; ie is the frequency space physical or a purely mathematical abstraction?

Ultimately: is the complex exponential function a natural algorithm?

⬐ dreamcompiler

Do EEs still study this stuff? Lots of people use plugin PID loops for control these days but if you know control theory you just derive them from scratch and know how they're going to behave rather than tweaking endlessly.

⬐ esmi

Definitely. The various transforms are useful for many things besides control. Although control is an important one.

⬐ throwlaplace

made an account to ask about this

>The Laplace Transform, on the other hand, decomposes a signal into both its exponential factors

in what sense is this true? yes the s in e^(-st) is complex but that factors the kernel of the transform into e^(-at)e^(-iwt). individually the factors do what you say (decompose over a basis) but as a product they do not do that.

⬐ mickael-kerjean

> The Laplace Transform ... decomposes a signal into both its exponential factors .. AND its sinusoidal components

This is the OMG moment from the video. I wish my teachers would have put it this way a few years back. It now feels like an obvious secret hidden in place sight but Laplace was a very abstract concept before