HN Theater @HNTheaterMonth

Rich Hickey's talks "Simple Made Easy" [1] and "Effective Programs" [2] provide a better explanation of these ideas IMO. The specific definition of "simple" is pretty crucial.

[1] https://youtu.be/LKtk3HCgTa8 [2] https://youtu.be/2V1FtfBDsLU

I guess no one uses Clojure/Script on HN anymore. All of your requirements fit the cluster in designspace that the language was designed to occupy.

(cf. 'situated programs' in https://youtu.be/2V1FtfBDsLU)

https://youtu.be/2V1FtfBDsLU?t=2268

Goes until 43:25.

The whole talk is worth watching. It's basically about the higher level problems of business app development, how to address them, and how Clojure does address them.

The creator, Rich Hickey, explains that Clojure has been explicitly designed for 'situated programs' in this talk: https://www.youtube.com/watch?v=2V1FtfBDsLU

Honestly, I think this series of blog posts could have had a more complete title: "Principals of Data Oriented Programming (aka, Idiomatic Clojure)"

All of these ideas (and I think they are good ones) are inspired heavily by Rich Hickey's talks and rational behind developing the Clojure language (the author of the post states as much). And while you can use these techniques in other languages/paradigms/problem domains, they are really intended to work well inside the constructs of Clojure, and when applied to "information-driven situated programs" [0] (read business applications with dynamic requirements).

As for some of the short-comings you mentioned:

"But you still need a mechanism to manage mutating data"

Clojure supports this through the use of locking constructs like atoms. [1]

"I think what it's getting at is that you don't really know the precise type of your data, over time, in a distributed system, so it's good to include the flexibility to handle that. That makes sense to me. but generic data structures aren't necessarily always the right way to handle that."

Clojure attempts to bridge the gap between generic data-structures and strongly-typed constructs using run-time specifications. [2]

I mean, the ideas presented here can be generally useful, but your mileage may vary if the principals take you too far out of the idiomatic for your particular language/paradigm/problem domain. If that's the case, you could find yourself wasting energy swimming up stream.

[0] - https://www.youtube.com/watch?v=2V1FtfBDsLU [1] - https://clojure.org/reference/atoms [2] - https://clojure.org/about/spec

⬐ None

None

That’s precisely what this is, including what he got right &c — “10 Years of Clojure” https://youtu.be/2V1FtfBDsLU

Rich hickey explains at this timestamp in the video,

https://youtu.be/2V1FtfBDsLU?t=2426

"The information programming problem"

Whole video is gold, is my fav programming talk of all time.

> Requirements change. Every software engineering project will face this hard problem at some point.

Rich Hickey calls this a 'situated' program in his talk from Clojure/Conj 2017, where the program has to deal with information and real-world irregularities for extended periods.

While agile disciplines help you from one perspective, your tools (e.g. program language) might also help you solve some of the problems. In this framing, Clojure certainly tries to help where the it matters most.

Highly recommended to read the transcript or watch the video, even if you are not interested in the language itself.

Video: https://www.youtube.com/watch?v=2V1FtfBDsLU Transcript: https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

⬐ tosh

I love the anecdote at https://youtu.be/2V1FtfBDsLU?t=515

when Rich talks about the lightweight languages workshop at MIT

⬐ andy_fingerhut

Transcript for this and many other talks is available in this repository: https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

Clojure is concise, expressive, not noisy, opinionated (with generally appreciated opinions), dynamically typed, a Lisp, functional with some escape hatches (but not confused with being FP and OOP), and built to solve real problems. Scala is either the opposite of these, or at least not as much of these.

IMO, less code is better. Clojure is about as terse as you can get and still be readable (by my opinion). Scala is very visually noisy in comparison. While you can do types with Clojure.spec, you get to choose when and use when useful to you.

Rich Hickey is the best communicator of why Clojure is good. Here's a relevant starting point a video talk he gave on the 10 year anniversary of Clojure: https://youtu.be/2V1FtfBDsLU?t=1636

All this said, Scala has more jobs than Clojure in the same way that Java has more jobs than Scala does. It's more Java-like than Clojure (and not just because it isn't Lisp). (And incidentally, only people who haven't tried at all to use Clojure complain about parens ().)

Sadly, I think Javascript is the long-term winner. Python is also a winner, but it doesn't have the front-end reach that JS has (while JS also has server side). Clojure and Clojurescript can do both, and are an awesome choice if you don't mind working alone and having few job opportunities. However, if you want to focus on one language and be able to do a lot of things, and have lots of job opportunities. Javascript is actually the one I would suggest.

⬐ slowmovintarget

Though Clojure jobs tend to be the highest paying.

https://insights.stackoverflow.com/survey/2019#top-paying-te...

⬐ christophilus

I suspect this is because Clojure developers tend to be more senior.

⬐ hota_mazi

It's pretty common for niche languages. The fewer developers there are, the higher they are likely to be paid.

These jobs are pretty much impossible to get, though.

here's a youtube link you were looking for: https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ ecmascript

That was disappointing, sorry. But that didn't really say anything about why functional programming would be better in the general sense.

IMHO Javascript is "unusable" because you can mutate state (among other things). Rich Hickey has a few other arguments against static typing and OO such as names dominating semantics and "types are an anti-pattern for program maintenance and extensibility because they introduce coupling. They’re also parochial." I think he said that in his talk called "Effective Programs":

https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ McWobbleston

As much as I liked Clojure when I dabbled for a few months it didn't really fix my dislike for dynamic languages, but I imagine core.typed / spec would fix that for me if I gave it another shot

I loathed dynamic typing mostly because you had to jump around code to understand what the shape of anything was, and it distracted me from the problem I'm trying to solve

In ML influenced languages types actively help me plan a solution, and rarely get in my way

Here's a video of the creator of Clojure explaining his preference for dynamic vs static languages, and how he came to the choice of making Clojure dynamic, that may help inform your decision:

https://youtu.be/2V1FtfBDsLU

As mentioned, types are in the standard, and certain implementations provide compile time warnings of type and other issues...[0]

The biggest non-feature advantage (the condition system is a pretty big feature though missing from everything else) is the mindset of writing programs, which other dynamic languages typically don't get right. The idea of "growing" a program, and responding to change, is part of that mindset and once it's internalized a lot of the language decisions make a lot more sense. e.g. "trace", "break", "compile" are all part of the standard, all available at runtime, whereas in other languages you tend to have a separation of the runtime from any "IDE feature" like debugging break points, tracing function calls, and compiling (not "eval"ing) new code. Other languages + IDEs are getting there but there's so much more to go.

In Java land, with the support of things like JRebel, we can make pretty significant modifications to a running program like add/change/remove methods to a class. But what about any existing objects I may have instantiated? Those are left around, referring to the old code. In Lisp, redefining classes works out of the box (no need to wait decades for JRebel to show up), and "update-instance-for-redefined-class" is a standard generic function, which you can write a version of to handle conversion of existing objects. No mysterious old versions of objects running around, you didn't have to bring down the program to restart it, nice. It's a very dynamic and inspectable system, going beyond just the types of things. That to me is still Lisp's advantage as other languages + language tools slowly add more and more of its features that are there already in Lisp independent of extra tools (and Lisp has extra tools too for more things).

I can't resist commenting a bit more on the type situation directly though... The creator of Clojure talks about "situated programs" in https://www.youtube.com/watch?v=2V1FtfBDsLU (he also has 20+ years of professional programming -- I on the other hand am only arguably on year 10 of "paid" exp (I wouldn't call all my gigs professional and not all were full time) but I tend to ignore how much experience someone has when people with equal or greater experience have such divergent views) and along the way I think he makes good points about how type systems get in the way of designing these big long-running systems that need to change as the years go by. Certainly I see this problem in the gigantic Java code base I deal with every day. Tight coupling is the nemesis of modularity, which you need for big systems, I tend to view static types as premature tight coupling. They're not without their benefits (typo protection less so for me but I love the easy speed wins, which you get in Common Lisp too if you bother to declare types; a form of QE via formal type proofs can be useful too), but they're also not without their downsides, and for many types of programs large or small the downsides can dominate.

[0] Examples: https://news.ycombinator.com/item?id=12222404 (told about state never being reached? awesome!) https://news.ycombinator.com/item?id=13389287 (notes about why it couldn't use a fast operator, so you need to fix your types) https://news.ycombinator.com/item?id=14780381 (other nice things like unused var, wrong number of arguments, typoed name...)

⬐ flavio81

>is the mindset of writing programs, which other dynamic languages typically don't get right. The idea of "growing" a program, and responding to change, is part of that mindset and once it's internalized a lot of the language decisions make a lot more sense

Excellent post.

⬐ jungler

There's a page on C2 Wiki that I'm reminded of: "alternating hard and soft layers" as a pattern. And I think this pattern tends to develop throughout large, long-lived systems. Shakeups like transitioning from native apps to the browser, or the replacement of the sysv init scripts with the much larger systemd, make some things "harder" and others "softer".

So you want some dynamism throughout so that a layer change is possible, but you also want some static elements so that they do the existing work reliably. It's a cyclical process, and often terminates abruptly when some other piece of software comes up and replaces the existing one.

I am finding, after spending some years locked into a static-minded viewpoint, a renewed appreciation for having dynamic types available. In the design phase, it assists in creating useful feedback about a system. Coming at it from a point of experience helps in avoiding abuse of reflection, metaprogramming etc. and keeping the basic structure and algorithm design simple and gradually discovering which techniques will move the system forward. What a static approach tends to do is "lock in" everything, like when a child grows up and their skeleton becomes rigid.

If your desire is code with eternal youth, then it would make sense to avoid locking things in! But that might mean denying it some of the wisdom of old age.

Rich Hickey has a lot of excellent stuff to say. Not only are his ideas philosophically sound, but also real, practical and ready to use in the form of Clojure.

This recent review "10 years of Clojure", encapsulates a lot of of the wisdom:

https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ squeed

Agreed! "Hammock driven development" definitely has stuck with me over the years: https://www.youtube.com/watch?v=f84n5oFoZBc

Agree with other ppl here, an unnecessarily violent rant, and the point the author makes is kind of lost in the title "Protobuffers Are Wrong". What they really mean is "I think Protobuffers type system could be better".

I recently saw a talk by Rich Hickey about Effective Programs [1]. The talk explains why Hickey favors dynamic types, with EDN [2] and transit [3] proposed as an alternative to more statically typed data exchange formats like protobufs and less structured ones like Json (the talk explains the reasons, nearing the end I think).

1: https://www.youtube.com/watch?v=2V1FtfBDsLU 2: https://github.com/edn-format/edn 3: https://github.com/cognitect/transit-format

I think Rich Hickey described it pretty well when he said (paraphrasing from [0]) that static languages present the programmer with neat little puzzles to solve that feels like we're writing applications but we're just creating intricate types and abstractions. I think he has a point, but I certainly don't want to give up the benefits of static languages, like being able to catch all of my silly errors.

[0] https://youtu.be/2V1FtfBDsLU?t=39m44s

⬐ codebje

His argument begs the question.

If you pre-suppose that describing types and finding appropriate abstractions aren't "writing applications", that is, they offer no value in the process, then spending time doing them is of course solving neat little puzzles to no benefit.

On the other hand, if you pre-suppose that types and abstractions offer some value to the process of writing an application, then solving those puzzles is adding value.

Calling types and abstractions a "neat little puzzle" is a diminution we could apply to other aspects of writing an application: implementing an algorithm to correctly process some data is a neat little puzzle presented by our test cases.

Rich Hickey would need to demonstrate the lack of value of types and abstractions for producing applications to make a solid argument here, and that I believe is a steep uphill battle, because Clojure would be a pretty terrible language if you took out the ISeq abstraction, and there's an awful lot of effort spent shifting the puzzle of "is this function being used right" from the type system game engine to the test harness game engine.

⬐ snaky

And most often the most beneficial is to spend more time on the problem and its domain, brilliantly illustrated in this comment https://news.ycombinator.com/item?id=17906700

⬐ rictic

Time spent with a type system buys you compiler-checked proofs of some properties. The important question is whether the time is worth the benefit.

One of many things that I love about the TypeScript type system is that it gives so much power to the author. It is the most expressive industrial type system I know of, but it also makes it easy for the programmer to say "I can't prove that this is true, just trust me that it is".

A nice side benefit is that this is also practice, so you can also gain skill. This is how I justify sometimes spending much more time with the type system than it would otherwise be worth when learning a new language or hacking on a personal project.

⬐ lmm

> One of many things that I love about the TypeScript type system is that it gives so much power to the author. It is the most expressive industrial type system I know of, but it also makes it easy for the programmer to say "I can't prove that this is true, just trust me that it is".

I really wanted to like Typescript, but once you're thinking in HKT it's incredibly frustrating to have to manually translate a function into its flattened expansion (just like it's incredibly frustrating to use a type system without generics once you've used one that has generics). Every serious industrial language allows a programmer to say "I can't prove that this is true, just trust me that it is"; I suspect many people who struggle to start out in Haskell would do well to make a little more use of unsafeCoerce and unsafePerformIO (they would no doubt give themselves runtime errors, but sometimes the easiest way to understand why you had a type error is to run the code and see what the values are at runtime).

(I made a small hobby tool with ScalaJS and was amazed how easy it was, so I'll be advocating for that over Typescript).

⬐ api

Go is static but doesn't encourage this. The more experienced I've become the less I find myself doing this in languages like C++ either.

It's seductive until you start getting excited more about higher order reasoning about algorithms and about what programs really accomplish and less about just showing off. A breakthrough program written in some uncool language like VB.NET is more impressive to me than yet another ________ written in Haskell or Rust with perfect abstractions.

⬐ acdha

I think a key part is never losing track of the real goal so you can remind yourself what percentage of your time is going to things which anyone who isn’t a developer on your project would care about.

⬐ tokyodude

I agree but I feel like the fact that it's a puzzle is a sign the language could be improved. Describing generic types shouldn't have to look like stl's or boost's implementation details. They are nightmares of "neat puzzle solving"

⬐ cynicalkane

Former Clojure programmer here. I eventually learned I'd rather write neat little puzzles about static types than solve frustrating little puzzles on the root causes of runtime exceptions.

I don't find that too surprising, Rich tends to let things bake internally (in his head and at Cognitect) for a while before releasing anything publicly for consumption and feedback.

Most of Clojure's current development is around tooling [1] and some changes to move clojure.spec out of alpha. I believe they want to accomplish this for 1.10, but I'm not sure where I remember hearing that.

Alex Miller has mentioned that Cognitect built and continues to work on a tool that tracks all of your functions with spec definitions and generatively tests them only when the function or one of its dependencies changes. This would be extremely helpful since generative testing can be expensive to run, especially if you want to generate huge amounts of test cases. It'd be nice to say "test all of my codebase's functions 10 million times" and then have that information stored locally with the code.

In Rich's "Effective Programs" talk [2] he outlines the "10x problems" that Clojure was targeted at solving. In it there are two 10x problems that weren't solved by Clojure: resource utilization and libraries. There's not much Clojure can do to solve resource utilization as it's dependent on the host (JVM), but the library problem (especially Clojure libraries with spec definitions) is solvable (though there seems to be some disagreement on whether only allowing growth in libraries in a practical).

Pure speculation: All of their tooling projects (spec, codec, tool.deps, and the above mentioned generative test runner/tracker) are going to come together in a tool that attempts solves the library dependency problem. Rich outlines a lot of his thinking about how to accomplish this in his "Spec-ulation" talk [3]. He wants Clojure to "become the first community to make [dependencies] great". I hope that we'll see more tools and guidance from the Clojure team on this soon (and maybe improved out-of-the-box error messages)!

[1] https://github.com/clojure/tools.deps.alpha [2] https://youtu.be/2V1FtfBDsLU?t=29m53s [3] https://www.youtube.com/watch?v=oyLBGkS5ICk

⬐ puredanger

It's a journey. :) Do not expect "one grand tool" but as you've noted, all of these things are pointed in the same direction.

⬐ shaunparker

That's a better way to put it. Thanks Alex. I'm excited to see where we end up.

That's quite a judgement call to make based on the utility of error messages. I can't remember the last time I was blocked based on a Clojure error message anyway. Errors are less frequent and usually pretty obvious.

I think you're wrong to say core devs don't understand modern programming, but you'd probably be right to say that they didn't prioritize making it approachable and stooping to the lowest common denominator. And it's not just Clojure, but datomic, core.async, spec -- all solving real problems in modern large scale programming with elegance.

Watch Rich's 'Effective Programs' talk [1] and lmk if you still feel that he's out of touch with modern large scale programming. I think he's pretty in tune.

[1] https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ noxecanexx

As a clojure programmer myself, it surprises me to see other clojure programmers say this. Error messages are an important part of programming and that you didn't face many errors doesn't mean others won't and doesn't mean they don't know how to program either. Even the article above clearly states that error messages are bad.

⬐ rboyd

Which part do you disagree with? Are you commonly blocked by Clojure errors where you have to stop and spend a material amount of time deciphering them?

Nowhere have I said that if you see an error message you don't know how to program.

I face errors every day, but I don't face errors that are so cryptic that they block my productivity. We agree errors are useful even if we disagree that Clojure's errors are bad.

I did find the errors very confusing when I was getting started, but I think that was because the errors use Clojure terminology that I had not yet learned.

⬐ noxecanexx

> Nowhere have I said that if you see an error message you don't know how to program.

> making it approachable and stooping to the lowest common denominator.

> We agree errors are useful even if we disagree that Clojure's errors are bad.

I would assume that you disagree with the article then, in which case I can't make any other point to emphasize that

⬐ hota_mazi

The simple fact that Clojure is dynamically typed tells me that this language is simply not suited for large scale programming.

The trend in programming languages is pretty clear, we are moving more and more strongly toward languages that are statically typed and with type inference.

And for good reasons, in my opinion.

⬐ gkya

It's not really clear to me what you mean with "scale" here. Are you referring to the scale of a program, i.e. size in LoC and/or complexity of components, or to the scale of a team of developers, i.e. the number of persons working on a given codebase?

⬐ hota_mazi

Both.

Scale in performance, since dynamically typed languages have fundamental hurdles that will always make programs slower than when written in a statically typed language.

And scale in code base and developer strain. The fact that it's mathematically impossible to reliably refactor code automatically if the types are absent encourage spaghetti code bases that developers are afraid to modify because they can never be sure they're not breaking anything.

Such hesitations never happen in statically typed languages.

⬐ didibus

The trend is as you said, a fashion trend.

The truth is, the issues that come out of your current model of programming makes you seek something that trades them out for other issues. Right now, those other issues are unknown to you, and you're not feeling the pain they bring. 10 years from now, they'll be all you complain about, and the trend will go back.

Now, maybe I'm wrong, but its rare that a truly better paradigm shows up. It happened before though, so might happen again. Maybe that'll be inferred static types, but we've had those for so long already that I'm doubtful. For example, Clojure has infered static types, a lot of people tried it and felt it was more trouble then worth. So in the small Clojure world, the trend to infered static types already happened and passed by back to runtime contracts.

⬐ dmitriid

> making it approachable and stooping to the lowest common denominator.

Why do people assume that being dev-friendly is the "lowest common denominator"?

Devs are not stupid.

Beginners are not stupid.

Moreover, the more senior you are, the more you begin to appreciate clear precise error messages that directly point to the problem.

There's only so much brainpower, and I'd rather spend it on solving a problem, and not on figuring out/memorizing yet another cryptic error message.

And yes, no amount of fluffy talks or "large scale apps built with datomic" will convince me that those devs are in touch with modern programming. They are just used to what they work with.

⬐ rboyd

My point was simply that Rich built Clojure to solve his own problems, not to coddle beginners. Beginners are the lowest common denominator not because they are stupid but because they are inexperienced, by definition.

> Moreover, the more senior you are, the more you begin to appreciate clear precise error messages that directly point to the problem.

Hopefully the more senior you are, the less you encounter errors in the first place.

> There's only so much brainpower, and I'd rather spend it on solving a problem, and not on figuring out/memorizing yet another cryptic error message.

And I'm saying that's not the Clojure experience once you're up and rolling. You iterate over program construction in small manageable chunks, with immediate feedback from an editor jacked into a cider repl, and your repl experiments evolve into long-standing tests.

Also, even if the Clojure language doesn't necessarily cater to the beginner, the Clojure community definitely does. There are constant efforts to evolve better docs, extend tooling, and build user groups that reach out to beginners.

⬐ bitwize

> My point was simply that Rich built Clojure to solve his own problems, not to coddle beginners.

And if Rich likes Clojure, he can go nuts with it.

But a significant chunk of Hackernews believes that Clojure is the greatest thing since sliced bread, or even the obvious successor/replacement to all other Lisps. As an experienced programmer myself, if someone told me that Clojure is better suited to my application needs than CL or Scheme, I would tell them no, and fuck you. And this is part of why; there are other deficiencies and quibbles I have with Clojure's design.

⬐ dmitriid

> coddle beginners > stooping to the lowest common denominator

You keep saying that people are stupid. And since you keep iterating it, I believe this is the point you are making, not some speculation about what Rich Hickey does or does not.

> if the Clojure language doesn't necessarily cater to the beginner

A dev-frienndly language does not cater just to beginners. Everyone benefits.

> There are constant efforts to evolve better docs, extend tooling, and build user groups that reach out to beginners.

So, they do stoop to the lowest common denominator and coddle? Why can't Clojure?

⬐ rboyd

> You keep saying that people are stupid

I didn't say that, because I don't feel that way.

I think that you're doing it right as long as you're using a language that helps you be productive and brings you joy. Sounds like you're already good where you're at.

Enjoy the rest of your weekend.

⬐ dmitriid

> I didn't say that, because I don't feel that way.

Yet your language betrays your feelings.

I would never in my life use words like "stoop to lowest common denominator" or "coddle beginners" when talking about beginners (or other developers in general).

The rest of your message is a thinly veiled contempt at best. So, good weekend to you, too.

We Really Don't Know How to Compute: Gerry Sussman - https://www.youtube.com/watch?v=O3tVctB_VSU

Zebras All the Way Down: Bryan Cantrill - https://www.youtube.com/watch?v=fE2KDzZaxvE

Jonathan Blow on Deep Work: Jonathan Blow - https://www.youtube.com/watch?v=4Ej_3NKA3pk

Simple Made Easy: Rich Hickey - https://www.infoq.com/presentations/Simple-Made-Easy

Effective Programs - 10 Years of Clojure: Rich Hickey - https://www.youtube.com/watch?v=2V1FtfBDsLU&t=845s

The Last Thing D Needs: Scott Meyers - https://www.youtube.com/watch?v=KAWA1DuvCnQ

⬐ christophilus

The first time I watched Simple Made Easy, I didn't like it, even though I'd written quite a few situated programs in my day. A year later, I'd learned Clojure and re-watched it, and it all made so much sense. It's now one of my favorite tech talks.

⬐ alecco

(via Deep Work)

How to Depth Jam: http://chrishecker.com/The_Depth_Jam

⬐ DaviedGabriel

I hope I found We Really Don't Know How to Compute: Gerry Sussman talk with better resolution and camera on the board

⬐ lerax

Gerry Sussman talk is awesome and reflects very well the currently state of computer programming. It's a shame. The worse part: there is people around us with a lot of pride ABOUT DON'T KNOWING TO COMPUTE BUT STILL DOING [INNEFICIENT] THINGS. (sorry for the caps, good bye)

⬐ 0xbadcafebee

Rich Hickey's Greatest Hits: https://changelog.com/posts/rich-hickeys-greatest-hits

⬐ corysama

More Rich Hickey: https://github.com/tallesl/Rich-Hickey-fanclub

⬐ stretchwithme

Rich Hickey is great. I remember his Simplicity Matters keynote at Rails Conf 2012. So clear and insightful.

  https://www.youtube.com/watch?v=rI8tNMsozo0

Being able to explain a complex topic to diverse audiences is not easy to do. Rich does it very well.

⬐ afro88

Link: https://youtube.com/watch?v=rI8tNMsozo0

⬐ stretchwithme

Thanks. Forgot about that.

One of the big benefits of clojure being dynamic is that everything is data (e.g. a map, set, vector or list).

This is what allows reuse.

- The vast core library of functions that manipulate those data structures can be used for everything in your program, cos it's all data.

- Most clojure libraries take and/or return data, reducing the need for clumsy adaptors, or even worse not being able to get at the data you need cos the library writer was really enthusiastic about encapsulation of everything they thought was of no use to consumers.

- You don't have a person class, you have a map with a first name and last name. Now the function that turns first + last name into full name can be reused for any other map with the same keys. (A rather spurious example, but a real one would take a large codebase and an essay to describe)

I can only recommend watching some of Rich Hickey's talks, particularly these ones, they're not entirely about types, but they express the above ideas much better than I can:

- Simple made easy https://www.infoq.com/presentations/Simple-Made-Easy

- Effective programs https://www.youtube.com/watch?v=2V1FtfBDsLU

- Are we there yet? (this one is more about OOP, but unless you're using something like haskell, idris etc its relevant for your type system of choice) https://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hi...

⬐ mbrodersen

The data types in Clojure can be very easily (and better) expressed in (say) Haskell. For example: http://tech.frontrowed.com/2017/11/01/rhetoric-of-clojure-an...

⬐ joncampbelldev

The main issue is that Haskell is not a data-oriented language by default, this means its no fun to push it to be that. For example, I also have to use java in my job, I use persistent (functional) data structures all the time, but Java is not built for it, its not fun. (Although definitely more fun that using Java's mutable structures, ewww)

Also I personally find that to be too much overhead and ceremony in return for some type checking at compile type, as opposed to spec checking at runtime.

⬐ tome

> The main issue is that Haskell is not a data-oriented language by default

What do you mean by "data-oriented language"?

⬐ joncampbelldev

In the grandparent comment's link (showing clojure data in haskell): I'm pretty sure that is not how people code in Haskell, its not how the libraries are usually designed etc etc. Using only data is definitely possible in Haskell, but it's not encouraged by default, the core abstractions are used for concretions of information.

In the same way you can do immutable and functional stuff in java, it's not going to mesh with the rest of the ecosystem or language around you.

⬐ wtetzner

> One of the big benefits of clojure being dynamic is that everything is data (e.g. a map, set, vector or list).

What about this can't be done with types? Simple parametric-polymorphism gets you pretty far. Row types allow you to handle "maps as records" in a type-safe way. The rest is just having support for some kind of ad-hoc polymorphism so that you can re-use your functions on that small set of types (type classes, ML-style functors, interfaces, protocols, etc.).

⬐ joncampbelldev

Again, I would refer you to the Rich Hickey talks, I'm not very eloquent on this. I think its about the manual overhead that constructing your hierarchy of types, plus the cognitive overhead of doing all the fancy things in your brackets.

I'm familiar with the advantages of type systems (my progression was Java -> Haskell -> Idris) but I found my personal productivity (even in larger systems built in a team) was best in clojure. I didn't feel that the guarantees given to me by the type system were worth the mental overhead, a lot of people feel differently (you amongst them I'm guessing :p)

As a closing point, if I were to ever build something that truly had to be Robust in a "someone will die if this goes even slightly wrong" way, I would reach straight for Idris and probably something like TLA+. However most of my development revolves around larger distributed systems communicating over wires, still resilient but in a different way. Mainly I use clojure.spec in core business logic and at the edges of my programs, for generative testing and ensuring that the data flowing through the system is sensible.

This[0] slide from Rich Hickey's Effective Programs talk[1] has really got me thinking about the benefits of costs of typing.

  "The Problems of Programming"

  Domain Complexity
  Misconception

  10x

  Place Oriented Programming
  Weak support for information
  Brittleness/Coupling
  Language model complexity
  Parochialism/context
  weak support for Names
  Distribution
  Resource utilization
  Runtime intangibility
  Libraries
  Concurrency

  10x

  Inconsistency
  Typos

There are some type systems, for example Rust's, which integrate RAII into the language as a first class concept and are able to address some of the upper level concerns like concurrency and place oriented programming. In general, however, there are many problems on this list where more complex type systems doesn't help, or even makes it worse.

[0] https://twitter.com/stuarthalloway/status/926065084652228609 [1] https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ dkarl

To be fair, there are languages where typos and inconsistency naturally fall in the middle tier unless programmers invest silly amounts of time writing and maintaining unit tests to push them down into the third tier where they belong. I can't blame Rich Hickey if he doesn't think much about those languages, but it's an important qualification for people to make when they read this slide in the context of their own work.

⬐ lewisl9029

I feel the context is important here.

This slide is not a comment on the frequency of these problems, because typos are definitely one of most if not the most frequently occurring issue in the day-to-day work of most developers, and I'm sure Rich Hickey realizes that.

Rather, this slide is a comment on what he defines as the "severity" of these problems, and one of the important manifestations of severity is the cost of getting these wrong.

With the exception of certain business domains where getting things perfectly right the first time is paramount, such as financial transactions and safety/security-critical programs, the cost of typos and inconsistencies is generally minuscule because they're practically costless to debug and fix. In fact, they often cost so little that we end up instinctively fixing typos and inconsistencies countless times throughout our day to day development processes, often without ever even making a conscious effort to identify and fix them. Even if the occasional typo makes it past our tests and into production, these issues are almost always trivial to debug and fix (and if they're not, and you don't happen to be in of these domains where getting things right the very first time is paramount, then oftentimes that can be the smell of some deficiency in your deployment/monitoring setup, or a manifestation of some other more subtle, but more severe problems listed on the slide, such as brittleness/coupling, place oriented programming, weak support for concurrency/names, etc).

Of course, being able to outright eliminate these entire classes of errors is a legitimate benefit of a static type system, and Rich Hickey acknowledges this benefit later in the talk. If you're in one of those business domains where you absolutely cannot afford to let typos and inconsistencies sneak their way into your production systems because the they'd be disproportionately costly to fix, or result in consequences that you're not willing to accept, then static typing can be very useful for providing those safeguards.

However, I think he also brings up an important point that static type systems and the act of flowing types around in your system is often a significant source of coupling, and that coupling is a much more severe problem when it comes to maintaining a piece of software over the long term. So it's important for each team to assess that tradeoff carefully in order to gauge if they're willing to take on that additional level of coupling in exchange for eliminating the possibility of typos and inconsistencies from reaching production.

⬐ thesz

You may decrease coupling with the use of interfaces or type classes.

This way you do not use concrete types in the construction, you link things with requirements on/between the parameters.

Please take a look at the Expression Problem: https://en.wikipedia.org/wiki/Expression_problem

This thing is an essence of question how to design low coupled system when initially high coupling is expected. There are solutions for most languages and, probably, for your statically typed language of choice (from statically typed languages).

⬐ s6o

After watching that talk, I would have loved somebody in the audience to point out instance like this https://raygun.com/blog/10-costly-software-errors-history/ and ask for Hickey's comments.

Needless to say I really don't agree with that list, as history shows the inconsistency, typos, lack basic units and the ability to express them in a computation etc., have been more catastrophic than he believes them to be. And these issues cannot really be fixed without a proper type system, that will not allow stuff like that to get into production.

And a good type system cannot be optional, e.g. clojure's spec, There are number of studies that show that will power, is not enough, there needs to be and environment - a type system - that helps to avoid and catch mistakes.

⬐ sooheon

I see nothing in those 10 costly errors that fundamentally disagree with Hickey's ranking, or advocate clearly for the need of type systems.

I also question the value of taking too much stock in outliers. How many of us will be working on the next Mars orbiter?

⬐ flavio81

>After watching that talk, I would have loved somebody in the audience to point out instance like this https://raygun.com/blog/10-costly-software-errors-history/ and ask for Hickey's comments. Needless to say I really don't agree with that list, as history shows the inconsistency, typos, lack basic units and the ability to express them in a computation etc., have been more catastrophic than he believes them to be.

From the list you cite, only three of 10 errors are caused by "typos", "lack of basic units" and type errors.

And this is just a particular list.

Recalling Rich Hickey list, the things that cause more problem in code, from worst to less damaging:

  Domain Complexity
  Misconception

  10x

  Place Oriented Programming
  Weak support for information
  Brittleness/Coupling
  Language model complexity
  Parochialism/context
  weak support for Names
  Distribution
  Resource utilization
  Runtime intangibility
  Libraries
  Concurrency

  10x

  Inconsistency
  Typos

I'm pretty sure that if we compile a list of real-life cases where the major reasons of failure according to Rich Hickey have caused trouble, the list of examples would be immense, compared to "typos".

⬐ vanilla_nut

I haven't written as much Rust as I'd like, but one of my favorite things about the language (there's a lot of competition for "favorite" things about Rust, to be fair) is the way that types are required in function signatures, but not within functions. So if you can see the declaration of something, you can always tell what it is -- it's right there. But if it's done in a function somewhere, or if you're just trying to get a sense for the types passed around in a library, it's all on the surface of the signatures. I personally find the lack of declared types in python/javascript absolutely debilitating on large projects, so that really appeals to me.

⬐ carlmr

You should check out F#! it has amazing type inference (even for functions in some simple cases, although I prefer my functions with types). The type system is still strongly typed and static. So it looks Pythonic, and behaves C#-ic.

https://fsharpforfunandprofit.com/posts/type-inference/

Rich Hickey, Effective Programs - an examination of the essence of what we do as programmers (for many of us, wirting "situated programs"), and a spirited defense of dynamically typed languages: https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ throwaway7645

Seconded...good talk!

Effective Programs - 10 Years of Clojure - Rich Hickey: https://www.youtube.com/watch?v=2V1FtfBDsLU

Might help to hear the man himself explain the motivations:

https://www.youtube.com/watch?v=2V1FtfBDsLU

Citation needed?

Anecdotally, I've developed large projects in C++ and Java (I know, they're pretty lame static type systems -- but certainly the most popular static type systems) and also in Python and Clojure and I really haven't seen much benefit in static typing in regards to software defect rate or quality. Static typing make auto complete and refactoring tools easier, for sure, but it also slows down ease of experimentation (and writing generic code can be painful, although other static languages especially type inferred ones fare better here). I buy into Rich Hickeys view on this topic[1] and that's one reason why I like Clojure: it gets out of the way, but it provides me with the tools I need to verify or validate my data (eg on the module or application boundaries).

I've played around with languages that have fancier type systems (Haskell, various ML's, briefly ATS) and am very interested in Rust (but have yet to use it), but they haven't really provided enough benefits for the effort of describing the types.

Note that I used to be very heavily in the static typing camp and I still very much like the idea of static typing, I just don't think we have found a static type system yet that has the right balance of convenience and safety and actually catches the right kinds of errors (as described in the below talk).

I guess my point is that its not quite clear that the next generation of successful languages will all be statically typed. In fact, current trends would suggest otherwise (most of the popular languages are dynamically typed) although perhaps that depends on your definition of "successful".

[1] https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ hasenj

> I really haven't seen much benefit in static typing in regards to software defect rate or quality

Hold it right there. I've never seen anyone argue that static type systems prevent bugs.

I mean they do prevent silly bugs that occur from mistyping variable/property names but I've never seen anyone claim that they eliminate other classes of bugs.

The biggest benefit of static type checking is you know what all the variables are.

    def checkout_cart(customer, payment_methods, cart, session):
        # body

What the hell is customer? What is payment methods? What fields are available on these objects? What methods can you call on them? no freaking idea.

Of course, this kind of code is confusing in Java as well, but for a different reason: Java conventions encourage a kind of obtuse programming style where everything is hidden behind layers of abstractions of factories and managers, so that even when everything is typed, you're not sure what anything is doing because all the data that matters is private and so are all the methods that actually do anything useful. All you're left with is an abstract interface that can sometimes be just as bad as an untyped variable. But this is mostly a cultural problem. (I've digressed).

> Static typing make auto complete and refactoring tools easier, for sure, but it also slows down ease of experimentation

Java slows down ease of experimentation because it requires tons of boilerplate code for even the simplest tasks.

It's not the static type checking.

If anything, static type checking helps experimentation because you can change your mind quickly and the compiler will help you catch all the stupid mistakes that can occur from mismatching types or mistyping variable names. This removes a huge cognitive tax and makes programming more enjoyable. Although I will concede this is subjective.

⬐ dkersten

Except that static typing doesn’t help much there either (unless perhaps you’re using Frink with its units). The type doesn’t carry enough information. For example, knowing that something is an integer really doesn’t give you enough context about what that integer means or is used for. If it’s an object or strict, ok, then it helps to document, but if it’s primitive or standard collections...

Anyway, in Clojure, we now use spec to specify the shape of data that we expect, with nice descriptive namespaced keyword names. It helps in validating data entering the system, generating test data and as documentation.

⬐ tome

> I've never seen anyone argue that static type systems prevent bugs.

It's extremely common to claim that static typing prevents entire classes of bugs (and I agree!). Here's just one instance of such a claim I found on Google:

https://news.ycombinator.com/item?id=10934134

⬐ crdoconnor

>Hold it right there. I've never seen anyone argue that static type systems prevent bugs.

Really? I see this every single time the subject is brought up. And, to be fair, they do catch some bugs, it's just that they do so at a cost.

>What the hell is customer? What is payment methods? What fields are available on these objects? What methods can you call on them? no freaking idea.

And, if they are all strings, how much more of an idea do you have?

Static typing does not necessarily help solve this problem - a combination of reduced scope(i.e. looser coupling), more specific variable naming and higher cohesion (e.g. having a customer object) do.

Moreover, there's a super easy way to figure out what all of those things are and figure out how you want to change it - run a behavioral test and launch a REPL when it hits that function.

At that point you can inspect customer, use autocomplete on it and even experimentally run code.

>If anything, static type checking helps experimentation because you can change your mind quickly and the compiler will help you catch all the stupid mistakes that can occur from mismatching types or mistyping variable names. This removes a huge cognitive tax and makes programming more enjoyable.

Behavioral tests perform this function equally well, if you have them.

⬐ hasenj

What is behavioral test? Like, I don't understand all these weird paradigms that people come up with to deal with the deficiencies of dynamic typing.

If declaring structs is seen as costly overhead that complicates coding, tests are when more cumbersome.

⬐ crdoconnor

>What is behavioral test?

A behavioral test is a test that tests the behavior of a piece of software, as opposed to a test that checks types or implementation details or something else that isn't behavior.

It is perhaps not necessary to write tests like these in languages that produce code that does not have bugs. I have yet to encounter such a language.

>If declaring structs is seen as costly overhead that complicates coding, tests are when more cumbersome.

You do not write tests then?

⬐ hasenj

No I do not. I specially don't write unit tests.

⬐ acdha

> IMHO behavioral tests perform this function equally well.

I think a common pitfall in these discussions is to compare the worst case examples rather than reasonable quality codebases. I'd be far more interested in, say, time/cost to correct result metrics for a well-maintained Python codebase which has reasonable use of tests & linting (e.g. flake8) to an equivalently-proficient team using a statically typed language.

⬐ dkersten

If we're discussing well-maintained code, then I would expect that the public interface is documented, at least in docstrings. Then I also know what the parameters are.

⬐ acdha

Agreed — I'm just wondering about how to quantify the impact of various changes. A dynamic language project with no tests, etc. is going to look like a selling point for static typing but I suspect the real-world bug counts for, say, a Python project using mypy (or even flake8 + tests + coverage) is going to be a lot closer than you might think from how heated these discussions get.

⬐ crdoconnor

There was a study that did a line by line translation of 4 python projects to haskell and caught some bugs (between 0 and 4 per project): http://evanfarrer.blogspot.co.uk/2012/06/unit-testing-isnt-e...

I got the impression that the bugs found were either not at all serious (e.g. throwing a typeerror on malformed input instead of some other nicer kind of error) or were in areas of the code not covered by tests.

Unfortunately the author does not rate them by severity.

⬐ acdha

Thanks - that's a lot like what I had in mind!

My gut feeling is that dynamic typing + tests & static analysis is faster than very heavyweight languages (e.g. Java) but probably near or less than languages with more advanced typing systems like Haskell or Rust, but I'd really like to see something more comprehensive than a subjective opinion.

⬐ tome

> I just don't think we have found a static type system yet that has the right balance of convenience and safety and actually catches the right kinds of errors (as described in the below talk).

"Please don't be an uninformed Rich Hickey talk"

<Clicks>

"Oh, it's an uninformed Rich Hickey talk"

⬐ dkersten

Explain?

Dubious. The maintenance argument is weak. https://www.youtube.com/watch?v=2V1FtfBDsLU&feature=youtu.be...

⬐ tome

I'm not sure what you mean. I thought it was self-evident that if your interface is an abstract type then you can change the implementation without breaking consumers. If you don't think it's true then please can you explain how? I thought the Clojure argument is not that types don't provide beneficial encapsulation, but that it's not worth paying the cost for them. Again if you know any different I'd be pleased to hear.

⬐ nickbauman

Apples and Oranges. We're talking about changing the interface. Did you watch the video segment?

⬐ tome

> We're talking about changing the interface.

Who's "we"? I'm talking about changing the implementation.

> Did you watch the video segment?

I watched the bit where he said if you add a constructor to a sum type you have to change everywhere it's pattern matched. True. I'd love to see Clojure's solution where you don't have to change any existing code!

EDIT: Oh wait, I think it's even worse than that. I think he's talking about product types. In which case you should use projection functions and indeed you need to make no changes to old code!

I would recommend taking the time to actually watch the talk [0]. It's well worth it.

[0] https://www.youtube.com/watch?v=2V1FtfBDsLU

⬐ hyeomans

Thanks for this.

I've really taken a shine to Clojure. Parentheses and dynamic typing initially kept me away, but they're far from the thorns they've been made out to be.

Parentheses? You don't even see them after a short while--as a beginner, you use something like parinfer, you are good to go. That, and like all lisps, you begin to appreciate programming directly in the abstract syntax tree: the language you type and the way it is interpreted aren't disjoint things.

Dynamic typing? Being functional, immutable by default, and data orientated steal the vast majority of the sting from that in my opinion--and leave you with often downplayed advantages. On the last point, "data oriented", to put it succinctly, data itself is first class or the prominent currency among Clojurists, rather than a zoo of abstractions that never quite fit and and all have to be reconciled and made to play with one another (and often the reason you need types everywhere to keep it all straight).

But where types--or validation--are more crucial near the boundaries of the API, there are outside the box solutions like Spec that don't rely on a baked in type system.

The recent keynote by Rich Hickey covers a lot of these points (and far better than me):

https://m.youtube.com/watch?v=2V1FtfBDsLU

⬐ jasim

If you have used a statically typed language in the ML family (OCaml, Haskell etc.), could you contrast the development experience between the two?

⬐ preordained

I spent a good year or so with Haskell. I guess at first blush, one thing that jumped out was that Clojure was made from the jump with a focus on productivity and pragmatism, whereas you never feel too far from the academic origins of Haskell. Clojure is a very shrewdly designed--it's not proud, it takes leverage where it can find it, and makes some very deliberate and sensible trade offs. A large aspect of that is the JVM it sits atop of. You will know that it's Java underneath the hood, but this is not a bad thing at all. You get to have your cake and eat it too. It's done really well in the sense that while it would be awkward and not Clojure-like to promiscuously call native Java--especially code with side-effects--it it is perfectly alright, and practical, to pull in any of the multitude of proven Java libraries where it can obviously save you time and effort. There is no snobbery in the core language itself or in the community about "ew, icky Java". Or put another way, there is no from-scratch ecosystem like in so many languages where a new language means starting over (so the language needs a crypto library, and a web server, and...) Not to mention, many of the utility libraries out there are largely functional in nature anyhow (you can use a massive chunk of the wealth that is Apache Commons as just an arsenal of pure functions).

In nitty gritty terms, programming in Clojure actually feels a lot like programming in Haskell to me, without the heavy type system. Lazy infinite lists, all your FP classics like map, reduce, filter, etc. work exactly like you would expect. But I don't miss having to create a stack of monad transformers in an area of code where I need to mix IO and state or something...and using type synonyms to mask the monstrous Turing-complete type signatures I had to make to describe it... In Clojure, you just put the "dirty" work in its own function, clearly separated from the pure functions (which are the de facto default), and you push them to the boundaries of the system while passing extracted data into pure functions (just like Haskell). If you did anything different, you'd get the stink eye from even the least principled Clojurists. It's just not done. A ball and chain type system is unnecessary.

The lack of (explicit) types has been at the very most a minor nuisance. Occasionally I mistype something or miss a parameter, you're going to see the error pretty quickly in the REPL. If you run or test your code at all, the pedestrian type errors (oh, that was a string not an int) will be wrung out immediately. That benefit of type systems (preventing typos, essentially) has been much, much overstated IMO. Given you're going to be in the REPL a lot with Clojure, you'll be iteratatively developing and running your code constantly, so it's really not an issue. As far as refactoring, I think that's where there is more meaningful benefit. I use Intellij/Cursive IDE, and the refactoring capabilities are good--much like if it were a Java project (and there are hints and such for the typos I mentioned earlier, autocomplete, etc). So, while it is more of a challenge for IDEs, there are some excellent ones that do a pretty bang up job of inferring types for you.

I haven't used Clojure on the job (Java dev by day), but I would have no problem using it for something "real", whereas I never felt anywhere near confident in using Haskell for something outside my own fiddling.

⬐ jasim

Thanks for the reply, it was very informative.

I've programmed in dynamic languages for most of my career (about a decade now), and have always missed types. I picked up OCaml (Reason flavoured) recently, and I've been able to enjoy the Hindley-Milner typesystem without worrying about side-effects with complex types, since OCaml is a fine mix of imperative and functional programming.

Language preferences are usually framed as an objective measurement, but I've always found it to be based on our personal experiences. Since you've been doing a lot of Java, I'd surmise that the clean, functional yet dynamic nature of Clojure with Java interop is appealing to you. Me on the other hand has been burnt enough times with dynamic systems that keep shifting from under your foot (Ruby metaprogramming), test suites that makes refactoring more difficult (it provides safety, but the manual labour is unforgiving), and a general sense of dread everytime I put systems into production. But more than anything else, the biggest pain has been the sheer difficulty to refactor growing codebases.

Large dynamic systems, if they manage to not crumble from the inside, usually gets hammered by a sweeping change in external reality (terminology changes, shifting of levels in domain hierarchy etc.). When there is a pressure to ship we add a new mapping without touching existing naming, and slowly the domain names in the system exhibits a tangled relation with the actual domain.

That's just been my experience, and so I'm here learning me some good old OCaml and being very happy with what I'm getting out of it.

⬐ freshhawk

Have a look at Spec and the design philosophy behind it if you are still interested in Clojure at all. Especially how Hickey feels about open specs, versioning, and names. It is a pretty opinionated stance to take on solving those issues, but it convinced me and I no longer have the worries you describe about the dynamic nature of a large Clojure system.

What do you mean by "doesn't have monads?" You just write monads in Clojure. They are only a structural formalism. [1]

If you meant doesn't have the Maybe monad built in... why would you want that tangled up in your data? [2]

  [1] https://github.com/khinsen/monads-in-clojure/blob/master/PART1.md

  [2] https://youtu.be/2V1FtfBDsLU?t=2381

⬐ platz

> "SPJ in an excellent series of talks lists these advantages of types... the biggest thing left out of clojure..."

> "The biggest merit he says is in software maintenence, and i really disagree with just a lot of this. it's not been my experience; the biggest errors are not caught by these type systems; you need extensive testing to do real-world effectiveness checking."

I think Rich is dead wrong on types not providing good tools for software _maintenance_.

He is attempting to discredit types because they do not catch all errors, or the toughest errors; that's fine, I don't agree with people that would say types do that for you.

I think dynamism is great for speed of development and flexibility; but my impression about trying to debug and maintain clojure code is that one has to spend time tracing back the source of errors and shape of maps (pre-spec).

Those issues are somewhat mitigated by having a good type system, and what you loose on the flexibility going with types, is a trade that I think returns much more that what you give up.

⬐ flavio81

>"The biggest merit he says is in software maintenence, and i really disagree with just a lot of this. it's not been my experience; the biggest errors are not caught by these type systems; you need extensive testing to do real-world effectiveness checking."

I think I fully agree with Rick Hickey on this, and I have mentioned the same before here in HN. The biggest and most serious errors are not caught by the (statically checked) type systems.

I agree that simple type errors that, on a dynamic system without compile-time checks are not caught, will be then caught at runtime and you will say "oh, i need to get back and correct this, damn it". But this is a minor annoyance, that takes small time compared to the bugs i'm mentioning in the previous paragraph; bugs that can take days to be resolved.

⬐ runT1ME

What kind of errrors do you think are not caught with static typing? Usually people who say this aren't familiar with languages that have type classes...

⬐ flavio81

>What kind of errrors do you think are not caught with static typing?

Citing awj's post above:

"Some of the biggest errors are subtle failures in the implementation of business logic."

Other examples:

- bumping into unkonwn bugs of a library you are calling - implementation of an external (library) function behaving differently than the documentation - various memory leak problems - locking issues - race conditions

etc

⬐ runT1ME

>"Some of the biggest errors are subtle failures in the implementation of business logic

Static typing can prevent errors in business logic.

>various memory leak problems - locking issues - race conditions

All are bugs that static typing has had tremendous success in preventing. Have you any experience in a HM like type system?

⬐ flavio81

>All are bugs that static typing has had tremendous success in preventing.

Please do elaborate, it could be a great post.

But If you mean to say that locking issues can be prevented because you can apply STM (software transactional memory) by using suitable Monads, then, well, i can also have STM in a dynamic language, even applying it by wrapping code in an "atomic" context (i.e. see the STMX library for Common Lisp) so the statements contained within are performed atomically (and thus guaranteed to work or either roll-back.)

As for memory leaks, Haskell (if that's the HM language you had in mind) automatically manages memory, so I would guess this isn't a problem at all. However, automatic management memory management is orthogonal to "static vs dynamically typed".

Thus, i'm curious, and yes, i don't have enough experience with HM-type languages, which should be the way to go if one wants to use static typing...

⬐ tome

> i can also have STM in a dynamic language

How can you roll back failed transactions without explicitly typed effects?

⬐ platz

Clojure tracks and rollbacks the STM-bound variables according the STM rules; it is not intended to prevent you from doing other effects inside those transactions; you are supposed to have an understanding not to do that in the transaction blocks.

It helps a bit that most clojure datastructures are immutable/persistant by default.

⬐ tome

In other words Haskell's STM prevents bugs that Clojure's merely supposes the programmer not to have written.

⬐ lliamander

I would say the main point of types is to prevent uninteresting bugs. For me that's a selling point because I don't want to be distracted by those kinds of bugs.

⬐ tome

What the two replies (at the time of writing) to this comment have missed is that maintenance and avoiding errors are not the same thing!

⬐ awj

No, I think his point is valid. Some of the biggest errors are subtle failures in the implementation of business logic. Cases where what the code does and what it looks like it does fail to line up.

Static typing can be brought to bear on those (encode your business logic in the type system), but what parts that fails to cover still need good testing. Plus, how do you test that your types correctly encode your business logic?

I'm still more in the static camp than out of it, but I can recognize that the problem is not as cut and dried as we want to believe. There's are reasons this pendulum has swung back and forth over the decades, and I don't see anything to suggest those reasons have changed.

⬐ platz

> how do you test that your types correctly encode your business logic

types, as most commonly used do not _directly_ encode business logic; they just provide constraints i.e. it is not an all-or-nothing situation. You can still write unit tests.

⬐ awj

Ehh, semantics. "How do you ensure that your constraints are correct?" is effectively the same question. Any nontrivial set of constraints will need to be validated, and validating those constraints is a pretty big challenge that the compiler by definition cannot help you with.

⬐ yogthos

What you have to demonstrate is that static typing catches a statistically significant amount of additional errors over plain unit tests. Nobody has been able to show this to be the case so far in practice.

⬐ platz

Requiring statistically significant studies pretty much guarantees no provable claims can be made for any side; since that amounts to a social science laden with shaky assumptions and ambiguous modeling. Since it's is a standard of proof that no-one can achieve, i don't see why mention it here.

But to go along with this for a moment, I liked your response to this paper here https://www.reddit.com/r/Clojure/comments/73q8c2/a_largescal... ;

Though I'm not sure how we're deciding what is "statistically significant" in contrast to what the researchers have decided is statistically significant. Were their calculations incorrect?

However, you claim to reveal their "real conclusion", in contrast to their Actual Conclusion, in which they do indicate that "Among functional languages, static typing is also somewhat better than dynamic typing."

Still, I do not place much stock in these studies for the aforementioned reasons.

I also think you have to be careful about where you have placed the "burden of proof". (You've placed it on requiring poof that static types provide X benefit. One can just as well turn this around and require poof that eschewing static types provide X benefit; so we cannot a-priori place such demands in either direction).

⬐ yogthos

The null hypothesis has to be that both approaches are equally effective. There simply isn't any evidence to suggest otherwise.

At this point, it's premature to discuss whether static typing or dynamic typing affords benefits. The first step would be to look at a large set of real world open source projects written in different languages. If we see empirical evidence that projects written in certain types of languages consistently perform better in a particular area, such as reduction in defects, we can make a hypothesis as to why that is.

For example, if there was statistical evidence to indicate that using Haskell reduces defects, a hypothesis could be made that the the Haskell type system plays a role here. That hypothesis could then be further tested, and that would tell us whether it's correct or not.

I agree that these studies are hard to do, and that they'll always be imperfect. However, that's still the best tool we have for approaching this empirically.

⬐ platz

> The null hypothesis has to be that both approaches are equally effective. There simply isn't any evidence to suggest otherwise.

Sure, I think that is in the ballpark of an acceptable null hypothesis, for a given experiment. I'm not sure one can identify a null hypothesis that is universal among all experiments, but this captures the idea, sure.

⬐ kasbah

I submitted this link before I had watched the whole thing. As someone who has only dabbled in Clojure I think there are a lot of interesting ideas in there but found the type-system bashing pretty off-putting.

I am now watching his "Simple Made Easy" talk [1] after I have heard it recommended on a few functional programming related podcasts. Again really interesting stuff but I encountered another cheap shot at typed functional programming ("You can't use monads for that! Hurr hurr hurr").

Given how well received these talks seem to be by people that enjoy programming with advanced type systems I would have have really expected a more balanced discussion and some acknowledgement of the trade-offs between dynamic and statically typed functional programming.

[1]: https://www.infoq.com/presentations/Simple-Made-Easy

⬐ gldalmaso

I really like Rich's views and find Clojure very interesting as well. That said, as a Java shop with Javascript frontend, nowadays the bulk of complexity in our code base seems to accumulate in the frontend due to mixed skill levels of the team and lack of opinionated structure in the language. This leads to some rather messy code that even skilled devs are afraid to touch because of lack of feedback from the IDE that some refactor is working without loose ends.

The same problem with the same people just doesn't happen in the backend and I link that to static typing and IDE maturity. We have started to adopt Typescript and are seeing improvements already.

We just have to live with the fact not all developers working in the code are mature enough to avoid language and code organization pitfalls. Refactoring should be mostly a safe endeavor, even if only structurally.

This is the main reason I wouldn't suggest Clojure for our team.

⬐ hellofunk

I agree that there is definitely added discipline needed to succeed well in large dynamically-typed projects. I also think that learning to build large projects in such languages is like running your marathon training high in the mountains, so when you get back to sea-level your body feels the joy. You are forced to write very clean code in Clojure if you want to easily maintain it later. That's a great skill that translates to any other language where less discipline might still get you far.

⬐ lilactown

I think it's something else, as well. Rich even mentions it in his talk: languages like Java (which I'm reading to mean "statically typed") are great at mechanical tasks. Front end programming is mostly filled with mechanical tasks: scaffold this structure/layout. Wire up these events. Make this thing blue/bold/etc. Change the state when these events happen. It's fairly predictable in structure in line-of-business apps, at least once you're following an intelligent structure, e.g. the Elm architecture.

UI/Front end dev, IMO, can gain quite a bit from static typing. I'm a huge fan of clojurescript, it's what I reach for whenever I want to work on something, but I'm super excited about ReasonML for the future of my team; we struggle with our JavaScript code base right now due to the lack of imposed structure and feedback for our weaker developers.

I love Clojure and I think it makes sense in a lot of domains; most of my back end development is "take this data, transform it according to some nebulous business rules, and poop it out to some other place," which Clojure is amazing for. It's great for applications that don't require a lot of "wiring", and require a lot of "flow". UI programming is, for the most part, wiring things up. It's not that Clojure/Script is not up to the task (I think e.g. re-frame, and the stuff being done with Fulcro, is amazing) but I definitely see the benefits of static typing more in that domain.

And like Rich said, if you're doing UI it will usually completely dominate the problem space you're working in. So pick the right tool for the job. I'm not convinced TypeScript is the way exactly, but like I said, ReasonML and Elm are super promising.

⬐ sheepmullet

> but found the type-system bashing pretty off-putting.

Why do you think it is type system bashing?

He is justifying why he didn't add types to Clojure. In his experience they add more complexity than they are worth.

The reason he talks about it at all is there are a lot of static typing enthusiasts who talk about static typing being a game changer.

In my experience static typing is a +-2-3% productivity influencer. You get a bit better IDE experience and refactoring is easier. On the other hand I've also found I need to refactor my C# code far more often than my Clojure code.

⬐ whalesalad

Gotta take the good with the bad. Tons of knowledge and wisdom to be gained from the FP folks but sometimes they do have the cheap shots and the bias of the community.

Ie, It’s easy to hate and joke about things like SQL databases and JSON when you live in your own utopian fairy land where everything is Datomic and EDN.

⬐ joncampbelldev

I believe the quote you're referencing about monads is "this is meant to lull you into believing everything I say is true, because I can't use monads for that" (referring to an animation of a stick figure juggling)

⬐ hellofunk

The new Conj talk is certainly an interesting look at one man's (or one community's) look at static typing. However, as much as I admire Rich, some of the points he made don't resonate with me, particularly the one about how compile-time checks to catch minor bugs in syntax are not a particularly useful feature of static typing. I certainly disagree. As someone who writes Clojure all day long right now for a living, I am constantly dealing with runtime errors that are due to minor typos in my code that I have to track down, and this time would be greatly saved by having a compiler tell me "on line 23 you looked for :foo keyword in a map but you meant to type :foobar, so that's why that was nil" and many other similar woes.

I love Clojure but I really miss static type checks.

The other item in his talk I do not agree with, he says (slightly paraphrasing) "in static typing, you can pattern match on something 500 times but if you add a case, you have to update those 500 matches to handle the new case, when really they don't care about this new case, only the new logic needs to consume this special case, it's better for the producer to speak directly to the consumer". Well, in languages like OCaml, Swift, Haskell, it is a feature that pattern matches much be exhaustive. This prevents bugs. In most cases, I'd expect that if I add a case to an enum, the chances are good my existing logic in pattern matches should know about that. Maybe not all, but a lot of them will. It's nice to have the compiler guide you to those places.

I certainly like how fast I can write programs in Clojure, and I like the minimal amount of code that makes refactoring and rewriting fairly straightforward since there is not a lot of time investment in the existing number of lines, and I like the incredible elegance of Clojure's approach to functional programming.

But I do miss having much greater compiler assitance with typos, mis-typed argument order to functions, mis-typed keyword names, etc. Would really save a lot of time.

⬐ DigitalJack

Still reading your comment, but after the first paragraph, I would kindly suggest looking at clojure.spec. It's helped me immensely in similar problems.

⬐ hellofunk

I suppose you'd have to use spec/assert for every instance of destructuring or "get" or "get-in" to avoid common mistakes. That's a lot of asserts everywhere.

⬐ DigitalJack

I don't understand this comment.

I spec types, and then I spec functions that need that type. But not all the function, just the heavy use ones.

I usually don't instrument the spec'd functions unless I'm actively debugging.

edit:

after having a minute to think on it, do you mean to catch a typo in the use of get, get-in, etc? I haven't tried that.

I suppose you could wrap get, get-in with a nil check or something.

⬐ hellofunk

> I suppose you could wrap get, get-in with a nil check or something.

Indeed I suppose the solution would be write wrappers around common getters that allow you to pass a spec to the query and have them automatically assert that everything is what you expect.

⬐ keymone

> As someone who writes Clojure all day long right now for a living, I am constantly dealing with runtime errors that are due to minor typos in my code that I have to track down, and this time would be greatly saved by having a compiler tell me "on line 23 you looked for :foo keyword in a map but you meant to type :foobar, so that's why that was nil" and many other similar woes.

i wonder if this is because it really takes a quantum leap in one's development style between <insert your previous programming language> and clojure/<insert your favourite lisp>? as long as your environment allows for effortless evaluation of code you're writing, you'd be getting this feedback no slower than the edit/save/compile/retry cycle.

⬐ hellofunk

If your typos are triggered by UI events, then you often won't see these problems until interacting with your UI (I work mainly in Clojurescript). Further, these typos may not get noticed at all for a long time if a code path is never taken. Of course, that's what unit tests are for. But writing tests takes time also. I'm not sure it's worth the trade off to spend the time writing those tests that I could spend writing in a more statically-typed language that would catch some things that tests wouldn't be needed for. (Besides, writing tests for UI stuff is pretty hard).

I am griping, really, because I cannot stress enough how nice it feels most of the time to write Clojurescript. But in complex projects, there is not doubt that a lot of time gets spent on things that wouldn't need to be spent if the language had even a very basic type system to back up the syntax for some things.

⬐ keymone

which ui library are you using? not claiming to be an expert, but i always found it easier to test programs when logic is completely decoupled from event flow. but yeah, UI can be pita.

also isn't clojure.spec useful for describing and asserting the shape of data taken and returned by functions?

⬐ hellofunk

Clojure.spec is useful for a lot of things, but unless you are adding spec/assert to nearly every destructuring or "get" or "get-in" then it's still easy get nils running through your data transformations because you mistyped a keyword or something.

Also there is not a good answer for asserting the value of a function passed to another function; the return values of functions can be spec'd but they are not included in an assert test.

⬐ sooheon

> the return values of functions can be spec'd but they are not included in an assert test

I agree this is a shortcoming, but that is why this library exists: https://github.com/jeaye/orchestra

⬐ None

None

⬐ christophilus

My team recently settled on TypeScript instead of ClojureScript, as TS is the safer bet, more familiar, more consistent with the existing project's tooling, etc. But man... I've taken a handful of files and written them in both TS and CLJS. CLJS is just so much shorter and elegant. I sometimes think we made the wrong decision.

⬐ athousandcounts

ClojureScript is great with Reagent or re-frame... If you write Angular use TypeScript. If you use React, ClojureScript! It's a match made in heaven.

⬐ christophilus

Yeah. I've built toy apps with re-frame, and really liked the way the code looked. But my team is pretty Jr other than me, and I wasn't sure if ClojureScript would work well for us as a team. VS Code is our editor of choice, and it is just really a good environment when paired with TypeScript.

Also, my experience with Rails really has me fearful of doing any serious, big work, in a dynamic language.

⬐ whalesalad

Rich is always a great speaker. Sometimes he gets a little esoteric and you hear all the monad loving neck beards giggle in the audience. Then again, he invented a lisp that runs on the JVM. Usually though he is very pragmatic and real-world. I love how often he says things like “the type checker in my compiler doesn’t matter to the users of my program” or, “the perfect, most beautiful search algorithm doesn’t matter if it can’t fit into a web page and work when your user hits enter”

So... even if you don’t care about Clojure or functional programming I’d certainly suggest listening to some of Rich’s talks. He’s a very sharp guy but his format is very friendly.

⬐ amrrs

I have been to a local Clojure conference once. Most of the guys who started practising Clojure was inspired by Rich Hickey's functional programming talks. I'm wondering how many such BDFL inspire users to adopt something new just with talks!

⬐ iLemming

I didn't even know about Rich Hickey talks and "Simple made easy" when I decided to start learning Clojure. I came to conclusion that I needed to learn a Lisp after learning Emacs and dubbing in writing small elisp functions. I had no idea and was really surprised when I found out how awesome Lisp can be, so I looked at the current state of Lisp at the time. And then after learning a bit of Clojure and Clojurescript and seeing things people were building with it, I quit my job. I really wanted to use Clojure full-time. Never in my life I ever before have had this urge to learn a language and build things with it.

⬐ keymone

are people without a degree in type systems even allowed to express their opinion on dynamic vs static? because if their opinion is automatically viewed as inferior whenever a person with a degree chimes into a discussion, isn't dynamic typing stuck in the limbo of not having competent defenders and the only path for a defender to become competent is to spend years of their lives learning the opposite thing?

i think (as a person without a degree in type systems) that relative success of Erlang and Clojure are a testament that pragmatic approach works, and that is not in any way evidence that static typing is inherently bad or can't work.

⬐ dgllghr

Erlang & Elixir have a very interesting approach to typing. The support for pattern matching in function definitions provides a kind of structural typing for functions. And they both embrace typespecs for type analysis through a separate program analyzer. So a lot of the benefits of static typing are there without the static types.

⬐ keymone

clojure has destructuring, which isn't quite pattern matching but it does help to detect wrongly shaped inputs. also dialyzer is amazing and i wonder if clojure.spec can be used like that.

⬐ DigitalJack

clojure has pattern matching too. As a library. Much of the interesting functionality of clojure comes from libraries. (core.async, core.spec, core.match, etc..)

The matching library is good. I wouldn't use it in a tight performance loop, but as a productivity booster it's very nice.

⬐ hellofunk

Without a degree? I think that work experience in both dynamic and static languages is enough to offer a perspective. I don't have a degree in type systems but have done a lot of work in both Clojure and also C++ and Swift. I still can't decide if I prefer dynamic or static, they have nice tradeoffs.

⬐ cutler

Relative success? In what sense? As far as the jobs market is concerned Erlang/Elixir and Clojure have failed to make an impression. Ok, Wallmart, Bleacher Report etc. but search Indeed.com by job title and you'll get what I mean.

⬐ iLemming

according StackOverflow survey - Clojure developers are the most well paid in the industry. What a miserable failures they are.

⬐ keymone

that's why it's called "relative", isn't it?

Erlang and Clojure are undoubtedly a success for me in terms of productivity.

Erlang is undoubtedly a success in a telekom industry since 90s, where extreme reliability is a basic requirement. isn't static typing promoted as a path to achieve reliability "once it's compiled"? go figure.

imo, that majority of developers are being indoctrinated into false-OOP paradigm is rather unfortunate historical curiosity than a success indicator.

⬐ whalesalad

When you say degree are you specifically referring to an academic certificate? Honestly, real world experience and decades of software engineering trump anything you can get from a university.

⬐ keymone

i agree, but not everybody does because i do find this sentiment repeated in such debates in different forms. for instance: https://i.warosu.org/data/g/img/0504/16/1442936748844.png

⬐ iLemming

One thing that was a personal shocker for me to hear, from not just anyone, but Matthias Felleisen himself - "My research group has investigated the topic for almost 3 decades, and we came to the conclusion that Hindley-Millner doesn't really work". I was really baffled. That was at ClojureWest 2016. Here's the talk itself https://www.youtube.com/watch?v=XTl7Jn_kmio. That talk is not about Clojure - Felleisen doesn't even use Clojure. That happened before Clojure.spec was announced. I couldn't really figure out what exactly he was talking about. Not until later, when I tried Spec.

⬐ yawaramin

I understand that Rich Hickey is a fantastic developer, a productivity multiplier, and an industry leader, but I have a few issues with his portrayal of types.

In this talk he has repeatedly cherry-picked the most negative examples from every type system, while ignoring the more positive, simple, and elegant type system features. E.g., he talks about languages which represent product types as `int * int * string * string * ...`, i.e. as nameless clumps of data, while ignoring that those same languages usually support much richer record types which are of course product types with field labels.

Then he talks about Java types being non-composable while ignoring generics and row polymorphism in other languages. He talks about Clojure maps being powerful while ignoring the same maps with the same power in static languages. He talks about type system complexity while ignoring the mental burden of keeping the types in your head in a totally dynamic system like Clojure. Ironically, he even talks about how you can handle five to seven arguments mentally before getting lost, while ignoring that in a dynamically-typed system you're left to your own devices handling much higher cognitive loads.

Another beef I have here is he repeatedly talks about pattern-matching in 500 different places, and having to go and update them, while ignoring the fact that _no one does that,_ we actually do write modular code with data structures as abstract as possible and try to provide powerful, general-purpose functions which manipulate them instead of exposing all the variant cases for raw pattern-matching. PM is great, but it's still important to limit its use for scalability reasons.

I don't know to what extent he has explored the type systems of ML-family languages. But he certainly does not present them accurately here. I would love to see him just do a debate-style talk with someone really well-versed in type theory as well as practical systems, say, Yaron Minsky of Jane Street.

⬐ steinuil

I don't think he has explored Haskell's type system that deeply, since he mentioned Haskell as a complex language along with Java and C++. Haskell has a very simple core.

⬐ platz

I think he's familiar enough; you also don't want to mistake the trees for the forest.

⬐ platz

Rich would probably do very well in a debate-style talk. The art of argumentation is very subtle. If you think there'd be a blowout that entirely discredits the dynamic typing camp—that would rather be like expecting a debate that completely invalidates one side of republican/democrat debate. There are simply core values each side has that are very hard to persuade someone out of. mostly, what people do is "talk to their base/tribe", and then snipe the other side with cheap shots.

⬐ yawaramin

I'm not even looking for a winner-take-all argument. I just want someone to be able to immediately correct Rich when he says something wacky about static typing. E.g.,

'You don't have labelled arguments!'

'You have labelled arguments since Standard ML i.e. at least the '80s, e.g.:

    $ poly
    Poly/ML 5.7 Release
    > fun add { num1, num2 } = num1 + num2;
    val add = fn: {num1: int, num2: int} -> int
    > add { num1 = 1, num2 = 0 };
    val it = 1: int

⬐ mpweiher

> wacky about static typing

No more wacky than the claims people make about dynamic types, e.g.

"You can't have refactoring with dynamic types"

Refactoring was invented on Smalltalk. etc.

⬐ yawaramin

Yeah, but there's 'people' i.e. randos on the internet, and then there's Rich Hickey giving a keynote at a major conference. It deserves a better standard of accuracy.

⬐ christophilus

Yeah. He's aware of that. I think he's generally talking to Java crowds, so the static typing experience of the general programmer is pretty bad.

That said, I liked this talk and agreed with much of it.

⬐ yawaramin

There are options on the JVM outside of Java, from Kotlin to Scala to Whiley to Eta, or even OCamlJava which might be nearing a release. An alternative to dismissing static typing entirely based on Java, is looking at other languages which have less verbose, more elegant type systems on the JVM. RH is ignoring that aspect completely. Understandable, based on his worldview and experience, but not very accurate.

⬐ crispyambulance

Can we please get this man a Wikipedia page now?

https://news.ycombinator.com/item?id=5619680

⬐ sooheon

Wow that man Artem Karibov quoted in the link has some problems.

⬐ kimi

Any transcripts yet?

⬐ grabcocque

I always enjoy Rich's and Cognitect's increasingly futile attempts to bait and switch the Clojure community into using his expensive proprietary database, Datomic.

I love Clojure, it's a beautiful language, but its march has stalled, perhaps even reversed, and I lay the blame of that squarely at the anti-community practices of Cognitect, especially surrounding Datomic.

⬐ keymone

is anybody forcing you to use datomic? no.

is cognitect doing something to clojure that is beneficial exclusively for datomic? no.

has cognitect banned you from making pull requests that improve clojure? no.

has cognitect banned you from forking clojure and fixing whatever you think they've done with some evil intent? no.

so wtf are you on about?

⬐ whalesalad

AFAIK clojure doesn’t accept pull requests.

⬐ keymone

"send a patch" sounds so 90s, but you're right.

⬐ geofft

> has cognitect banned you from forking clojure and fixing whatever you think they've done with some evil intent? no.

I don't understand this objection. The value in a free software project, and especially of a programming language, is primarily in its community and the expectation of a future around the project, not in the existence of some code somewhere. It's extraordinarily hard to build a community around a fork, and if you need to fork to do reasonable things, that's a legitimate criticism of the people running the original community.

(That said, if you only need to fork to do unreasonable things, that's good community management. I don't pay nearly enough attention to Clojure to know what's actually happening - but I have seen both of these cases in other free software communities.)

⬐ DigitalJack

"It's extraordinarily hard to build a community around a fork"

This may sound snarky, but I mean it earnestly: It's extraordinarily hard to build a community. Period.

⬐ keymone

> It's extraordinarily hard to build a community around a fork

it follows quite naturally, that reasoning to fork doesn't have enough community support, which only means one thing - you're wrong about clojure community's opinion on state of things around clojure.

⬐ krisdol

Adding to that, there are a few forks of popular projects that were quite successful: io.js, libreoffice, mariadb.

⬐ evilduck

Blink, Ubuntu, neovim, LibreSSL, Plex, XOrg, etc, etc.

If you're dedicated to it and correct about your choices, forking doesn't seem to hamper anything.

⬐ geofft

I'd believe that if it were ordinarily hard to fork. But it's disproportionately hard to fork because of network effects. It's true that if the community approves of the current direction, a fork won't succeed. But that doesn't mean that if a fork doesn't succeed, it implies the community approves.

⬐ pjmlp

People that give you programming languages for free need to pay their bills in some way.

⬐ cutler

It could be argued they'd make more money from consulting with a much wider adoption of free Datomic. The problem in the Clojure community is much wider - it's not interested in wider adoption, competing for mindshare and rallying round a web framework to attract new users. I think it's partly a Lisp thing.

⬐ Sinidir

You mean that they want to make some money?

⬐ vog

Do you imply that in general, the (perfectly valid) desire to make money would have justified bad behaviour towards a community?

In general, if you have a community around your comany, you can do it properly, or you can do it Oracle style. (pushing MySQL, OpenOffice, OpenSolaris, etc. into forks or death).

So the question is only whether Cognitect is good or bad for the community, not whether they make money or not. Making money is not a good excuse of bad behaviour. And while we are at it, doing voluntary work isn't a good excuse for bad behaviour, either, but that's another topic.

Having said that, I don't see where Cognitect is bad for the community. Rather, they seem to interact very well with the community.

⬐ None

None

⬐ sooheon

> Do you imply that in general, the (perfectly valid) desire to make money would have justified bad behaviour towards a community?

But the OP made insinuations that the act of selling software was de facto negative to the community. It is only necessary to point out that making money is not automatically harmful to the community.

⬐ iLemming

> its march has stalled

Clojure being a Lisp probably exceeded and overachieved its goals when it comes to popularity. Sadly, not a single Lisp probably will ever become a mainstream.

And for the same reasons why Vim and Emacs would never become more popular than IntelliJ and Sublime. People tend to choose something easy to start with, out of the box thing, not something that has a small, simple core yet extremely extensible and rich ecosystem. Most of the innovation in Clojure-space happening outside of Cognitect, driven mostly by individuals.

Look at Clojurescript. It hasn't stalled, au contraire - with every release it's getting better. Pick any large Clojurescript project - PrecursorApp or Circle CI, look under the hood. Compare Re-frame and Fulcro projects with other popular Javascript frameworks. Check out Re-Natal, Lumo and Planck.

Right now Clojurescript probably the most pragmatic choice if you want to build web-apps using a functional language. GHCJS and Purescript (that appeared few years before Clojurescript) - still hasn't quite reached popularity. Elm - which is 5 years old (same as Clojurescript) - still feels a bit experimental. ScalaJS - I don't even want to talk about Scala. Fable/FunScript/WebSharper? Microsoft has failed to convince even its own herd to start using F#. Largely - C# still in use. And those guys prefer Typescript - not really functional. Only ReasonML someday soon may become true competitor to Clojurescript simply because: a) Facebook behind it; b) ain't a Lisp.

⬐ agumonkey

Oh noes, the ML fad got to Rich !

⬐ fergyfresh

Not to be rude, but this guy could have had a rough time growing up DICK HICKEY!!!!

⬐ hellofunk

No one in this thread is yet talking about the content of this talk, which I think is interesting. Rich is doubling down on dynamic typing. As the world is gradually moving to more statically-typed languages (Rust, Swift, Go, Elm, Purescript, Typescript are all popular tools that come to mind), Clojure remains in the embrace of dynamic typing, and this talk is an interesting look at the perspective of why that is, and adds perhaps a little bit to the debate between static and dynamic typing.

I admire Rich a lot, but I also admire John Carmack, and it is apropos that yesterday another comment was shared on another post here [0] about his opinions on static typing, and how in just a few years (his talk was in 2013), the industry as a whole has made considerable changes in its opinions of static typing; according to his talk, in 2013 most of the industry was still not convinced of the benefits of static typing.

[0] https://news.ycombinator.com/item?id=15460604

⬐ ceronman

Static vs Dynamic typing... Why one has to win?

I don't think it's about being convinced of benefits of one or another. We need to be convinced about the drawbacks of each approach. Then we will understand that there is no silver bullet and each kind of language has strengths and weaknesses.

I love static typed languages. I love that they show me silly errors as soon as I type them on my IDE. I love that I can get auto complete and automatic refactoring tools. I love that I can easily jump from one function to another by just clicking on their names names as if they were hyperlinks.

I also love dynamic typing. I love how few lines of code you need to do some complex tasks. I love how easy is to manipulate complex structured data coming in JSON, YAML or XML forms. I love how easy is to do metaprogramming with them, how easy is to create your own DSL and make tasks like DB access a piece of cake. I love how you can just REPL into your server running in production, change things, test them and leave.

I like that we have both kind of languages. I chose static typing for some projects and dynamic for others. I hope no paradigm wins. Let us the best tool for the job.

⬐ he0001

For me static typing is one less thing to think about and I can concentrate on the problem.

⬐ hellofunk

Ironically, those who prefer dynamically typed languages would make the exact same argument.

⬐ throwaway7645

Yea as someone who really learned coding in a dynamic language, static typing drives me bonkers and makes me think about things I'd rather ignore. I know static typing has many advantages, especially when coupled with an IDE, but I don't want that. I only really need a REPL to test out snippets and a text editor to assemble them into a coherent program before feeding it to the compiler. Well, that and Stack Overflow. Some of my ideas might change if we're talking about a large project though as I mostly code for task automation, data analysis, and helper scripts.

⬐ he0001

Well just because static typing isn't present in your language the benefits of the concept is still something one has to consider in a non static language. My experience of people coming from non static typed languages to a non static language really have a problem with the effects of what static typing brings, like encapsulation. While the contrary, when I write in non typed languages I really benefit from the thought patterns static typing creates and helping me to produce better code.

Not having static typing is just freecard of doing what I want instead of relying on explicit (language) structure. With that said you can still produce good code in non static typed languages, it's just damn easier to not to since the language doesn't help you with it.

⬐ Jach

I think it's important to note Carmack also tried Racket after that, even wrote a server that was in production for a while. As I recall from his twitter feed he really likes it especially for beginners (got his son to make games with it) and especially for getting things done as a beginner to the language. Later he discovered Typed Racket and appreciated it, I remember something about with proper types at least at the interface level he found some design flaws to fix.

Just like some people think dynamic types are limited to the subset of static types with Any type for everything and think JavaScript is a great example of its utility and power instead of e.g. Common Lisp, a lot of people think static types are limited to what you get in C++98 instead of e.g. Haskell. The type flame threads would be a lot more interesting if people had tried using more than the most popular representatives of each side before forming opinions...

⬐ ludwigvan

I love Clojure and dynamic typing and actively use it in one of my commercial projects.

However, my observation as an instructor is that static typing fixes a lot of issues in day-to-day coding of average (boring) projects, especially in projects where the average caliber of the programmers is low. It is not that it prevents bugs per se, but makes the creation process much smoother for people via instant feedback that the code won't work/compile at all.

⬐ sanderjd

I believe it also fixes a lot of issues in day to day programming of non-boring projects where the average caliber of programmers is high.

The "me and my team are too smart and our project is too interesting to benefit from static analysis!" meme is silly.

⬐ Hupriene

It's less about being smart and more about having habits that fill in the gaps that a lack of static analysis leaves. This can happen through TDD or through REPL driven development. Each has their own strengths and weaknesses.

⬐ sanderjd

REPLs are super nice, TDD is super useful, but neither of them either preclude or obviate the advantages of static analysis. Although in practice, none of the popular statically typed languages has a standard REPL (except maybe TypeScript?) which is really a shame.

But I agree with you that it has nothing to do with smart vs. less smart.

⬐ throwaway7645

On the other side of the coin, static typing doesn't catch a lot of things that TDD & REPL development can, although nothing prevents you from using tests with static typing.

⬐ sanderjd

No static typing advocates suggest that you shouldn't write unit tests - they just suggest that you can write dramatically fewer tedious but important tests that boil down to verifying type safety.

⬐ erokar

His argument that static typing introduces coupling and reduces modularity is convincing. His argument that Java's Spring framwork is little more than necessary dynamism forcing its way back into a statically typed language was eye opening to me. Certainly an interesting talk.

⬐ _pmf_

> His argument that static typing introduces coupling and reduces modularity is convincing.

It's absolutely bogus. In Java, C#, C and C++, I can pull in dependencies via scanning a plugin directory that searches for JAR resp. DLL files. The only coupling is the interface, which in a properly factored system will be a single module containing only the related set of interfaces.

Claiming this exposes the author of never having worked on any actual medium or large system in any commonly used static language. (Maybe the more academic ecosystems like Haskell or ML require implicitly pulling in dependencies via textual imports, but this is absolutely not the only way to do it.)

⬐ kbutler

Clojure is a reasonably sized system written in Java.

⬐ tim333

By the way he doesn't get to static types till 1:06:06 out of a 1:15 talk - there's a lot of other stuff.

⬐ flavio81

>As the world is gradually moving to more statically-typed languages

Depends on how do you see it; because Python has grown a lot in the last 2 years, and Javascript (pure) is still growing.

And don't discount Julia; i can foresee good future for it.

⬐ jhbadger

Exactly. Static typing isn't some new technology -- languages like Pascal (which was very popular in the 1970s and 1980s) had it. Scripting languages (which tend to be dynamically typed) took off in the 1990s as a reaction to the inflexibility of statically typed languages. If static typing was really the silver bullet its current defenders think it is, why do they think scripting languages took off?

⬐ bbatha

Because the static languages of the time, C++, Java, etc had serious deficiencies that made that style of static typing a hindrance. Type inferences, type class style generics, and gradual typing have been huge wins for the ergonomics of static typing. Additionally IDEs have improved immensely since the 90s and static typing makes the experience much better.

Finally I’d argue that the size of apps has increased a lot. Dynamic typing is great for small apps that you need to get out the door. Static typing shows its advantages in the long tail. The more people you have working on a codebase the more the static typing pays off. Typescript for instance was created specifically to manage large JavaScript code bases.

⬐ flavio81

Really big apps have existed since the early 70s. I'd argue that the average application size gets lower every year due to the increasing number of specialized libs every year.

And C++ has supported generics long before the boom in popularity for dynamic scripting languages...

⬐ s_kilk

I think it's interesting that Rich seems to miss out on Erlang (and Elixir) entirely, even when he gets to talking about Situated Programs and Runtime Tangibility, not even to speak of Concurrency.

⬐ erokar

Elixir seems very similar to Clojure to me (the former being influenced by Clojure).

⬐ s_kilk

Absolutely, (I moved over to Elixir from Clojure) and hence it seems like a strange blind-spot.

When Rich talked about Systems programming and Runtimes, I expected Erlang/Elixir to come up as obvious examples along with Smalltalk and Common Lisp.

⬐ christophilus

I'd love to read a writeup on your experience doing this.

I love Clojure, and really don't like the Elixir syntax (though I used to dislike the Clojure syntax, so...) Have you found Elixir's oddities (like calling lambdas in a different way than you call functions lambda.(:bar) vs func(:bar)) to be annoying?

⬐ erokar

Chiming in here, having only dabbled with Elixir for a couple of months. My gripes:

- Calling lambdas with a dot is a real wart. I find it very easy to forget and it looks ugly.

- I also don't like the Rubyesque end keyword you have to provide for functions. Get's verbose. Whish Elixir was white space sensitive like Python, Haskell, Elm et al.

- Pattern matching in functions is an idiom that is overused. It easliy gets verbose since you have to type out the full function signature.

- A proliferation of data structures dictated by performance issues (lists vs. tupels, maps vs. keyword lists). Ugly map litteral: %{}. (Personal issue I guess, I tend to find $%& etc. ugly in a programming language. It's swearing at me!)

Apart from these gripes I like Elixir a lot. A clean and pretty simple language. Protocols are used in a nice way to acheive polymorphism (similar to Clojure, I think). Good as a first FP language. Gives you access to an interesting runtime.

⬐ hlship

I do 100% Clojure work, but am learning Elixir (and may someday use that at work as well). In any case, there's a lot to like on both sides, but there's also a dichotomy: communicate by sharing state (Clojure - atoms, channels) vs. share state by communicating (Erlang/Elixir - everything is a one-way message).

To be honest, what I really like about Elixir so far is the developer affordances: excellent REPL, super-fast startup time, and building on 30-ish years of OTP.

⬐ sova

OTP in this case means "on time performance?" I love Clojure and your comments about Elixir intrigue me.

⬐ hellofunk

No. See here: http://learnyousomeerlang.com/what-is-otp

⬐ amelius

Okay, but at least with static typing one can have dynamic typing simply by using an all-encompassing type.

With only dynamic typing, one cannot have static typing.

⬐ hellofunk

I invite you to watch this talk.

Dynamic typing is about a lot more than having an "Any" type. It's about widespread lack of extra code and cognitive overhead associated with the imposed structure of adding static types to a project, as well as the verbosity that static typing typically adds (including in type-inferred languages).

You can't just turn off the static typing in a static-typed language; you can turn off the static type for a specific type, yes, but that doesn't give you the experience that dynamic languages offer.

⬐ lucozade

> imposed structure of adding static types to a project

I'm sure you didn't mean it this way but, for me, I think I reach for my pet dynamic or static typed language depending on whether or not I'm expecting to think about the structure of the program up front. Typing is not something I've thought of as an addition.

If I'm doing something small or I'm expecting to iterate around some ideas/play with some data, then a dynamic language lets me try things out sooner.

If I'm expecting to spend a fair amount of time on the project or I need to think hard about how I'm going to achieve stuff then thinking about the types up front is valuable. I'm likely to go for a statically typed language.

I also have a bias towards statically typed if the project is going to get large. This is because I've tended to have more serious problems with large dynamically typed codebases. But I appreciate this is a bias; there are many factors that affect maintainability.

⬐ agumonkey

I'd add one thing. The way we structure "logic" in subpart of systems is often one or nothing, while quite often we could check parameters by necessity.

F a b c d can be valid if called in F a b c d e f g, because F gets what it needs (a b c d), the rest can be ignored. Like an implicit subclass relationship. A subclass B of A can have more features, but as long as it is an A, something depending on an A can enjoy it.

Of course this can cause issues (stack use in function application with unnecessary information, albeit a non naive interpreter/compiler could prune this).

In the end we spend a lot of time circling things around for safety, when we could have something more relaxed and thus more resilient.

⬐ amelius

Yes. So how about a workflow where you write dynamically typed code, and then use a tool to convert that code (with user input) to statically typed code.

You could do this incrementally. So once you have "frozen" your code into statically typed form, you could add dynamically typed code to it, and then "freeze" that. Or you could "thaw" the whole codebase, and add dynamically typed code, etc.

> You can't just turn off the static typing in a static-typed language

I suppose one could design a language where that is possible (although you wouldn't use the full power of static typing in that case).

⬐ _dps

> Yes. So how about a workflow where you write dynamically typed code, and then use a tool to convert that code (with user input) to statically typed code.

This describes how I do a lot of my work. I prototype quickly in Python or Lua, and I have some in-house tools that let me migrate highly constrained subsets of those languages into C, with automated Quickcheck-style checking to make sure the translated code generates the same log messages as the source code.

For me the experience of starting out with something extremely flexible, with a repl, easily mocked components, and tons of "kitchen sink" functionality makes me feel free to experiment quickly. Once I have something that looks like it's working I iteratively remove reliance on dynamic features or built-in libraries.

At the end I have a Python function that looks a lot like a C function, but I got to that function a lot faster and more comfortable than I would have had I started in a pure C workflow. And from there, it's a quick build step to convert that function to a C function.

I haven't found a good common name for this pattern, but I think of it as "plastics and metals", analogous to industrial design. Even though you know a component will end up being made of steel, lots of the design questions are more easily and cheaply solved by making a similar thing in plastic. The plastic of course won't stand up to "production" load, but it usually doesn't have to do so.

⬐ zimablue

I think the thing is that you write code in dynamic languages that you would never think to write in typed languages because it's just too against the culture/grain. Two common situations in my job of throwing data around: Dataframes and pivoting- here the type of a thing (its columns) is never going to be easily statically provable because operations modify it dynamically. Reactive/dataflow style programming- look at the python MDF library, you annotate functions and it dynamically infers a dataflow graph from your code. You can do dataflow in typed languages but nowhere near as elegantly normally.

These two things are core to what I do and drive a train through a typesystem.

I think that the best possible solution would be to have some way to embed proofs and individual typesystems in different parts of your program, like a dynamic language with plugged proofs and types not a single typesystem language that you sometimes go around or switch off.

⬐ sooheon

How much experience do you have with clojure.spec? I see it as a way to add dependent-type-like functionality to parts of Clojure, so my impression is that it addresses your final need very well. But if you know more about it than me and disagree, I'd like to know.

⬐ stewbrew

(I haven't watched the talk because nothing you said makes me expect hearing something new.)

WRT verbosity and cognitive overload: I honestly challenge this statement. In a modern statically typed language, you have to add a few type annotations here and there but in exchange you can skip most runtime checks (and sometimes nullity checks). You don't have to remember the type of a variable because the compiler can tell you.

I admit dynamically typed languages are nice for interactive exploration in the REPL but that's about it.

Too bad typed clojure didn't become the mainstream closure.

⬐ hellofunk

The cognitive overhead is really not about type annotations. It's about program structure. If you watch the video, I think there are some good points made.

⬐ gw

> I admit dynamically typed languages are nice for interactive exploration in the REPL but that's about it.

For Clojure programmers, "interactive exploration in the REPL" is the primary way we write programs. So, if one admits that dynamic typing is ideal for this, then making Clojure dynamic was the right choice.

⬐ stewbrew

No, because you usually have to maintain programs, add features later on, rewrite/refactor parts of the program. Do you write only one-off scripts?

⬐ sbov

As someone who recently moved to Clojure, REPL based programming is a completely different experience. It isn't like using python's REPL.

⬐ gw

I must emphasize that any time we "maintain programs, add features later on, rewrite/refactor parts of the program", we are doing so with a REPL. We don't just use the REPL for one-off scripts.

⬐ kenshi

I think its a mistake to directly compare the experiences (and conclusions) of these two exceptional programmers.

Rich Hickey explains at the start of his talk where he is coming from, what the context of his development work, and he calls it "Situated Software". His background and the context for his development is building information systems in organisations where the rules are messy and no doubt change often. One of the things he calls out explicitly in his talk is the messiness of dealing with the "Two for Tuesday" rule in one of the systems he worked on. He also spoke about how different working in that domain was, to doing compiler development.

John Carmack's experience and domain expertise is building high performance game engines. I'd argue this is closer to compiler writing in the sense that it is a much more rigid problem than say having to serve the needs of an organisation whose requirements may change often and in seemingly arbitrary ways, and usually on a tight deadline.

I am not trying to elevate one domain over the other. They are different kinds of problems, and have different kinds of limitations and constraints applied to them. As such I don't think its any surprise that the priorities for each of these developers is different.

Choosing between more dynamic or more static approaches without considering the context in which you are working in, is a mistake.

⬐ munificent

    > I'd argue this is closer to compiler writing in the
    > sense that it is a much more rigid problem than say 
    > having to serve the needs of an organisation whose 
    > requirements may change often and in seemingly
    > arbitrary ways, and usually on a tight deadline.

Ex-game developer. If only this were so.

When making a new game, the requirements are constantly in flux and the driving force changing them is incredibly elusive: "fun". This depends a lot on the experience level of the designers and how original the game is trying to be, but it's very common for features and systems to completely change as designers try things out and discover what does and doesn't feel good.

And, of course, game development is always under very heavy time pressure because the margins are narrow.

Carmack's situation is a little different because he's been making similar games for much of his career and much of the stuff he's building (rendering and low level infrastructure) are relatively decoupled from the gameplay and game features that are more volatile.

Game developers are under a really tight crunch. They need to write software that's nimble and flexible so they can iterate on the game design quickly and figure out what's fun. At the same time, the code needs to be very efficient, even during early phases of development (it's hard to tell if a gameplay idea is fun or not when the game is running at 3 FPS) and certainly by the time it ships.

⬐ spiralganglion

Current game developer.

This is why you tend to use a low level / static / concrete language to write the (relatively constant, well-defined) game engine — the graphics pipeline, model loading, animation playback, sound, etc — and you use a high level scripting language like LUA to write the (constantly changing, unique) game logic.

To tie this back to the OP, this is one of the reasons I'm fond of Clojure (even though I don't get much of an opportunity to use it). It's written to be concise and expressive (as described in Rich's talk, when compared to C++) but also relatively high perf (by leveraging the native capabilities of the host on which it sits). It has the potential to be a great language for both the high-perf core of a game and the high-level logic.

Sadly, it doesn't yet target a host without a GC (and probably never will), so you're going to have a devil of a time using it in a way that guarantees pause-free execution.

⬐ stcredzero

Choosing between more dynamic or more static approaches without considering the context in which you are working in, is a mistake.

It's amazing how well the parallels and metaphors work between the study of weapons and historical martial arts and programming. (Perhaps not so amazing, since both have a certain nerdy component.) As Matt Easton of scholagladiatoria says, "Context, context, context!" It's erroneous to say that sword X is the best/ultimate sword! It all depends on context. Are you up against armored opponents? What kind of armor? Is this on a battlefield? Is it a duel? Is it on horseback? Do you need to wear one at court?

Design is all about context and response to the forces in that context.

⬐ mightybyte

This is an outstanding point. When watching this talk, I got the distinct impression that Rich has simply worked on a very different class of problems than the ones I have worked on. As we all are wont to do, we extrapolate our experience to the whole universe. But that's not always the right thing to do.

I would say that for the most part, a statically typed language can do the same things a dynamic language can do, but not the other way around. Another commenter mentions the Any type, but I don't think that's it. It's maps. Rich wants to be able to combine maps with a union operation, take subsets of keys, etc. And that is exactly what maps afford. I'd rather work in a language like Haskell where I can use strong types when I need them and drop down to untyped maps when I need them than a language like Clojure where all I ever have is the maps.

Another thing static types do for you is that they make things in your application more discoverable. Instead of tracing through to figure out which data is available and operations something supports, the compiler tells you. I'd rather work in a language where my compiler can help me this way than in one where it can't.

⬐ DigitalJack

I mainly program in clojure. The experience is just too good to be swayed by the bad points.

That said, I wrote a moderately complex program recently where having dynamic types was a hindrance. I had to do mental book keeping in order to make sure I was constructing correct structures. I frequently didn't and the bugs were hard to track down.

However, Clojure Spec came along, and I applied it to the problem. The difficulties became surmountable.

It's a case where if I had been fluent in haskell, I probably would have used it. But I'm not, and this project was for work, so I didn't have the luxury of time to figure it out.

Spec alleviates mental book keeping of structures by adding checkers and clarifying/codifying intent.

⬐ hellofunk

How much Spec do you sprinkle in your code? I'm often using a keyword to query a map. Should I always spec/assert the return of this simple and common task? Because if you type the wrong keyword, a typo or just because you confuse different keywords in your mind, then it doesn't matter how much spec you have applied to the top-level data structure, the function arguments or other things, you still end up with a very easy nil or just incorrect value floating through your data transformations.

The only way I see to have a lot of confidence is to add Spec to every new binding that is created, confirming that all named values are exactly what you think they are. Perhaps that is how most Spec users are working? That would mean asserting every item in a "let" for example.

⬐ GlennS

I would suggest that, rather using spec everywhere, you can use it as a barrier to contagion. That is, only checking that data conforms to your spec when it goes through the major interfaces between parts of your program.

In this approach, you're not relying on spec to verifying the correctness of everything. Rather you use to reduce surprise bugs where a little change here makes something weird and unexpected happen all the way over there.

⬐ DigitalJack

Agree, this is my approach.

Also, I do checks on the data structure, less on how they are used.

So if a function expects a certain structure I’ll spec that. But I don’t do checks on getter setter type operations.

⬐ GlennS

I'd add that I think the correctness problems with (good) dynamic language really only start appear as the program gets larger.

Dividing your program up into large chunks with rigorously defined and checked interfaces (e.g. using spec) does a lot to mitigate this. Meanwhile, you still get the benefits of dynamism (concise code tightly focused on the problem domain) within each module.

⬐ kaeluka

My take: "a statically typed language can GUARANTEE the same things a dynamic language can GUARANTEE, but not the other way around. A dynamically typed language can PERMIT the same things a static language can PERMIT, but not the other way round."

⬐ AriaMinaei

Sorry to go off-topic, but this was a very nice sentence: "We extrapolate our experience to the whole universe."

⬐ sova

Nicely sieved out! I too found this sentiment very essential.

⬐ stuartaxelowen

Dynamism is certainly available to statically typed languages, but I believe the problem comes at how difficult it is to write dynamic code in static-typed languages, and to use the common building blocks that help you solve real problems.

Writing dynamic code in static languages is like running through mud, just like writing static checking in dynamic languages is.

I think both paradigms are important and valuable for different applications, but I have never seen a language that does both in any truly useful way.

⬐ erokar

I think the gradual typing TypeScript provides is pretty useful, with relaitvely low friction.

⬐ weavejester

"a statically typed language can do the same things a dynamic language can do, but not the other way around"

It's interesting you'd say that, as I'd consider the opposite to be true. There are functions that are trivial to write in a dynamically typed language, but are hard to statically type. For instance, the `assoc-in` function in Clojure.

"Another thing static types do for you is that they make things in your application more discoverable. Instead of tracing through to figure out which data is available and operations something supports, the compiler tells you."

You don't necessarily need static typing to give functions some form of type signature or specification.

⬐ flavio81

I also find the opposite to be true, as you do.

The human mind and human problems, operate more like a dynamic language. To put it in a simple, silly example: When i tell you to "cut something with a scissor", the scissor doesn't specifically cut paper; there are many things that can get cut by a scissor. Or when you take a pot, put it over the stove and boil water, you do it and the pot is not specifically designed to boil water, the pot can heat anything; so you can say that many operations (verbs) in real life do not get performed in a "static typing" way, but in a "dynamic typing" or at least "duck typing" way.

⬐ bbatha

Both of those examples are trivially modeled with typeclasses (java style interfaces will also cut the mustard here too).

Your paper example I would have a “Cutter” and “Cuttable” type classes to describe things that cut and can be cut. In languages like Haskell this only involves a couple extra lines of ceremony over writing a “cut” method on each type.

The pot example is even cleaner with types and show the power of them. I’d have a Pot typeclass that can heat things. I’d have a liquid class for all liquids that water implements and finally a boil free function that takes a pot and a liquid. Exactly as expressive but now with compile time checking.

⬐ dragandj

All this is covered in Rich's talk. Your Cutter and Cuttable are different from my Knife and Cuttablito. When your program needs to talk to outside world, it all becomes messy.

⬐ spion

Here is assocIn variant 1 type in TypeScript:

  declare function assocIn<T, K extends keyof T>(t: T[], x: [number, K], val: T[K]):T[]

You can try it out: https://goo.gl/3JrmX3

⬐ weavejester

It generates a type error when I write:

    assocIn({ a: 1, b: 2 }, ['b'], 4)

⬐ mightybyte

> There are functions that are trivial to write in a dynamically typed language, but are hard to statically type. For instance, the `assoc-in` function in Clojure.

This is not a hard function to write in Haskell. You make it operate on a list of maps, which is what the Clojure one is doing too.

> You don't necessarily need static typing to give functions some form of type signature or specification.

But you do have to have static typing to make sure those signatures are automatically verified to be coherent.

It seems like you're thinking about my statement from a turing-complete computable point of view. But I meant it from the point of view of what the compiler can do for you. It's not what things you can compute, but what things will be automatically checked.

⬐ weavejester

"This is not a hard function to write in Haskell. You make it operate on a list of maps, which is what the Clojure one is doing too."

It's harder than you think. Just try it. I believe it's possible to achieve in Haskell, but it requires some rather esoteric type system extensions.

"But you do have to have static typing to make sure those signatures are automatically verified to be coherent."

If you want a guarantee, then yes, but you can get an good approximation of correctness with runtime specs coupled with test generation.

⬐ mightybyte

> it requires some rather esoteric type system extensions

No, it's just a simple [Map Text Text]. Or if you want a little more safety, then [Map Text Value].

> you can get an good approximation of correctness with runtime specs coupled with test generation.

But this requires more code. More code is error-prone and takes time to write and maintain.

⬐ weavejester

"No, it's just a simple [Map Text Text]. Or if you want a little more safety, then [Map Text Value]."

I think you're confusing `assoc-in` with `assoc`. The latter is trivial to type; the former is not.

The problem with typing `assoc-in`, is that the type of the vector of keys is derived from the type of the nested map in a way that's hard to generalize.

For example:

    (assoc-in m [a] b)

Has (approximately) the type signature:

    Map a b -> a -> b -> Map a b

But:

   (assoc-in m [a b] c)

Has the type signature:

   Map a (Map b c) -> (a, b) -> c -> Map a (Map b c)

And so forth.

"But this requires more code. More code is error-prone and takes time to write and maintain."

No it doesn't; you can generate and run the tests from the function's spec automatically. If I run `(clojure.spec.test.alpha/check)` in Clojure, it will generate and run tests for all functions that have specs.

⬐ tel

You can use lenses, it's something like

    over (at "foo" . at "bar") :: (a -> b) -> (Map String (Map String a) -> Map String (Map String b))

Neatly, this system generalizes to typed realizations immediately, too.

    over (foo . bar)

⬐ weavejester

Sure, lenses are a solution. My point isn't that an alternative solution to this is impossible (or even difficult) in a statically typed language, but that static typing makes some solutions infeasible. There are some very trivial functions that Clojure has that are nevertheless impractical to write a robust type signature for.

⬐ tel

There's also the way where you encode `Map String Value` as a member of `Value`. This is closer to what Clojure does and does allow assoc-in to be written directly. I was just playing along with the other comments' encoding of maps.

    data Value = 
      ...
      | VMap (Map String Value)
      | VNull
      ...

    assocIn :: [String] -> (Value -> Value) -> (Value -> Value)
    assocIn (name:names) f (VMap m) = _

In general, the more type information you leave around the more you have to do to ensure that it remains meaningful. Erase enough of it and you can arrive back at Clojure.

⬐ weavejester

You're not solving the same problem. You're fixing the keys of the nested maps to a single type, but the difficulty is when the keys are all different types.

In other words, you want to type a general form of:

    Map a (Map b (Map c ... z)) -> (a, b, c...) -> z -> Map a (Map b (Map c ... z))

And even that is more limited that Clojure's `assoc-in`, as the type signature still assumes each nested map is homogenous.

⬐ tel

When they keys are all different types then you've got

    data Value =
      ...
      | VMap (Map Value Value)
      ...

    assocIn :: [Value] -> (Value -> Value) -> (Value -> Value)

I was again assuming types more informative than Clojure. Also again, lenses actually solve the higher information type situation that you're pointing at in your comment.

⬐ weavejester

Yes, you can write the function using entirely dynamic types:

    Map Dynamic Dynamic -> [Dynamic] -> Dynamic -> Map Dynamic Dynamic

But would you actually write a function like that? Of course not. Not only is it a pain to unwrap nested maps of dynamic types, but you lose almost all the advantages of a static type system.

So what's the alternative? As you point out, a more idiomatic way is to use lenses, as they solve a similar problem but can be typed.

However, in Clojure we can use assoc-in or lenses. Dynamically typed languages are not constrained by what is feasible to statically type. In Haskell, it is hard to the point of being infeasible to robustly type assoc-in, so it's not idiomatic to use it. But in Clojure we're not restricted by the limitations of a static type system; we can use functions like assoc-in.

This is the trade off of static typing. A static type system limits what solutions are feasible and idiomatic; even functions that are simple to express and reason about, can be extremely hard to accurately type.

⬐ tel

I wouldn’t because once I’ve got a type system the tradeoff never works out. But the same thing can be expressed.

I don’t see the thrust of your argument. You can write both styles of program in both languages, but you can only have type guarantees in a language with a type system.

So, you’ve shown that assoc-in can be written so long as type guarantees are thrown away. That seems like a reasonable reason to pick lenses either way.

⬐ weavejester

"You can write both styles of program in both languages"

No you can't, not in any way that's practical.

It's possible to write Haskell by wrapping every value in a Dynamic type, but possible does not mean feasible.

If you don't believe there's a distinction, then try writing a functionally equivalent `assoc-in` in Haskell. I don't mean sketch out a design in pseudo-code; I mean produce a fully working function that operates seamlessly with all possible types of Data.Map.

"So, you’ve shown that assoc-in can be written so long as type guarantees are thrown away. That seems like a reasonable reason to pick lenses either way."

Well, that's the question: is it?.

Lenses can be typed, but `assoc-in` is conceptually simpler. Haskell advocates using only solutions that can be typed; Clojure advocates using the simplest solution.

Static typing is a constraint, because it limits the solutions that are feasible and idiomatic. In return it provides a compile-time guarantee. The question is whether the guarantee is worth the solutions that are (for all practical purposes) removed.

Immutability is another type of constraint. As is automatic memory management, variable scoping... programming languages are designed around constraints. Pretending that you can introduce a constraint and not narrow the solution space is wishful thinking.

⬐ tel

It’s feasible and easy to write assoc-in atop of Dynamic... it’s just not done because nobody programming in Haskell thinks it’s valuable. We could choose to support much more opt-out dynamic behavior no sweat, but nobody wants to.

The idea of assoc-in working for every type of Map also doesn’t make sense. Map only exists statically. Types as Clojure discusses them exist at runtime, are statically Dynamic, and it’s trivial to write assoc-in that works over all of them in any language.

Your argument hinges on the idea that it’s impractical to write using Dynamic in Haskell. I do not think this is true. There is perhaps some value in greater library support, but it is easy to do this. It is simply not considered valuable.

⬐ weavejester

"It’s feasible and easy to write assoc-in atop of Dynamic..."

Then why don't you do it and post up the code? If it's as easy as writing the same function in a dynamically typed language it shouldn't take more than a minute or two.

"The idea of assoc-in working for every type of Map also doesn’t make sense. Map only exists statically."

Maybe you're misunderstanding me? I mean it should work for every `Data.Map k a`, where `k => Ord`, just like all the other Data.Map functions.

"Your argument hinges on the idea that it’s impractical to write using Dynamic in Haskell. I do not think this is true."

Then try it?

⬐ tel

> I mean it should work for every `Data.Map k a`, where `k => Ord`, just like all the other Data.Map functions.

This is where we're got a difference of opinion. My statement is that assoc-in doesn't do anything close to working over "every `Data.Map k a`". Data.Map is a compile-time idea classifying names in the language and the Data.Map functions are designed to respect that both dynamically (manipulate some memory properly) and statically (create updated contexts with new and different information).

On the other hand, assoc-in works only dynamically/at runtime over all kinds of (Clojure and Java) data. In particular, it has meaningful semantics for every possible input value although it only has "interesting" semantics for certain runtime shapes.

That's easy to write in Haskell.

https://gist.github.com/tel/bb84fcc3f1b7488b53349156284b509a

⬐ weavejester

Your code doesn't compile. I can fix the obvious typos and missing imports and language pragma, but the type errors that remain are beyond my knowledge to correct.

⬐ tel

Apologies, quick sketch in an airport between flights. May fix this later when I’ve actually got an internet connection.

⬐ tel

I fixed the type errors in what I presented before. Little changed: I had to fix the typos, bad imports, and missing pragmas, and then I had to instantiate Val as Hashable and Eq, but since all things inside of Val are Hashable and Eq it was straightforward.

https://gist.github.com/tel/bb84fcc3f1b7488b53349156284b509a

⬐ weavejester

Can you give an example of use? It's not clear to me how this interacts with standard Haskell data structures. If I have an existing map, do I need to walk the tree and encase everything in a `Val`, and vice versa when I want to pull values out?

⬐ tel

Honestly, at this point if you’d like to keep chatting, would you mind emailing me? I’d like to discuss these ideas further if it’s of interest.

⬐ weavejester

We might be at the point where we should to agree to disagree. It's interesting to debate a point, particularly since it forces me to justify my viewpoint to myself as well as to you, but I think the discussion may have run its course. :)

⬐ tel

It doesn’t really. I’m not claiming this works with standard Haskell practice. Typed techniques don’t translate directly to untyped things.

The act of traversing each time is “forgetting”/“synthesizing” type information and is exactly the issue. Dynamic code destroys type information.

⬐ weavejester

"It doesn’t really. I’m not claiming this works with standard Haskell practice."

Then I don't understand where you disagree with me. I'm not saying assoc-in is impossible; just that it's impractical.

We can write assoc-in with dynamic types, but that doesn't integrate with existing functions, and is generally a pain to work with. Alternatively we can try and to statically type it, but then we run into the limitations of Haskell's type system.

assoc-in is a solution that doesn't really work in Haskell, because the constraints of the language prevent it. Adding static typing to a language adds practical limitations in exchange for compiler guarantees.

⬐ tel

I disagree it’s impractical—it’s just not valuable. Assoc-In reads as convenient to you, and it is if you’ve already thrown away type information, and reads as destructive to me, a force for destroying information.

Lenses aren’t better because they’re inconvenient but happen to work with types. They’re better because they properly relate to the typing context and persist important information. Lenses in dynamically typed languages don’t exploit this fact and usually are implemented in a way that’s “easy” but destructive.

What you see as practical limitations are seen by me as practical advantages. What you see as convenience and power I see as loss and destruction.

To my eyes, the reason is that you’re most interested in abilities that help you fluently write programs/create values. I’m interested in balancing the power to create with the power to analyze. I want my programs to have meaning both as runtime interpretations and as logical statements. I can choose other type systems to satisfy this balance, but if you just side with “power to create” then I think you lose big.

⬐ weavejester

"What you see as practical limitations are seen by me as practical advantages. What you see as convenience and power I see as loss and destruction."

That's not quite how I see things: I see static typing as a constraint that trades solutions for guarantees.

"To my eyes, the reason is that you’re most interested in abilities that help you fluently write programs/create values. I’m interested in balancing the power to create with the power to analyze."

I don't believe that power and convenience should be pursued above all other considerations; if I did, do you think I'd use a language where data structures are immutable by default? Immutability is a significant constraint, yet clearly I think it worth the cost.

Constraints like static typing and immutability are useful only if the guarantees they deliver are worth more than the solutions they remove. You say that assoc-in is "not valuable", because it cannot easily be written without discarding type information; it's a net loss, because type information is more valuable to you.

Where we ultimately disagree, I think, is the value we place upon static types. I don't think they're particularly valuable at all. Their cost hugely exceeds their usefulness, which to my mind is minor at best, and non-existant at worst.

But my problem is not that you disagree with me over how valuable static types are, but that you seem to think they're not a trade-off. Forgive me if I've misunderstood you, but you don't seem to acknowledge that static types have any disadvantages associated with them at all.

⬐ tel

> Their cost hugely exceeds their usefulness, which to my mind is minor at best, and non-existant at worst.

This value difference is indeed core. I value them exactly as "information that can be discerned from reading a program without having to try to execute it in your brain". There is a cost to retaining this information in that there exist programs which have operational meaning but lack this structure. At the same time, I think that this information is incredibly valuable.

So, yes, if you don't see the benefit then there's little more to say.

⬐ mightybyte

> The latter is trivial to type; the former is not.

That's why I'm not trying to make it typed. Instead of

    Map a (Map b c)

I'm dropping down to something like

    Map Int (Map Text a)
    Map Int (Map Text Text)
    [Map Text Text]
    Vector (Map Text Text)

...which is more like what's really going on in Clojure.

Come to think about it, assoc-in is a great example of why I want static types. It's very hard to figure out what that function does from reading the documentation. There's no type signature to help me, the description doesn't give me much more to go on. Reading the examples leaves me wondering whether I actually understand what it's doing or if I'm missing some corner case.

> it will generate and run tests for all functions that have specs

But you have to write the specs. And that's the code I'm talking about. Fundamentally the generator can't just know what behavior you want and what you don't want. You have to tell it.

⬐ weavejester

"That's why I'm not trying to make it typed."

Are you proposing that a distinct `assoc-in` function should be written for every combination of types you'd need in your program?

Doesn't that rather prove my point that some functions that are trivial to write in a dynamically typed language are hard to write in a statically typed language?

"Come to think about it, assoc-in is a great example of why I want static types. It's very hard to figure out what that function does from reading the documentation."

I've always thought the function was trivial. I mean, it's just four lines of code:

    (defn assoc-in [m [k & ks] v]
      (if ks
        (assoc m k (assoc-in (get m k) ks v))
        (assoc m k v)))

I have a much harder time relating back a complex type signature in Haskell back to what the function actually does, but perhaps that's something that just requires more practice.

"But you have to write the specs."

Sure, but you have to write the type signatures, too. Type inference certainly cuts down on a lot of work, but it doesn't entirely eliminate the need for explicit typing.

I also tend to prefer adding explicit type signatures, even for functions that could be inferred. It makes type errors easier to catch.

⬐ mightybyte

> Are you proposing that a distinct `assoc-in` function should be written for every combination of types you'd need in your program?

No. I'm proposing something like what tel described elsewhere in the thread. If that's not dynamic enough for you, then you can go with something like JSON's Value type. In both cases lenses give you very convenient and composable access and manipulation.

> I've always thought the function was trivial. I mean, it's just four lines of code:

But you have to read the code. The type signature / code boundary is very useful for allowing you to chunk things and abstract over implementation details. This particular case may not be much code to read, but that is often not true in the general case.

> Sure, but you have to write the type signatures, too. Type inference certainly cuts down on a lot of work, but it doesn't entirely eliminate the need for explicit typing.

> I also tend to prefer adding explicit type signatures, even for functions that could be inferred. It makes type errors easier to catch.

That's exactly my point. Type signatures can be inferred, specs cannot. Choosing to add them is irrelevant. If you want the add them that can be done automatically.

⬐ weavejester

"No. I'm proposing something like what tel described elsewhere in the thread."

tel's solutions are "use lenses" or "assume all keys are strings", both of which miss the point.

Yes, you can work around the limitations of a static type system, either by finding another solution (lenses) or by making functions more specific (assume all keys are strings), but that doesn't mean the limitations disappear. It just means you're working around them.

In a statically typed language the solutions to a problem are constrained by the type system. The question is not whether statically typed languages are more constrained than dynamically typed languages, but whether the constraints that static typing introduces are offset by the guarantees they purchase.

"That's exactly my point. Type signatures can be inferred, specs cannot."

The compiler can infer some type signatures, and if you're not explicitly typing your named functions (which is generally cited as good practice), that does mean you get some checking "for free".

But specs can also test more things than types; they occupy a space somewhere inbetween static types and a generative testing solution like QuickCheck. Depending on the function, a spec might be more or less verbose than the equivalent checks in a language like Haskell.

⬐ mightybyte

> Yes, you can work around the limitations of a static type system, either by finding another solution (lenses) or by making functions more specific (assume all keys are strings), but that doesn't mean the limitations disappear. It just means you're working around them.

I'm less concerned about the theoretical limitations of types that you seem to be talking about and more interested in looking at concrete real-world applications where you think you need dynamic types and seeing exactly what we can do to address them with a type system. I know there exist things that are difficult or maybe impossible to define types for (the Y combinator for instance, or possibly this assoc-in). My argument is that you don't need this power. I don't care whether difficult to type things exist, I care about whether real-world problems require them. I have yet to encounter one that I felt requires the same amount of dynamic typing power that you seem to be arguing for.

See this thread for some concrete examples:

https://www.reddit.com/r/haskell/comments/792nl4/clojure_vs_...

⬐ gmfawcett

Well it's strictly true, in the trivial sense; although the ergonomics might be unpleasant. Many static languages provide a Dynamic type, into which you can stuff any value. The type supports a mechanism for querying the inner type of the Dynamic value, and extracting it. So yes, you can implement dynamically-typed data structures and logic in a statically-typed language. But to use this approach in a widespread way, it might take a lot of scaffolding to make the experience pleasant. As Greenspun warned us, you might just end up implementing a crappy Lisp. :)

https://en.wikipedia.org/wiki/Greenspun's_tenth_rule