HN Theater @HNTheaterMonth

EDN has some really good ideas in it. Here's the main spec: https://github.com/edn-format/edn

The Learn X in Y Minutes: https://learnxinyminutes.com/docs/edn/

A related talk by Rich Hickey that I think you'd find interesting: https://www.youtube.com/watch?v=ROor6_NGIWU

For a schema, I'd start with what CUE has done. The idea of types that constrain down as a lattice + a separate default path really resonates with me. https://cuelang.org/

Simple Made Easy (https://www.youtube.com/watch?v=LKtk3HCgTa8)

The Language of the System (https://www.youtube.com/watch?v=ROor6_NGIWU)

The Value of Values (https://www.youtube.com/watch?v=-6BsiVyC1kM)

All by Rich Hickey (Clojure creator) - strongest programming speaker I've found to date.

>Languages are solved, not important; I build systems.

The language of the System - Rich Hickey https://www.youtube.com/watch?v=ROor6_NGIWU

I would say Clojure more than substantial returns of invested time. Not just the language itself, but also the wider approach to software composition, feature accretion vs backward compatibility, schema'd dynamic typing seem benefictial no matter what your development tools you use. Maybe check out the talks below and see if they don't enRich you as software engineer.

https://www.youtube.com/watch?v=ROor6_NGIWU The Language of the System

https://www.youtube.com/watch?v=MCZ3YgeEUPg Design, Composition, and Performance

https://www.youtube.com/watch?v=oyLBGkS5ICk Spec-ulation

https://www.youtube.com/watch?v=YR5WdGrpoug Maybe Not

> Clojure with the same exact constructs represented with more C-like syntax would, at the level the presentation speaks to, allow the same level of simplicity.

I don't think this is true. I think it would be easier for "most people," but definitely not simpler. Easy meaning close at hand, simple meaning one strand, one braid, independent, less context necessary. Clojure syntax is the AST of the program, right there in front of you, in literal notation. There are fewer special cases, fewer moving parts interacting. C syntax requires spinning up a virtual machine in your mind and executing multiple statements. C is easier because we've already spent the time and effort to familiarize ourselves with it, but it has more complexity. Compare a 954 line ANTLR grammar for C [1] with a 261 line Clojure grammar [2].

> Cognitive overhead is lower working with a language that at least "looks like", what you're used to, and more developers know C-like languages, thus a language like Kotlin is "easier to approach" but necessarily "easier" in the way the presentation talks about.

I would agree, using Rich's definitions of simple and easy, that Kotlin is easier for the majority of developers than Clojure. This follows immediately from the definition of easy.

> This is a misapplication of the presentation really. It speaks to a level above selecting a language and is really about the design of systems.

I would recommend Rich Hickey's talk "The Language of the System" [3]. The programming language(s) used are part(s) of the system and have an effect on its design. I don't think this is a misapplication of the "Simple made Easy" presentation, I think it hits the nail on the head.

[1] https://github.com/antlr/grammars-v4/blob/master/c/C.g4 [2] https://github.com/antlr/grammars-v4/blob/master/clojure/Clo... [3] https://www.youtube.com/watch?v=ROor6_NGIWU

⬐ BoorishBears

I feel like this comment is throwing semantics in a blender and pouring it out into the shape you want... but I guess that's the thing about arguing semantics, it usually devolves to that...

So I guess I'll just keep my recommendation to Kotlin and you can keep your recommendation to Clojure

⬐ iLemming

> I feel like this comment is throwing semantics in a blender and pouring it out into the shape you want

I don't think it is, though. But it is clear that you are arguing with absolute confidence about a thing you have never given a heartfelt attempt to try first. You are debating like a medieval 13th-century mathematician that Roman numerals are elegant and more comfortable to understand, and people been using them for centuries and no need for this Indo-Arabian numeral non-sense that Leonardo, son of Bonacci so passionately keeps talking about.

I don't want to sound patronizing (I guess I'm already are, although not intentionally), but let me give you an advice - never trust your inner skepticism, fight it, dig for the answer - why are you so skeptical about it. Progress pushed forward by individuals who continuously challenge their beliefs. And from what I can see - you are not a mere consumer of progress, you too, do want to be at the front line where it is being made.

⬐ BoorishBears

Hahaha

Thank you for the laugh, I imagined you typing that last paragraph, reading it, and thinking you had said something pithy and being proud to share that hackneyed screed with the world.

Up until this point I haven't even shared my opinions of Clojure (which I've used) in absolute terms, did you realize this is all in relation to OP's description of "dorky languages", so I was speaking to OP's PoV as someone who probably doesn't use non C-like languages, not myself. Erlang, my pet language is plenty dorky, you seem to have confused "dorky" with "bad" or lacking utility.

But alas, let me just be straight forward, Clojure is bad.

A masturbatory aid for bored developers burning perfectly good time and money for their own overinflated sense of accomplishment and their quirky resumes.

Imagine being a language that literally lists it's half it's rationale as "our customers who won't let us run what we want, so we stuck what we actually wanted to make, on this JVM thing that they all know real well".

Clojure code bases devolve into contrived spaghetti so blindingly fast, but by god will the people writing it get off to how dense the code they're writing is while the decent into madness marches on, and boy will they enjoy how they're really sticking it to those stupid Java guys with no types... while 90% the code they interop with was clearly designed to be used in a typed setting.

And you can count down on a M-F calendar view how many days before the codebase will feature a different DSL for each programmer who's touched it which allows them to define complex business rules as a new sub-language instead of icky "normal" shudder code. Java did only a few things right, and no macros was one, imagine thinking undoing that is the right choice.

Clojure devs love to hold up the few high-profile successes and a bunch of no name success stories that are small enough to probably have served just as well by anything from Clojure to writing out Java bytecode in pico.

The funny thing is the most common successful cases actually went and tacked on a freaking type system! https://typedclojure.org/

Have they heard of F#? And if they're so allergic to types, good god why are you on the JVM and trying to interop with JVM code. If you're not trying to interop with JVM code, why Clojure? Why not Elixir or Erlang, which kick Clojure's ass at the other half of the rationale it always gets, concurrency and immutability.

Actually, don't answer that, we already know. Because JVM contains Java, and Java = business, and you're not going to get to jerk off at work with an unproductive language if it doesn't have something a business type can latch onto! You don't want to admit "we want to use this language with a much smaller hiring pool, much less mindshare, unnecessary barriers to interop with one of the largest ecosystems in tech, which is very prone to creating unmaintainable nightmares in the long term by it's very nature."! You want to express it as "we want to use Java but with parenthesis can we huh can we pls pls k thnx".

Clojure is a garbage language that always gets defended with "you just don't get it". What a joke.

Rich Hickey, the creator of Clojure, gave a talk with some very relevant points in this space (The Language of the System): https://www.youtube.com/watch?v=ROor6_NGIWU

⬐ cgdub

His point about protocol buffers (i.e. schema-out-of-band protocols) is unfortunately brief in this talk.

Depending on your use case, you may have to do a lot to work around protocol buffers not being self-describing. I haven't seen a good description of the problem online, but if you find yourself embedding JSON data in protobufs to avoid patching middleware services constantly, you should look at something like Avro or MessagePack or Amazon Ion.

⬐ dustingetz

"gRPC is great because most systems just glue one side effect to another side effect, so what's the point of packaging that into REST" – I think a HN comment from a googler

The great thing about Clojure is that you can make holistic systems that are end-to-end immutable and value-centric, which means the side effects can go away, which means gRPC stops making sense and we can start building abstractions instead of procedures!

I've watched one of Rich Hickey's talk and at some point he brings that issue on the table, what happens when the receiver is not responding to the message.

He is advocating Queues and one of his arguments is that Queues decoupling the requester from the receiver. So you don't really have to worry about things like this by using a queue.

[1] https://youtu.be/ROor6_NGIWU?t=1955

⬐ Fellshard

That's abstracting in time more than in implementation; and it's primarily useful with stateful objects that are best reasoned about as a single, synchronous timeline (which most objects are). You'll see this pattern used in, say, Actor systems, as a queue-backed inbox.

I love this talk, but it does throw out a lot of complicated ideas somewhat loosely, so I get why reactions and interpretations are all over the place.

I think a good companion to understanding this better is Rich's talk on "The Language of the System" (https://www.youtube.com/watch?v=ROor6_NGIWU&t=2810s).

I interpret his overarching thesis as this:

Most "situated" programs are in the business of processing information according to complex and changing rules, in cooperation with other programs (i.e., a system). Many languages, though, are overly-concerned with how they represent data internally: classes, algebraic types, etc. This "parochialism", he calls it and "concretion" about how data aggregates should work, make them hard to adapt when the rules, data, or other parts of the system change, and make it hard for their programs to work in systems. At some point your Java class or Haskell ADT has to communicate its data to other programs that have no notion of these things. So you end up w/ a ton of surface area in your code with mappers and marshallers totally unrelated to the content of the data and purpose of the program.

The idea behind Clojure is to provide easy access to simple, safe, and relatively universal structures for holding data, and a library of functions to manipulate those structures. Its "big picture" design bits are about providing semantics for multiple "programs" (from threads to services) in a system to operate on data robustly and reasonably (concurrency semantics, time and identity models, pervasive unadorned data, etc.) At some point you're going to be sending this program's data over a wire to another program, and things like "a map of strings and numbers" is pretty straightforward to transport, while a sum type implementing functor with a record data constructor that contains a Maybe SSN is not. It overly couples the underlying data to the language's representation.

The plus side of doing this is that the language can check internal consistency for you. The downside is that you're carrying a lot of baggage that you can't take with you over the wire anyway. Communication in systems is also why Rich thinks making "names" for data first class is important. Existing strongly typed languages can sort of accommodate this, but don't really privilege names.

So I think a lot of strong typing advocates are upset because they think Rich is saying types don't have value within programs. I don't think that's right. I think he's saying they have very limited value in open systems, which makes their costs often overwhelm their benefits in the individual programs within those systems.

In general, I feel like the debate has been about examining Rich's claims in the context of programs (is Maybe String good or bad, etc.), whereas he's really interested in what works in systems. I think that's indicated by his focus on the term "parochialism" which I have not seen a lot of folks address.

⬐ tome

> At some point your ... Haskell ADT has to communicate its data to other programs that have no notion of these things. So you end up w/ a ton of surface area in your code with mappers and marshallers totally unrelated to the content of the data and purpose of the program.

No it doesn't.

> things like "a map of strings and numbers" is pretty straightforward to transport, while a sum type implementing functor with a record data constructor that contains a Maybe SSN is not.

Yes it is.

Has this guy ever heard of Generics?

⬐ lucozade

This guy was a professional C++ programmer for a couple of decades so he probably came across generics.

I think it's possible you didn't catch the parts where he talks about what he wants from his data structures. There were 2 key pieces:

+ that he can transport them between environments, possibly remotely, possibly written in different programming languages

+ that parts of the system only need to know about parts of the data structure. More, that as the data structure is passed around the system, only the producer and consumer of changes to the structure are affected by the change.

I'm not aware of any static type system, with generics or otherwise, that would meet these goals. At least not post-facto and highly artificially.

Whether or not you agree with the priority he gives to these goals is, of course, a different matter.

⬐ Tehnix

I'm fairly certain the comment you are replying to isn't talking about "generics" but "Generics" with a capital _G_.

This bit,

> while a sum type implementing functor with a record data constructor that contains a Maybe SSN is not

indicates Hickey was talking about Haskell's type system, where you in fact can derive a Functor for you data type by using Generics.

> + that he can transport them between environments, possibly remotely, possibly written in different programming languages

There exist tools to generate types between languages.

> only the producer and consumer of changes to the structure are affected by the change.

If your function doesn't alter the data type, it needs no info on the structure of it. Perhaps you can expand on what you meant here?

EDIT: Ehrm, why the downvotes? If you disagree with the above, explain what.

⬐ carljv

Generics can make serialization easier in Haskell, but that's not exactly the point. The point is, once your Haskell program is done with that data, it's getting tossed into a message queue or database, or whatever, that doesn't really care or have any concept of what typeclasses it implements, whether one of its constructors is an Either, etc. In open systems you don't really get to decide who consumes your data or how--your program can't communicate anything other than data to them--and so you often don't have a way of enforcing your types on eventual consumers. Haskell has strong opinions about how it thinks data should be represented and aggregated. But in large open systems, as the saying goes, "opinions are like aholes; everybody has one."

When I think about the popular tools for moving data around large open system: the message queues, key-val stores, pub-subs, etc. --- it seems to me that the idea moving and communicating about types and objects over wires has largely been a dud. Thinking RMI, OODBs, etc. It's just hard to get other people (tools, services) to care about how you've decided to organize the entities in your program. It's a lot of work, and the benefits over throwing mostly "plain" data may not be compelling enough.

Again, I keep coming back to his term "parochialism" and why he's focused on it. I think it's an under-appreciated point amongst all the language wars.

⬐ Tehnix

I feel like this thread[0] in the discussion sorta delves into that. It's certainly an area with differing opinions and I can see why some might prefer having it simply be strings the whole way down, but at some point you are going to need to interact with the values you have, and at that point you need to know what type you are dealing with, so I really feel the serialisation argument is a bit weird. In databases you also have types on everything, albeit often less powerful. If not caring about types is really what you want, nothing stops you from treating everything as a String in Haskell. Heck, you can even do dynamic programming in Haskell with Data.Dynamic and Data.Typeable if you wanted to, but that sorta defeats the whole point of a nice and powerful type system.

I think it's kinda ironic for you to bring up "parochialism" or narrow-mindedness when that is exactly what I was thinking throughout Hickeys talk.

[0] https://www.reddit.com/r/haskell/comments/792nl4/clojure_vs_...

⬐ tome

> you often don't have a way of enforcing your types on eventual consumers

A type isn't something you enforce on consumers. It's something you enforce on yourself to help shape your code.

Regardless of how you put something onto the wire you're giving it a specific format that your consumers need to know about. This is the same whether it was serialised from Haskell or Clojure or Coq or assembly.

⬐ tome

> In open systems you don't really get to decide who consumes your data or how--your program can't communicate anything other than data to them--and so you often don't have a way of enforcing your types on eventual consumers. Haskell has strong opinions about how it thinks data should be represented and aggregated. But in large open systems, as the saying goes, "opinions are like aholes; everybody has one."

True, and edn is equally as parochial as any Haskell serialization format. I don't see how Hickey can claim primacy here.

⬐ wtetzner

> that he can transport them between environments, possibly remotely, possibly written in different programming languages

This is pretty easily solved in static languages with "serializable" interfaces, which can usually be automatically derived. E.g., in Rust, you can use #[derive(Serializable,Deserializable)], in OCaml, you can use [@@deriving sexp]. This also allows you to know at compile time which types can safely be serialized. In Clojure, if you have a type that contains an InputStream of some sort, it's not reasonable to serialize it. But you won't find out until runtime, when you happen to have an instance of that map with the InputStream.

⬐ dragonwriter

> Most "situated" programs are in the business of processing information according to complex and changing rules, in cooperation with other programs (i.e., a system). Many languages, though, are overly-concerned with how they represent data internally: classes, algebraic types, etc. This "parochialism", he calls it and "concretion" about how data aggregates should work, make them hard to adapt when processing rules change, and make it hard for their programs to work in systems.

It doesn't if the systems are well-designed sytems, comprised of loosely-coupled components, something like you'd get if you used 1970s structured analysis and then actually modeled the implementation closely on the DFD with communication over a message bus with a fairly neutral messaging format.

When you start tightly coupling components (e.g., by using a messaging format tightly bound to an internal representation), using ad-hoc component-to-component integration rather than a common message bus that is abstracted from the individual components, and generally do the system engineering badly, then you have a whole pile of problems, some of which are exacerbated (but not caused) by static typing, sure.

By static typing is not the problem here.

⬐ wellpast

> It doesn't if the systems are well-designed sytems, comprised of loosely-coupled components

I've been going down a similar line of thought. But I went the other direction. That perhaps in "poorly designed" systems where there is lots of coupling, static typing at least gives you the "maintenance" benefit that is one of the bigger justifications that the static type apologist tends to give. You actually hear this a lot: "in large systems, static typing is a must..."

So it's interesting to me that you're going the other way and saying that actually in big, messy systems that static types may hurt you. That's not a common position.

When I walk through some of the big problems in "poorly designed" systems it almost always comes down to coupling: I can't touch one part of the system without having an effect on other parts of the system.

Interestingly, Rich Hickey criticizes the common static typer's idioms (like pattern matching and ADTs) as coupling. And he's right. What always surprises me though is that the static typer doesn't disagree -- they look at this coupling as a feature! They usually say something along the lines of "I choose static typing because if I change my Person class, then the compiler reminds me of all the places in my code that I need to go fix." What's remarkable about this is that it's not a reminder...it's an obligation that your choices plus the compiler are burdening you with: you must go update all those places in the code. This is the very definition of coupling.

There is a way to architect code such that you don't have to revisit 100 places in your architecture when some new data model decision is made/discovered. There is a way to build systems wherein you only have to touch one place in your code when some new feature or data information is needed.

⬐ kbp

> They usually say something along the lines of "I choose static typing because if I change my Person class, then the compiler reminds me of all the places in my code that I need to go fix." What's remarkable about this is that it's not a reminder...it's an obligation that your choices plus the compiler are burdening you with: you must go update all those places in the code. This is the very definition of coupling.

> There is a way to architect code such that you don't have to revisit 100 places in your architecture when some new data model decision is made/discovered. There is a way to build systems wherein you only have to touch one place in your code when some new feature or data information is needed.

The way to avoid that has nothing to do with static or dynamic typing, though. If you change a protocol then you have to change anything that relied on the old protocol if you want your program to keep working, regardless of your language's type system; in a statically typed language it will tell you where those places are, and in a dynamically typed language it's up for you to find them. If your change doesn't break a protocol that old code relied on, then you won't have to change old code. The only changes dynamic typing "saves" you from making after you break a protocol are bugfixes.

If your code is tightly coupled so that changes ripple through the entire codebase, using a language that doesn't tell you where those changes have to ripple for things to keep working won't solve that.

⬐ benfle

I don't see the difference between sending a Person instance and sending a map of keywords about a person. The coupling is the same.

⬐ wellpast

If my function only needs to know the "age", then why am I having to fill out my Person class with all the other stuff? Why, if I have facts about a Cat in hand, must I coerce it to a Person? These are hoops you're typically jumping through when you're dealing in ADTs.

⬐ cube2222

Not in OCaml. You can have a function like that:

let printAge object = print object.age

And it'll just check if anything passed to printAge has a field "age".

⬐ wtetzner

Well, in Haskell, this seems like a case where you'd want a typeclass for getting the age out of your type.

More generally, though, it seems like row-types might be a form of static typing that would fit Rich's preferred style of programming.

⬐ tome

That doesn't seem to describe any hoops I've ever had to jump through when using Haskell. Can you give concrete examples?

⬐ wellpast

I just did. Having a Person vs. Cat taxonomy. The claim is about ADTs, not Haskell. When a "name" property will do, why do we need to introduce an ADT? Why do we need to taxonomize?

⬐ tome

Then you can use a "HasName" typeclass. Admittedly that adds a bit of boilerplate (in one single place).

⬐ carljv

I think the constant replies of "Oh there's a way to deal with that." Miss the point. You should keep asking yourself, "Am I fixing a problem that didn't need to be there?" Sometimes, the answer is: No, I do want this structure, and it's worth it overall to write interfaces, etc. to add some polymorphism or dynamism to it where needed. In lots of cases, though, you're just writing stuff to accommodate the language. In lots of languages I feel like I'm fighting an internal battle between static-ness and dynamism. Start with static types or classes, then add interfaces or typeclasses, oh and overload these functions. Now make sure these other things things implement this new interface so they can participate, etc.

Sometimes it feels like a real burden for not much gain over just passing around the basic data (a name, an age) I wanted to deal with to start with. Clojure's proposition is that in many many cases, not getting fancy with the data or over-engineering your problem representation will lead to simpler programs that are easier to maintain, giving you an alternative route to safety and maintenance instead of type-checking.

⬐ kbp

> If my function only needs to know the "age", then why am I having to fill out my Person class with all the other stuff? Why, if I have facts about a Cat in hand, must I coerce it to a Person?

If your function only needs to know the age, then why would it take a Person or a Cat at all, instead of just accepting an age parameter? But assuming you have a reason, who says you do need to coerce anything or add any dummy data? You don't even have to go very niche to get that functionality, eg in Typescript:

    class Person {
      age: number
      constructor (age: number) { this.age = age }
    }

    class Cat {
      age: number
      constructor (age: number) { this.age = age }
    }

    const printNextAge = (thing: { age: number }) => {
      console.log(thing.age + 1)
    }

    // These all work
    printNextAge(new Person(12))
    printNextAge(new Cat(23))
    const someRandomObject = { age: 10, colour: 'green', weight: 'heavy' }
    printNextAge(someRandomObject)

    // These don't:

    const lady = { name: 'carol' }
    printNextAge(lady)
    // error TS2345: Argument of type '{ name: string; }' is not assignable to parameter of type '{ age: number; }'.
    //  Property 'age' is missing in type '{ name: string; }'.

    const caveman = { age: 'stone' }
    printNextAge(caveman)
    // error TS2345: Argument of type '{ age: string; }' is not assignable to parameter of type '{ age: number; }'.
    //  Types of property 'age' are incompatible.
    //    Type 'string' is not assignable to type 'number'.

Now, if the function takes a Person, then the reason you need to fill out the rest of the stuff is because it probably wants an entire Person, not just their age. The fact that the function can tell the compiler it needs an entire Person (and not a Cat) and have it ensure that it only gets valid Persons doesn't stop you from doing anything a non-buggy program should do, it just makes the language more expressive. Even in a wordier language with a less powerful type system like Java, which obviously isn't the gold standard for static typing (and where for some reason your function was still taking an object instead of just an age int and leaving it up to the caller to extract it), it's as simple as saying:

    interface Aged {
        int getAge();
    }

and adding 'implements Aged' to your Person and Cat classes.

⬐ benfle

If "age" is an important property in your system shared among different kinds of entities then you need to have an Interface or a Protocol to retrieve the age of an entity.

The same way you would create a keyword in Clojure to represent the age of an entity (e.g. ':entity/age') that can be put in a map describing a person or a cat.

In both cases you minimized the interface between your modules and you have less coupling.

⬐ tome

> So it's interesting to me that you're going the other way and saying that actually in big, messy systems that static types may hurt you. That's not a common position.

I didn't interpret it that way. I interpreted it as "If you have a big, messy system you can tame it into a nice, loosely couple system by adding some types".

⬐ dragonwriter

> So it's interesting to me that you're going the other way and saying that actually in big, messy systems that static types may hurt you.

In an overly-coupled late system, static typing increases the potential effect of excessive coupling in forcing changes to remote parts of the system when making what seems to be a point change. But that effect, while magnified by static typing,, is a product of coupling.

And static typing in that situation, OTOH, mitigates (as to out note static proponents are quick to point out) the chance of missing a change that will produce incorrect behavior.

On the gripping hand, reducing the excessive coupling gets to the root of the problem, while static v. dynamic is just choosing how to allocate pain that could be avoided with better architecture.

But languages are sexier than architecture.

⬐ carljv

I think type systems that try to "close" aggregates (i.e. saying "an Employee is these fields that have these types and no more") kind of do contribute to the problem. I sort of agree that static types are not "causing" such problems. They don't "cause" bad design, but they tend to make it too easy to set bad designs in stone (and most designs are bad in some way). It's not so much about types causing problems or being bad, but having costs, and thinking hard about those costs vs. the benefits. Different folks will, and should, make conclusions for their problems.

I read him as exaggerating his critique a bit because types are often oversold (static type people can be really dismissive of dynamic languages). But I think he's mostly making a "no silver bullet" kind of argument.

⬐ dragonwriter

> I think type systems that try to "close" aggregates (i.e. saying "an Employee is these fields that have these types and no more") kind of do contribute to the problem. I

They are part of the problem if such types are shared among components; perhaps because of a design in which messages or data transfer object types are tightly coupled with the working representations in components.

But that's an unnecessary form of coupling.

⬐ carljv

Agreed that's bad. But then if the transit/messaging/persistence components of your systems are independent of the type system (good), to really use your type system you have to do work pushing things in and out of it, in return for type safety (and sometimes not much) that only lasts until the border of your program. It's really easy to over-engineer your types because you want really want to pin down the representation of your problem in the idioms the language gives you. ORM (ab)use is a good example of this, I think.

I've often made the mistake myself of architecting a too-clever type or class system for my problem, and then been faced with writing tons of crap to wrestle it in and out ofprotobufs, etc. that needed to be more general than my problem. When my program was running, it was like, woo, I made some illegal states unrepresentable, which felt great! But I could almost never do that in a way that didn't quickly reveal itself as too brittle.

I like types (mostly). I wish gradual/partial typing was a better solved problem. Clojure's goal is to make it so that you don't over-engineer and tangle up your systems by passing around simple immutable data. If you keep your system nicely decoupled, the types, which are good at finding when I've forgotten a coupling in my code, seem less valuable to me.

⬐ tosh

The best talk on systems architecture I've seen so far.

Is there a functional-programming-first curriculum? Even though knowing OOP is important (if even just for the vocabulary), I wonder if a lot of pain could be sidestepped by talking about data as a separate concern than ADTs, and state as a fact rather than a place. [1]

[1] https://www.youtube.com/watch?v=ROor6_NGIWU

⬐ zodiac

I think this book is supposed to be used after HtDP, which I would consider functional-programming-first.

⬐ zeckalpha

https://en.wikipedia.org/wiki/How_to_Design_Programs

⬐ macco

Yeah, there is: https://www.edx.org/xseries/systematic-program-design-0

It uses the functional approach, that also Felleisen uses.

⬐ yodsanklai

Yes, there are many books that start with functional programming. If I remember correctly.

"Concepts, Techniques, and Models of Computer Programming" start with pure functional program, and define objects as function with state. An other one is "Essentials of programming languages". Also Pierce's "Types and Programming languages"....

⬐ zeckalpha

TaPL doesn't teach FP, but rather how to design languages.

⬐ samth

This book is designed to be taught in the second course of a CS curriculum, after students take a functional-first course based on How To Design Programs.

You can see the home page of a course taught using this book here: http://www.ccs.neu.edu/home/vkp/2510-sp14/

⬐ agumonkey

SICP, HtDP, then RealWorld{Haskell,OCaml} are easy starters.

⬐ spion

I find that it helps me to think of objects as tiny machines rather than places to hold data.

It then becomes easier to reason about which classes make sense and which don't. A "User" class doesn't make sense, but UserTable (machine to access the users) and Authenticator (machine to authenticate them) do. A BlogPost class doesn't make sense, but a PostRenderer (machine to render blog posts) does.

OOP then becomes about decomposing computers into smaller communicating computers, not about lumping together data and all the possible code related to that data. Which incidentally means that there will be things that are pretty much just dumb data / facts (e.g. User)

⬐ agumonkey

And tiny machines are functions with a bit of state.

⬐ raducu

I highly recommend the Domain Driven Design approach.

But I've found it a lot harder to comprehend and make others give it a try (probably orders of magnitude harder than object oriented design or design patterns or normal refactorings).

I like to think about designing classes and aranging them in packets/modules/applications more like creating birocratic systems. You have to think in general terms what an institution/building does, what is its core activity, what it is authoritative over, then you design each room/department with its own mini-authority, make sure there is no duplication of authority/data between rooms, make sure each person inside a room does/knows about only the stuff in that room so they can really perform by being specialized and not commuting between rooms with their arms full of customer/citizen's files.

Make sure, as much as possible that each room knows about adjacent rooms only (or maybe at least at the same floor). At the same time you have to think how the regular citizens use these institutions and what interfaces you provide them with to solve their requests.

You don't want a taxpayer modifying his own tax file, you probably want a clerk to do that; you also don't want a taxpayer visiting 100 rooms to submit his tax files.

Then you have to think about how many customers you would serve, how wide the coridors should be made, what kind of security you need at the entrance and so on.

I came across this after watching Rich Hickey's The Language of The System [1], and it has really changed the way I look at distributed persistent storage.

Any thoughts?

[1] https://www.youtube.com/watch?v=ROor6_NGIWU

The downvotes are because it comes across as a shallow, middlebrow dismissal.

An interesting and useful criticism would first engage with the strongest arguments of Rich Hickey (creator of Transit and edn). If you find something in his Language of the System talk (https://www.youtube.com/watch?v=ROor6_NGIWU) that you either (a) disagree with (b) think XML solves already, I and many others would certainly be interested in having that conversation.

Note that this doesn't mean the exposition of "Why Transit" can't be better, but that calls for constructive criticism on how explain the ideas better, or a question made in qood faith. What it doesn't call for is a hostile reply saying in effect "pff, already been done already, stop reinventing the wheel".

⬐ calibraxis

Transcript of that talk: https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

I would suggest to you that Clojure has a very clear "the Clojure Way" I would say, more so then almost any other language. It goes beyond syntax to a pretty deep design level. Its not even all that unique to clojure, you could do 'the clojure way' in other languages, but they usally dont.

Some of the importent keynotes from some of the early conjs are really importent, and they really went deep in the community, you can see it in almost every major library.

Another commenter has mentioned the notion of 'simple' but thats not really all that clear. I not enougth of a writer to tell you about it, but I can give you the resources:

- This classic got Clojure on the map for many people: Are We There Yet? - Rich Hickey www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey

- Simple Made Easy - Rich Hickey - http://www.infoq.com/presentations/Simple-Made-Easy-QCon-Lon...

- Simplicity Ain't Easy - Stuart Halloway - https://www.youtube.com/watch?v=cidchWg74Y4

- Ousterhout's Dichotomy Isn't (Part 2 of the Simplicity/Power/Focus Series) - Stuart Halloway - https://www.youtube.com/watch?v=cidchWg74Y4

- The Language of the System - Rich Hickey https://www.youtube.com/watch?v=ROor6_NGIWU

- Hammock Driven Development - Rich Hickey https://www.youtube.com/watch?v=f84n5oFoZBc

In addition to these, there are many talk that are about people trying to run with these ideas. Datomic is a example of taking these ideas to the max, it really standas as a nice example. You can find others in talks by David Nolan for example, but there are tons of applied talks.

- The Design of Datomic - Rich Hickey http://www.infoq.com/presentations/The-Design-of-Datomic

This paper describes ideas around #2: http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf

But yes, your database stores the status of a message. At this point you could drop every single message on your queue and have enough information to resend them. Each message could have its own resend (SLA) semantics.

Amazon Simple Workflows is an implementation of this pattern: http://aws.amazon.com/swf/. I've never used SWF but the docs are great food for thought.

[edit]

Also might be of interest to watch Rich Hickey's 'Language of the System'. (https://www.youtube.com/watch?v=ROor6_NGIWU&feature=kp -- there are a few versions of this talk, not sure if this is the exact viewing I saw). The talk is not really about queues, but he tries to break things up a bit. You need a data store, you need a queue, etc. As soon as a queue tries to have durable messages it is becoming a database and has all of the problems a database has to deal with. Instead you could keep data storage being solved by the data storage provider and let queues focus on passing messages.

This raises the issue of how to deal with dropped messages, but that can be solved without durable queues (like that paper describes / SWF / etc).

⬐ ryanjshaw

Thanks for those links. There seem to be some good concepts in there for formulating a solution to my present personal challenge: integrating multiple disparate sources of real-time events, some transient and being delivered with low latency, and some persistent but retrieved with high latency (up to 30min!), which needs to be analyzed (and potentially replayed and reanalyzed), producing a best-effort real-time feature stream while populating (and repopulating) a reliable data warehouse. It's taking me a long time to break the problem down to the right level of components.

> As soon as a queue tries to have durable messages it is becoming a database and has all of the problems a database has to deal with. Instead you could keep data storage being solved by the data storage provider and let queues focus on passing messages.

Yes; mind you, that doesn't exclude the queue from having a persistent backing store (to reduce the instances where your application has to be involved in replay), it just means applications shouldn't use queues as the golden source of events.

This is the perfect case for hypermedia in the RESTful sense.

Rich Hickey recently gave one of his talks[1] where at one point he offhandedly mentioned that hypermedia (which gives URIs for related HTTP resources, with some link-like metadata) doesn't make sense for APIs because APIs don't have human "drivers" that can read the link text and make meaningful sense of what relations actually mean in the domain model.

I think the genius of RESTful hypermedia, though, could really come through in a system like this. How else do you show relations? Last time I checked, RailsAdmin can't do much for you here except provide the resource ID as a link to that resource's table and row.

As a benign example, imagine a guest-party admin page. What today would look like "Guest: 'Pitt, Brad', Party: '43'", tomorrow could be be "Guest: 'Pitt, Brad', Party: 'Pitt-Jolie'" with a link to '/parties/43'.

I sense that developers (or at least Crockford[2]?) don't want JSON to become overloaded with XML equivalents like XML schemas and metaschemas and XSLT...but think of what we could do!

[1] http://www.youtube.com/watch?v=ROor6_NGIWU

[2] http://www.infoq.com/presentations/Heretical-Open-Source ?

⬐ vbsteven

Json-ld takes this even further, it also supports linking data across multiple domains.

http://json-ld.org/

⬐ zachrose

I'll also mention HAL, which is spec'd in both JSON and XML and has been serving me quite well.

http://stateless.co/hal_specification.html