HN Theater @HNTheaterMonth

Stefan Karpinski (one of the core Julia contributors) gave a talk (in 2016, so rather dated as far as the Julia syntax and whatnot go) about how Julia is secretly more lisp-y than even lisp & Scheme. The core of this argument is that most language specifications dedicate many pages to defining how arithmetic works, whereas Julia is actually just able to define that in Julia code as part of the stdlib (the implementation just calls out to LLVM intrinsics, but nonetheless, super neat!). `Int64` is actually defined in a Julia file.

https://www.youtube.com/watch?v=dK3zRXhrFZY&t=2s

⬐ crabbygrabby

Julia's syntax is it's greatest strength but also it's biggest weakness. Large Julia code bases without team standards are complete soup. For small to medium size projects it's all good though. Just wish the community wasn't overall so crappy.

⬐ travisd

What do you mean by crappy? I've always found it very lovely and welcoming (though a touch small). The Julia community is also (in my experience) skewed towards scientific computing rather than software engineers which can definitely have an impact on things like "codebase quality" even in big important libraries. That's not a dig or insult – there are different priorities (privileging exploration, innovation, new ideas, and code that only needs to work until the paper you're writing is done rather than long term maintainablity is not a fundamentally wrong tradeoff to make).

⬐ crabbygrabby

I met a guy who got kicked away from the community because he had different political beliefs than a lot of the people there. Immediately after he got the boot he stopped contributing, they took control of his two yrs of research(multiple repos). I get it, it's OSS but at the end of the day it kinda looks like stealing someone's work. There's other instances of stuff like this too... Just hang around for a while and watch...

Not here to say it's all like that. But keep in mind if you aren't paying for the product, you probably are the product.

⬐ DNF2

There's only one person who's been kicked out of the community (in the sense of being banned from the discourse forum), and that was not due to 'political beliefs', but repeated abuse and personal attacks.

Anyway, how can anyone 'take control' of someone's repositories? Was this person kicked out of github too?

⬐ crabbygrabby

They forked them. Also this is not that person. And there's more instances of this than I think you realize.

⬐ DNF2

I don't know anyone else who got kicked out (that takes a lot). But I know of a situation where someone walked out due to a non-political disagreement. Perhaps that's the one.

I follow the community pretty closely, posters being banned (except pure spam accounts) is something I think I'd notice.

Besides, what can you do if someone walks away from an important package? Should everyone, including collaborators on that package just start from scratch? What?

⬐ crabbygrabby

Can't reply to you depth of discussion got too long I think. Yea they didn't get kicked out entirely not banned or anything but they became "persona non grata" over stuff in their personal life. At least that's how it was explained to me.

I dunno what you should do if someone leaves, but bullying someone until they leave, then after they do forking all their work is kinda crappy. Again I get that it's OSS, but a lot of people don't make OSS contributions and hope for that kind of outcome. It's worth putting out there imo.

⬐ StefanKarpinski

From this description this is fairly obviously the LightGraphs situation, and that's a pretty misleading account of what happened. The person was not kicked out of anything—they were not blocked or banned from any platforms or forums. They choose, of their own volition, to stop participating in the community. I've never seen any evidence of bullying for political views or otherwise; maybe there was some, but if so it was never reported, and it would have been a clear and actionable community standards violation. Whatever their reasons, this person decided they wanted to leave, which is unfortunate—we don't want anyone to feel unwelcome—but it's their prerogative.

That would have been fine, if unfortunate, but they also wanted to "take their work with them" in the sense of archiving their registered open source package repos preventing any further maintenance or development. This desire was not about not wanting the maintenance burden—they were not willing to grant ownership of the repos to other maintainers. In short, the original author wanted to force all development of the packages by anyone to stop. Of course, that would have left all the people who had come to depend on those packages high and dry, since the code they'd come to depend on would get no bug fixes, security patches, etc. Despite the fact that there were active contributors to that code who were happy to take over maintenance.

Imagine if Linus Torvalds got mad one day and decided to insist that no one could do any further development of the Linux kernel. No bug fixes, no security patches, no new features. Linus out. That was the situation here. Fortunately this is not how open source works: open source licenses are not revokable and the ability to fork a project is baked into each license for this exact reason—so that a disgruntled author cannot screw over an entire community of people who have come to depend on their work. They don't have to keep doing work, but they also can't take away they've done. If Linus threw a tantrum and refused to allow any more work on Linux, the rest of the community could take over and continue maintaining the kernel—fixing bugs, patching security flaws, even adding new features. Linus could close down his git repo and never touch the kernel again, but other maintainers could continue to develop Linux and support the vast community of users who have come to depend on it.

Similarly, it would have been perfectly legal to fork LightGraphs and continue development in a new repo with the same name. Out of respect for the original author's wishes, however, the LightGraphs package was allowed to be "frozen" with no further development. But it would have been deeply irresponsible to cease all maintenance and leave all the people who use and depend on LightGraphs hanging, especially given that there were willing maintainers. So LightGraphs was forked and renamed to "Graphs"; the old repo has been allowed to remain frozen, while maintenance and development has continued under the Graphs name in a new repo. The author of LightGraphs got their wish for work on the thing called "LightGraphs" to cease. The users of the package didn't get screwed over since they can do a simple search and replace and keep using a maintained graphs package. Personally I think the community handled it with responsibility and grace.

⬐ DNF2

There's another issue. In addition to the open source license and what it promises, when you accept contributions from others it isn't just your work anymore. LightGraphs had 100 contributors, what about their efforts? Not to mention additional work that others have done on top of that in other libraries.

Who would contribute to a software library if they knew that the main dev could just mothball their efforts at any moment.

If you have donated a ball, you can no longer just pick it up and go home. If you don't want to donate work, don't do open source and invite others to join in.

⬐ DNF2

If you click on the timestamp ("17 minutes ago"), then you will be able to reply below the post.

OK, I don't know that person's situation then, and cannot speak to it. But bullying is definitely against the community guidelines, and my experience is that there's not a high tolerance for rudeness, in fact I think the community is quite conflict shy.

That this happens with some frequency is a pretty big surprise to me, as I said, I follow the community closely.

Is it possible that you know only one side of the story?

⬐ JonChesterfield

Defining parts of the language as executable code instead of prose is attractive. One drawback is over specification - the reference implementation may have inessential features that a prose version could leave to the implementor - but overall I think that risk is minor compared to having executable test cases/examples embedded in the standard.

Arbitrary size integers written in lisp is a thing. Maybe gambit and sbcl? Can't remember offhand. Calling down to llvm.sum.i64 or w/e is very similar to calling down to x64.

Personally, I think when you write Julia in shorthand form, it looks nothing like ruby or basic.

for example, check out parts of the stdlib: https://github.com/JuliaLang/julia/blob/master/base/operator...

but in the end, julia really is a lisp: https://www.youtube.com/watch?v=dK3zRXhrFZY

I'm a great admirer of Julia, but I doubt it would appeal to those seeking a "true" lisp/scheme. Even if it's has a sane macro system, and much of the same power. You can indeed see s-expressions (search for "s-expression") :

https://docs.julialang.org/en/v1/manual/metaprogramming/inde...

This nice talk mentions a bit on how Julia is (not) a lisp (if I recall correctly):

"Julia: to Lisp or not to Lisp?": https://m.youtube.com/watch?v=dK3zRXhrFZY

This might also be of interest: https://tpapp.github.io/post/common-lisp-to-julia/

As well as (maybe) clasp: https://github.com/clasp-developers/clasp

You might also want to have a look at Julia. It's not really a lisp, but it sits on top of a scheme - and it has a rather interesting, pragmatic type system:

"Julia: to lisp or not to lisp?": https://www.youtube.com/watch?v=dK3zRXhrFZY

https://julialang.org/

Maybe compare and contrast with Maxima?:

http://maxima.sourceforge.net/

> ... a relatively fast functional language that replaces most of the brackets with indentation.

You might like sweet expressions coupled with a common lisp, a scheme or racket:

http://readable.sourceforge.net/

http://docs.racket-lang.org/sweet/index.html

... Or perhaps have a look at Julia:

https://julialang.org/

"Julia - to Lisp or not to Lisp":

https://m.youtube.com/watch?v=dK3zRXhrFZY

[ed: and as mentioned below, there's also wisp:

http://www.draketo.de/english/wisp

]

> Actually, in general Julia is surprisingly Lispy for a language with Algol-family syntax.

It's less surprising if you know there's still a version of the Julia REPL that works with sexpressions: https://youtu.be/dK3zRXhrFZY?t=5m58s

My initial thought was that this was about a project building atop of clasp, like CANDO:

https://youtu.be/5bQhGS8V6dQ?t=322

https://github.com/drmeister/clasp

https://github.com/drmeister/cando

But this does seem a bit more like Julia?

"Julia: to Lisp or not to Lisp?"

https://www.youtube.com/watch?v=dK3zRXhrFZY

Then there's also the Axiom system, which I guess is primarily for symbolic computation, but I don't know if it might make sense to use it as a building block for statistical software?

http://axiom-developer.org/

I'm not sure if "special language" is a particularly apt description of what Julia is. SQL is a "special" language.

At any rate - a lisp-like language with llvm backend, package manager, and a (small, but not trivial) group of users writing real software - what's not to like?

I (still) recommend the talk: "Julia - to lisp or not to lisp?", that gives a quick overview of some of the design choices:

https://youtu.be/dK3zRXhrFZY

For me, I like that it has a real story trying to balance "actual integer math" (not this silly machine constrained twos-compliment hack" and "you could conceivably write a ray tracer that wasn't unusably slow" :)

Speaking of talks on Julia, I'll again recommend:

https://www.youtube.com/watch?v=dK3zRXhrFZY

"Julia: to Lisp or not to Lisp?" from the European Lisp Symposium in May 2016. I found it was rather illuminating on Julia in particular and lisps and programming languages in general.

From the linked [lwn] article:

"Another example he gave demonstrates the slowness of the C runtime:

    import itertools
    sum(itertools.repeat(1.0, 100000000))

That will calculate the sum of 100 million 1.0s. But it is six times slower than the equivalent JavaScript loop. Float addition is fast, as is sum(), but the result is not.

Larry Hastings asked what it was that was slowing everything down. Modzelewski replied that it is the boxing of the numbers, which requires allocations for creating objects. Though an audience member did point out with a chuckle that you can increase the number of iterations and Python will still give the right answer, while JavaScript will not."

Reminded me about the excellent talk about Julia: "Julia: to Lisp or not to Lisp?" https://www.youtube.com/watch?v=dK3zRXhrFZY

One things he points out early is that both the C99 and the R6RS Scheme spec is 20% numerical. Because correct and (reasonably) fast numbers and arithmetic is actually pretty hard to get right on a computer - if you want to abstract away hardware "short-cuts" and allow for precise arithmetic by default.

It will be interesting to see how much type hints (eliminating some of the boxing/unboxing) will help python. And if it turns out to really be a good fit for the language -- everyone wants "free" performance, but transitioning to a (even partially) typed language is certainly not "free".

Another point with the above loop, is that ideally, even if it can't be optimized/memoized down to a constant - it really shouldn't have to be much slower than its C counterpart. Except for handling bignums in some way or other (perhaps on overflow only).

[lwn] https://lwn.net/Articles/691243/

⬐ hyperpape

My understanding is that there is no plan to use type hints for optimization. I'm on a phone so I can't find it now, but I previous talked to some people on here about it in comments.

⬐ e12e

Please let us know if you find a reference to that - I've not seen anything either way (other than perhaps eg: cython piggybacking on the syntax for its own use - but not something strictly speaking "in (c)python". That said, according to: https://www.python.org/dev/peps/pep-0484/#rationale-and-goal...

"This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information.

Of these goals, static analysis is the most important. This includes support for off-line type checkers such as mypy, as well as providing a standard notation that can be used by IDEs for code completion and refactoring."

So optimization doesn't appear to be a non-goal - as much as not a primary goal?

⬐ kylebgorman

Optimization is only a distant goal according to Guido---I think he said so in the PEP or the dev mailing list discussion thereof.

⬐ hyperpape

I did say "no plan", not "never gonna happen." I don't think even Guido could really promise the latter.

I went back and looked, and this is the best reference I could find: https://news.ycombinator.com/item?id=9847740. It's not much more informative than the present discussion, and the best I can say is that I don't know anything about efforts to make cpython use the annotations for any changes in runtime behavior (either through code generation, or at runtime).

⬐ bjourne

> Another point with the above loop, is that ideally, even if it can't be optimized/memoized down to a constant - it really shouldn't have to be much slower than its C counterpart. Except for handling bignums in some way or other (perhaps on overflow only).

Sadly, bignum handling is very expensive even with type hinting. Essentially, every time two numbers is added or subtracted you have to check for overflow:

    int z = hw_add(x, y);
    if (over/under-flowed?()) {
      bignum* bn = new bignum();
      bignum_add(to_bignum(x), to_bignum(y), &bn);
      return bn;
    }
    return z;

So most modern statically typed languages (none named, none forgotten!) actually cheat and use modular arithmetic instead. For example, the straightforward way of computing 9223372036854775807 + 1 on a 64bit system in one of those languages is likely to yield an incorrect result.

Which imho is complete bullshit because there is no reason to sacrifice correctness for performance other than to win benchmarking contests.

⬐ rurban

This overflow check is extremely cheap. Just check the overflow flag on the cpu. jo +4; jmp bignum; 6 byte. C compilers didn't support it until last year, so everyone just used inline assembly. Now we have it in gcc 5 and clang 3.4

One good trick started by lua is to use double for everything and check the mantissa of the normalized result for overflows. A double can represent the whole int32 range. With double and nan-tagging you have a unique fast representation of all primitives and can use a double word stack. Which is the biggest win. Look at how python, perl and ruby are doing this. Horrible. And then look how a fast dynamic language is doing it.

⬐ bjourne

Everything is relative. :) Compared to the add instruction itself, the overflow check is very expensive. But the big problem is that your types widen. E.g:

    // x and y are hardware integers >= 0
    var z = x + y
    for (var i = 0; i < z; i++) {
        ...
    }

If your language used modular arithmetic, 'z' would also be a hardware integer and the induction variable 'i' could be kept in a cpu register. But with real arithmetic using bignums you can't assume that anymore and each iteration of the loop must do a relatively expensive comparison of 'i' against 'z'. In pseudo code:

    var z = x + y
    var i = 0
    if (!hw_int(z)) i = new_bignum(0)
    while (true) {
      // i < z
      if (hw_ints?(i, z)) {
        if (i >= z) break
      } else {        
        if (bignum_gte(i, z)) break
      }        
      ... body ...
      // i++
      if (hw_int?(i)) {
        i++;
      } else {
        i = bignum_add(i, 1)
      }
    }

⬐ rurban

No, it's super cheap. overflows happen <1%, so you mark it as UNLIKELY, and the branch prediction units just don't care about all those bignum branches.

If you know to use bigints or bignums, just type it and use their variants from the beginning.

Most better dynamic languages already use tagged ints, so they have only say ~30bits for an int and do a lot of those overflow checks and still beat the hell out of the poorly designed dynamic languages with full boxed ints or nums.

This reminded me of the recent brilliant talk about design of Julia: "Julia: to lisp or not to lisp?" https://m.youtube.com/watch?v=dK3zRXhrFZY

Best talks in my opinion:

* https://www.youtube.com/watch?v=5bQhGS8V6dQ&index=15&list=PL...

* https://www.youtube.com/watch?v=dK3zRXhrFZY&index=11&list=PL...

Notable mentions:

* https://www.youtube.com/watch?v=6GO1rust6SU&index=8&list=PLA...

* https://www.youtube.com/watch?v=--gqVzhLYoI&list=PLA66mD-6yK...

* https://www.youtube.com/watch?v=JU-BL4XAOHo&list=PLA66mD-6yK...

I liked OpenGL lightning talk as well:

* https://youtu.be/2ZEYxxFR_Gs?t=27m34s

⬐ Flow

Ah, the Julia talk ( https://www.youtube.com/watch?v=dK3zRXhrFZY ) was just great. Showing the generated machine code is so hackerish yet so rare these days. :-/

⬐ agumonkey

> https://www.youtube.com/watch?v=5bQhGS8V6dQ

no wonder CANDO is impressive, the scope and fields involved are pretty rare...

I'm very eager to learn if C. Schafmeister work will bear 'disruptive fruits'.

ps: a talk about clasp https://drmeister.wordpress.com/2015/06/15/i-gave-a-talk-on-...

⬐ eudox

At around 16:00 he mentions a water-filtering molecule that works more efficiently than reverse osmosis.

⬐ agumonkey

I'm just passed his simulation demo, I have to rewatch it to get the details.

My above message was more about actual production. I find his work already revolutionary enough. Feels like a new bridge between computer aided <x> and biochemistry. I don't know if that part is completely new though.

⬐ golergka

https://www.youtube.com/watch?v=8X69_42Mj-g

This talk is even better. "C++ templates are to Commmon Lisp macros what IRS tax return forms are to poetry". I don't know both well enough to tell if it's true, but damn it's a good metaphor.

⬐ TeMPOraL

I know both well enough and I think it's very true :).

Christian Schafmeister's talks are great and I'm very impressed by his work. It was a great experience to meet him personally on this year's ELS. One thing I learned from chatting with him during the conference break is that his work on Clasp is actually pushing LLVM forward - when implementing Common Lisp / C++ cross-linking he found some bugs that were there because apparently no one before has been trying to do the stuff he's doing - so he had to nag some Apple engineers to get to work and fix them ;).