HN Theater @HNTheaterMonth

The best talks and videos of Hacker News.

Hacker News Comments on
Viral B Shah - The many ways of parallel computing with Julia

HasGeek TV · Youtube · 49 HN points · 0 HN comments
HN Theater has aggregated all Hacker News stories and comments that mention HasGeek TV's video "Viral B Shah - The many ways of parallel computing with Julia".
Youtube Summary
Julia is a high performance dynamic language, primarily designed for technical computing, but increasingly seeing applications in a wide variety of domains.

This talk will provide an overview of parallel computing in Julia. It will start with an introduction to using built-in Julia primitives for parallel processing, such as pmap, @parallel, remotecall, spawn, fetch, etc. Based on this low-level primitives, shared arrays and distributed arrays have been built. We will try some Parallel Linear Algebra using packages such as ScaLapack along with some MPI programming. We will also look at the possibilities of data processing with data loaded from the Hadoop file system (HDFS) and/or S3. We will also preview the upcoming multi-threading capabilities in Julia.
HN Theater Rankings

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this video.
Jul 30, 2015 · 49 points, 6 comments · submitted by leephillips
jopython
Did the developers fix the start up time yet?
StefanKarpinski
Yes and yes. Julia's startup time has been fast for over a year – accomplished by pre-compiling the base language as a shared library (.so, .dll, or .dylib). As of a few weeks ago, packages get the same treatment and thus also load quickly.
jtwebman
Why would you use Julia over something like Erlang for parallel? Is it better math support? Seems like Erlang has a good model for this but does lack in other areas.
CyberDildonics
Erlang is very good at concurrency, this talk is about parallelism. For a purely parallel task Julia should be much faster. It has many other advantages like being very good at generic programming and interfacing easily with native shared libraries.
blintzing
A big thing Julia has going for it is that it plays nicely with Fortran. Lots of legacy numerical computing code has already been written in Fortran, so it's a big draw for some companies.
tormeh
Erlang is particularly good at distributed concurrency. Erlang loses much of its value when deployed only on a single machine. Also, Erlang's single-minded focus on keeping the application going may be a little overkill for compute.

There are at least two reasons why Erlang for single-machine concurrency is bad:

1: Erlang is slow. It just is. They're working on making it faster, but Erlang has a inherent overhead coming from its focus on responsiveness and uptime. Let-it-crash, actor monitoring and one-GC-per-actor isn't free.

2: Erlang's concurrency model is based on the actor model, which assumes infinite message queue capacity. In case of insufficient memory, messages are simply discarded. While this is something you have to deal with anyway in multi-machine systems (think network buffers) it's an unnecessary worry in single-machine ones.

Now, Erlang is designed to call into native binaries for all of its single-thread high-performance needs, so assuming you only use Erlang to handle inter-thread communication on a grid/supercomputer and that communication isn't a super-frequent event, then I guess Erlang+C/FORTRAN/Julia makes sense. I guess it's the convenience of having a single language that speaks for a pure Julia solution, more than anything else.

HN Theater is an independent project and is not operated by Y Combinator or any of the video hosting platforms linked to on this site.
~ yaj@
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.