HN Theater @HNTheaterMonth

Is that web scale[0]?

[0] https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ quickthrower2

It is inflation, the central Xer bank keeps printing better developers.

> I understand “web scale” in the marketing term Mongo sense

That's the joke.

(And for anyone who hasn't seen it: https://www.youtube.com/watch?v=b2F-DItXtZs)

Sorry, but when I see "Web Scale" or "MongoDB" I can't help but recall https://www.youtube.com/watch?v=b2F-DItXtZs

MongoDB is web scale. https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ atmosx

LOL. Okay...

You are (presumably inadvertently) mirroring a character who is the butt end of the joke in a well-known meme.

<https://www.youtube.com/watch?v=b2F-DItXtZs>

Adding the reference [1] in case it is obscure for some.

[1] - https://www.youtube.com/watch?v=b2F-DItXtZs [video][language]

"Mongo DB is a web scale DB and doesn't use SQL or Joins so it's high-performance. Everybody knows that relational DBs don't scale because they use Joins and Write-To-Disk."

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ barking_biscuit

I always love watching this.

> "We've hit an error an error that the manual says nothing about, but our DB expert saw this once, 7 years ago, and remembers how to recover"

Yup, this rings true. Once worked on a large Cassandra deployment – holding people's bank accounts, so, ya know, somewhat critical – and we eventually had to get one of the authors of Cassandra onto our on-call team. Even he was stumped by some of the bugs and failure modes. (And, incidentally, the word 'anticompaction' is still enough to trigger my PTSD.)

The notion that the documentation is an accurate and exhaustive explanation of every single thing about a product is a junior engineer mistake that no one with rough ops experience would say. Nor, for that matter, anyone who's written a database that's in use (as I unfortunately have), or really any other piece of software.

And the same goes for the notion that "everyone should go with MySQL or Postgres for everything because they are Proper Databases that are scalable and won't go down". If I were being charitable, I'd say it's a 'no one got fired for buying IBM' argument; if I were being honest, I'd say it puts me in mind of 'MongoDB is webscale' (https://www.youtube.com/watch?v=b2F-DItXtZs). I don't know why bad engineers are so terrified of thinking for themselves specifically – well, generally, but especially – when it comes to databases. They aren't magical contraptions. They are just portable filesystem wrappers.

Still an excellent comparison https://youtu.be/b2F-DItXtZs

> Simply bring your code and scale up.

reminds me of

> MongoDB is scalable, you turn it on and it scales right up

https://youtu.be/b2F-DItXtZs

This video should speak to you

https://youtu.be/b2F-DItXtZs

⬐ Beltiras

I could find a single sentence from the noob that I could agree with. Redis is superior to memcached. Not due to being faster or scalable, I've just had less operational problems with it than memcached. It's on my list of supersoftwares that are first choices to solve a problem in it's domain (nginx, uwsgi, postgresql, redis).

Glad you liked it! The reference to this series of videos was intended of course.

The specific video I had in mind was actually https://youtu.be/b2F-DItXtZs

Enjoy!

⬐ fzil

Oh yes! I’ve seen this before as well. Should’ve guessed it’s this considering we were talking about sql.

The hvasilev's comment got flagged and I could not reply to it anymore, so I'll reply here (sorry) and copy-paste the hvasilev's comment verbatim below, for the sake of commenting on it's claims:

---

Reality is not on the side of this language. 11 year old, has a very low adoption with virtually no jobs associated. (https://www.tiobe.com/tiobe-index/)

On the other hand if you search for "Rust" in the latest "Who wants to be hired?" thread, you will see it is quite popular with unemployed people.

The reality is that the language has a lot of friction, the ergonomics are bad, the syntax is heavy and some poor decision making has been made there for a systems-level programming language.

There are a lot of ideological traps in this industry and many people that fall for them. Why people are interested in ideologies and cults is beyond me.

[ A screenshot: https://imgur.com/a/tgAETjh ]

---

<rant>

Now my comments on the issues mentioned in the hvasilev's comment.

> Reality is not on the side of this language. 11 year old, has a very low adoption with virtually no jobs associated. (https://www.tiobe.com/tiobe-index/)

Tiobe index is shit. The most flattering thing I've read about it states that it (poorly) depicts quantity of educational materials available online for particular programming language. Unfortunate naming of programming languages after letters of alphabet, symbols (++, #) and real-life stuff (like islands) doesn't help this rating either.

That said, Rust isn't that popular and isn't growing much according to other better language ratings:

1. https://tjpalmer.github.io/languish/

A Github-based rating created by the author of Context Free YouTube channel. For 2021Q3, Rust is on 18th place with Mean Score of 0.82% (up 0.01% from 2021Q2).

2. https://madnight.github.io/githut/#/pull_requests/2021/3

A Github-based rating. For 2021Q3, Rust has 0.64% of Pull Requests, 0.30% of Pushes, 1.29% of Stars and 0.65% of Issues. Growth dynamic is quite flat.

3. https://redmonk.com/sogrady/2021/08/05/language-rankings-6-2...

Latest RedMonk language rating, dated June 2021. Rust is on 19th place with 0 growth.

---

About jobs comparison.

Indeed.com for California.

Rust: 527 jobs (with some unrelated stuff mixed in)

https://www.indeed.com/jobs?q=rust%20developer&l=California

JavaScript: 16,792 jobs (31.8x)

https://www.indeed.com/q-Javascript-l-California-jobs.html

Java: 12,418 jobs (23.5x)

https://www.indeed.com/q-Java-Developer-l-California-jobs.ht...

C++: 4,172 jobs (7.8x)

https://www.indeed.com/q-C++-Developer-l-California-jobs.htm...

Indeed.com for New York.

Rust: 85 jobs

https://www.indeed.com/jobs?q=rust%20developer&l=New%20York%...

Java: 4,815 jobs

https://www.indeed.com/jobs?q=java%20developer&l=New%20York%...

Javascript: 4,037 jobs

https://www.indeed.com/jobs?q=javascript%20developer&l=New%2...

C++: 1,126 jobs

https://www.indeed.com/jobs?q=C%2B%2B%20Developer&l=New%20Yo...

glassdoor.com without location set.

Rust: 492 jobs

https://www.glassdoor.com/Job/rust-developer-jobs-SRCH_KO0,1...

Java: 45005 jobs (91x)

https://www.glassdoor.com/Job/java-jobs-SRCH_KO0,4.htm

Javasript: 30952 jobs (62.9x)

https://www.glassdoor.com/Job/java-script-developer-jobs-SRC...

C (with unrelated stuff): 9594 jobs (19.5x)

https://www.glassdoor.com/Job/c-developer-jobs-SRCH_KO0,11.h...

Go(lang): 1406 jobs (2.85x)

https://www.glassdoor.com/Job/golang-developer-jobs-SRCH_KO0...

Judge for yourself if a few hundred jobs in places like CA and NY count as "virtually no jobs".

For comparison, in Ukraine (population 35-41 millions) there is 6 Rust jobs:

https://jobs.dou.ua/vacancies/?search=Rust

... 190 C++ jobs:

https://jobs.dou.ua/vacancies/?search=C%2B%2B

... and 676 Java jobs:

https://jobs.dou.ua/vacancies/?search=Java

... listed on the largest Ukrainian programming site.

---

> The reality is that the language has a lot of friction, the ergonomics are bad, the syntax is heavy and some poor decision making has been made there for a systems-level programming language.

Well, this is matter of taste, largely. But I have a few issues with Rust syntax too (IMHO):

1. F--king single quotes. Eww, really?! IIRC, a tilde (~) character was used for lifetimes until some Europeans (?) complained that their keyboards have no tilde. I wonder, how they programmed in C++ all that time? For years, if I met online a piece of code that was highlighted as a comments mishmash I knew exactly in what language it was. Ugly as f--k.

2. Closures using pipes (|). With no arguments they look like OR operator (||). Distracting.

3. Using angle brackets for generics.

4. Double colons (::) as "path qualifier" produce too much visual noise. Java likes long pathes too and uses dot (.) as separator just fine.

5. What with this arrows (->) before return types? Seems unnecessary. Couldn't return types be purely positional as in Go?

I don't use Rust so it's mostly "glimpses from the outside".

Speaking of friction. This reminded me of a video by Jonathan Blow (creator of Braid and The Witness games and Jai programming language). The video is worth watching whole but piece about friction in gamedev starts approximately at 49:23.

"Rant: Entity systems and the Rust borrow checker ... or something."

by Jonathan Blow

Sep 14, 2018

https://www.youtube.com/watch?v=4t1K66dMhWk&t=2962s

As for poor decision making, it was pretty poor decision to include an npm knock off into the language. I'm speaking of crates.io repository. For some time it has a squatting problem that isn't fixed yet:

https://old.reddit.com/r/rust/comments/9aaanw/cargo_crate_na...

3 year old thread, the squatter is still there holding 104 packages. At least npm has namespaces.

I wonder, if cargo will turn into malware-ridden micro-dependency hell too?

---

> There are a lot of ideological traps in this industry and many people that fall for them. Why people are interested in ideologies and cults is beyond me.

"MongoDB is web scale" video nicely illustrates "ideological traps" and cult-like behaviors in "this industry".

http://www.mongodb-is-web-scale.com/

https://www.youtube.com/watch?v=b2F-DItXtZs

And what else illustrates cult-like behaviors? Flagging an innocent comment you don't agree with. What was in the hvasilev's comment that warranted its removal? In my opinion, nothing. It contained no insults, no personal attacks, and was more or less factually correct. Rust is relatively unpopular, complex, ideological, syntactically-heavy language with relatively few job offerings, i.e. pretty much what the hvasilev's comment said.

Complaining about the comment being "off topic" is somewhat funny given Rusters' penchant for inserting their language into discussions about other programming languages (especially C, C++ and Go).

</rant>

⬐ the_only_law

> Complaining about the comment being "off topic" is somewhat funny given Rusters' penchant for inserting their language into discussions about other programming languages (especially C, C++ and Go).

The childlike “well they do it too” argument is almost the perfect example of how the anti-rust crowd is becoming even more obnoxious than the infamous Rust evangelism strike squad.

For the record , I’ve never written a line of rust in my life and am not particularly invented in its success or failure. And no, I didn’t flag GP.

⬐ fee1-dead

> What with this arrows (->) before return types? Seems unnecessary. Couldn't return types be purely positional as in Go?

I guess you could say `->` is too verbose and you can omit it in other languages. Rust has a complex type system and there can be confusing code when you omit `->`.

1. Closure Return Types.

You can define a closure with an explicit return type:

``` let my_closure = |i: u32| -> u64 { i as u64 }; ```

Now how do you omit the arrow here? How do you know `u64` is the return type and not constructing a struct?

2. Parsing stuff

It becomes impossible to parse none-delimited types. Is `fn() fn()` two different types or a function pointer returning a function pointer?

3. Readability

I mean, tokens can be read out loud and omitting it stops making sense.

`fn foo(bar: i32) -> f32` can be read as a "function named 'foo' that takes an argument named bar with type i32 and returns f32". The word returns directly corresponds to the `->` token.

Rust also has the `!` (read: never) type. Poorly formatted code when `->` is omitted is very confusing: `fn a()!` when compared to `fn a()->!`, or just one character generic types: `fn a<T>()T` compared to `fn a<T>()->T`.

⬐ arkush

Thank you for clarification.

Seems like arrow (->) is the best choice for Rust, given arrow's visual distinctiveness and search-ability.

> I watched C series startups throwing it all away in a move to NoSQL

have you forgot that MongoDB is web scale?

https://www.youtube.com/watch?v=b2F-DItXtZs

https://m.youtube.com/watch?v=b2F-DItXtZs

MongoDB is web-scale: https://www.youtube.com/watch?v=b2F-DItXtZs

NSFWish; it gets a bit personal around 3:11

⬐ mlindner

I miss 2010.

⬐ vinay_ys

Ah, 2010 - when web scale and its secret sauce – sharding was all the rage.

⬐ alexjplant

I had a similar conversation with a heavily-intoxicated MongoDB sales guy in a diner at 1AM after the second day of KubeCon 2019. My concerns were primarily around data consistency issues during denormalization and lack of schema. H pitch was essentially "Who cares?! I'm getting [three-letter agency] to move _everything_ to Mongo because it's so cheap and easy! It's all just JSON! Why does it need a schema?!"

He probably made more than I did that year so maybe he has a point ¯\_(ツ)_/¯

Link for the Mongo video: https://www.youtube.com/watch?v=b2F-DItXtZs

And bonus lean startup video: https://www.youtube.com/watch?v=3J9KhpgYVB0

Much more infamous.

"Mongo DB Is Web Scale" - https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ keymone

s/in//

⬐ shirleyquirk

famous ternal?

⬐ tusharsadhwani

that would be s/in//g :)

MongoDB is a web scale

https://www.youtube.com/watch?v=b2F-DItXtZs&t=2

But is it webscale? [1]

1. https://youtu.be/b2F-DItXtZs

obligatory https://youtu.be/b2F-DItXtZs webscale database discussion

Reminds me of the second half of 'MongoDB is web scale' https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ da39a3ee

Yep, you got the quote! And I've even (narrowly) had more vote ups than down.

Remember in the early 2010's when every bro on the Internet became an expert on scalability because he used MongoDB? https://youtu.be/b2F-DItXtZs (sort of like how Rust users are security experts today) I've been working on building a better database than MongoDB and it's called redbean. https://redbean.dev/ It supports web-scale nosql because you can use the executable pkzip structure as a document object store. It also embeds SQLite for SQL too.

I heard Mongodb is webscale[1]. Would Mongo work as a replacement for TCP?

[1] https://www.youtube.com/watch?v=b2F-DItXtZs

For those who didn't get the reference: https://www.youtube.com/watch?v=b2F-DItXtZs

"mongo said" - the obligatory mongo DB is webscale video:

https://m.youtube.com/watch?v=b2F-DItXtZs

So they ended up adopting MongoDB? :)

https://www.youtube.com/watch?v=b2F-DItXtZs

But is it webscale? :) https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ davidlatwe

Not just yet, stay tune for /dev/null based storage engine !

⬐ MaBeuLux88

It's already implemented actually :') ! Trust me. I'm an engineer :D. https://github.com/mongodb/mongo/blob/72ed8227aa029afd554aa5...

Sorry I can't resist: https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ srswtf123

This video is one of the best critiques of modern software development, and I suppose IT in general, that I've ever seen. Even if I've seen it a hundred times, I always appreciate seeing it again. Thank you!

⬐ numpad0

I have suspicion that a lot of people who jumped to NoSQL just didn’t like the aesthetics of SQL syntax, in the same sense as people would not be ecstatic about Visual Basic.

Reminds me of the meme/video about mongodb: "It's webscale".

https://www.youtube.com/watch?v=b2F-DItXtZs

Only if I _know_ I'm creating something that will definitely have huge amounts of concurrent users and someone pays me to make it scale from the start.

For a hobby project that might take off or might not, there's really no point in making everything "webscale"[0] just in case.

[0] https://youtu.be/b2F-DItXtZs

MongoDB is webscale:

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ krylon

I almost messed my pants watching this! It's hilarious! Thanks!

⬐ hinkley

Too soon, man. Too soon.

⬐ closeparen

The medical profession's use of this format is also hilarious: https://www.youtube.com/watch?v=3rTsvb2ef5k

https://www.youtube.com/watch?v=b2F-DItXtZs

https://www.youtube.com/watch?v=bzkRVzciAZg

For the uninitiated: https://m.youtube.com/watch?v=b2F-DItXtZs

Mongodb is Webscale - https://youtu.be/b2F-DItXtZs

⬐ ericpauley

> Does /dev/null support sharding?

Timeless.

This reminds me of that joke video about replacing MongoDB with /dev/null. Look at the write speeds! /dev/null is web scale!

https://www.youtube.com/watch?v=b2F-DItXtZs

The "Mongo DB is Web Scale" video just turned 10 years old. I'm pretty sure that was a big factor in MongoDB not being taken seriously by many people.

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ gramakri

Ah good memories. Such gems.

"relation dbs have impotence mismatch"

"does /dev/null support sharding"

"redis will kick memcache's ass"

⬐ rudolph9

This video is pretty funny! I’ve used mongodb extensively over the years. It’s pretty solid and enjoyable to use, but yeah the js community tends to have a higher percentage of “fake it until you make it” type people and it seems like a lot of them use mongo too...

⬐ athms

Plus, it was designed by people with no database expertise. Lots of people were burned by MongoDB stupidity. I have heard the company hired competent people since. Once burned, twice shy.

⬐ simplyaccont

Not this https://aphyr.com/posts/284-jepsen-mongodb ?

⬐ threeseed

Anybody who makes complex technology decisions based on marketing copy doesn't deserve to be taken seriously either. Not that I think these people actually exist other than in the minds of many HN commenters.

Oracle on their website right now says "we lead the market in autonomous, cloud, and applications technologies".

I assume you think developers are going to suddenly abandon AWS, Python etc and move entirely to an Oracle stack based off this quote ?

⬐ ralph84

Oracle's marketing is more to preserve their existing install base than acquire new customers.

⬐ blibble

at large firms the clueless technology managment often buy based on the marketing crap, then force it down the throats of the engineers

at my firm we're being all but forced to use Azure even though it costs more and makes absolutely no sense whatsoever for the domain, but since it's the "technology strategy" fighting it becomes exceptionally difficult

⬐ jd_mongodb

In seven years working at MongoDB the one thing I have learned is technology is never "forced down the throat of the engineers". Developers have choices and would not stay in an organisation that made decisions that way. The reason developer relations exists is the acknowledgement that developers ARE the decision makers in most technology selection.

⬐ NegativeLatency

I recently worked for a company where a different popular JSON document database was in fact forced down my throat.

I proved that a regular database would be orders of magnitude faster with a real example on real data and while the project/product was canceled for marketing/sales reasons. Until it was canceled we were told by multiple levels of "technical" managers to use the worse database because of unknown reasons.

⬐ jd_mongodb

Technology selection is a black art in many companies. Sounds like people were open to listening to the opinion of a developer in your case. Often-times the selection heuristic is habit or history. The right companies will pay more attention to reasoned arguments of their own developers.

⬐ NegativeLatency

The wild part about it was that they were trying to use elastic search for relational queries because some manager thought it would be faster than a relational db. (It was about 60x slower)

Which reminds me: it's almost the 10 year anniversary of "Mongo DB Is Web Scale" https://www.youtube.com/watch?v=b2F-DItXtZs

Haha yeah me too :)

This is the video if anyone is wondering:

https://www.youtube.com/watch?v=b2F-DItXtZs

I just want remind people that this video exists.

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ sorokod

Well, does dev/null supports sharding?

Obligatory https://www.youtube.com/watch?v=b2F-DItXtZs

Some readers might not be familiar with that particular meme: https://m.youtube.com/watch?v=b2F-DItXtZs

IMHO it perfectly describes the hype-reality disconnect at the early days of MongoDB. Yeah it was that bad.

Mongo has improved since, the hype has toned down and the NoSQL space is more crowded these days.

⬐ collyw

The damage is done. I have to use this crap at work when we should be using an SQL database. We have been planning a migration since before I started a year and a half ago. I won't be surprised if we are still on Mongo in another year and a half.

⬐ karatestomp

Fantasy: "Don't use special database features (by which I mean, like, any features) and make sure our ORM supports a ton of different datastores because we might want to change to a different database at some point and don't want to be tied to this one."

Reality: Three app rewrites later plus another application written talking to the same DB, and the database is still the same.

⬐ znpy

i remember diaspora chanting about using mongodb.

then a year or two later they admitted that their data model mostly fitted the relational model, and that they spent a lot of time basically reimplementing relational integrity in application code, in ruby.

yeah, diaspora has never been fast. I'm not sure they can blame it on mongodb though.

⬐ collyw

I remember the Mongo hype when it came out and I really couldn't understand it. You are just throwing away a lot of useful features of a relational database because "schemaless" and "big data". The majority of people using it were on single server setups.

> "Unfortunately, it turned out that when you're a database, it doesn't matter how much data you lose, only how fast you are while losing it."

Ahh yes, just like the dialogue of the never-stale "Mongodb is Webscale" meme video:

https://www.youtube.com/watch?v=b2F-DItXtZs

http://www.mongodb-is-web-scale.com/

> MySQL is slow as a dog. MongoDB will run circles around MySQL because MongoDB is web scale.

> "MongoDB does have some impressive benchmarks, but they do some interesting things to get those numbers. For example, when you write to MongoDB, you don't actually write anything. You stage your data to be written at a later time. If there's a problem writing your data, you're fucked. Does that sound like a good design to you?"

> If that's what they need to do to get those kickass benchmarks, then it's a great design.

> "..... If you were stupid enough to totally ignore durability just to get benchmarks, I suggest you pipe your data to /dev/null. It will be very fast."

> If /dev/null is fast and web scale I will use it. Is it web scale?

> "You are kidding me, right? I was making a joke. I mean, if you're happy writing to a database that doesn't give you any idea that your data is actually written just because you want high performance numbers, why not write to /dev/null? It's fast as hell."

> Does /dev/null support sharding?

⬐ WrtCdEvrydy

> > Does /dev/null support sharding?

I watch the video every time I see it just for this shit.

⬐ lgl

Those videos (that being the most infamous) will never stop being hilarious. The node.js one is also pretty amazing:

https://www.youtube.com/watch?v=bzkRVzciAZg

This made my day. Instantly reminded me of the MongoDB webscale video https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ jpxw

“Does /dev/null support sharding?”

Lmao

I read 3-4 sources and none linked to the video[0] for some reason.

Anyone know how these are made? In many ways it reminded me of the MongoDB is Webscale[1] one - is there some kind of popular tool or framework that is used to make them?

[0] https://youtube.com/watch?v=Q5BZ09iNdvo

[1] https://youtube.com/watch?v=b2F-DItXtZs

⬐ serf

> I read 3-4 sources and none linked to the video[0] for some reason.

probably because it's straight propaganda.

seems that focus on recovery and fear-mongering is slowly shifting to the blame game and finger pointing propaganda.

Can't wait for some American equivalent to be labeled racist and immature.

That reminds me of the delightful "MongoDB is web scale" spoof video from years ago: https://www.youtube.com/watch?v=b2F-DItXtZs

Clearly Uber scale is one level up!

MongoDB is web scale. [1]

[1] https://www.youtube.com/watch?v=b2F-DItXtZs

Web-scale stuff, of course! https://www.youtube.com/watch?v=b2F-DItXtZs

Jokes aside, I had to do some analytics work at my last job, off data that was on Mongo (arguably with no good reason, other than it being 'web scale'), and it was a major pain.

Two things:

1. I embraced that Perfection is a mortal enemy of It Works. Our customers pay us for it to work now, not be perfect at some point in future.

This allowed us to save millions of dollars a month and made us a viable going concern.

2. "Mongodb is Webscale" - https://www.youtube.com/watch?v=b2F-DItXtZs

We aren't Google. We don't have Google-level global problems that we need to solve. Anything below Google level can be solved using boring technologies that just work.

This allowed us to significantly and rapidly improve customer experience.

In a non serious fashion, I make people watch this - https://www.youtube.com/watch?v=b2F-DItXtZs

Like https://www.youtube.com/watch?v=b2F-DItXtZs but for devops.

I don't think you understand. Maybe this video would help enlighten you on the subject.

https://www.youtube.com/watch?v=b2F-DItXtZs

Is it web scale? :-) Sorry couldn’t resist... https://m.youtube.com/watch?v=b2F-DItXtZs

⬐ misterman0

I actually thought that by now, in 2019, this forum would have realized that big data is real. It's a thing. Some of us are forced to deal with it. It's a hard problem that is well-suited for having discussions around on forums that are populated by intelligent people, but apparently it's not a suitable topic on this particular forum. I will certainly keep that in mind from hereon out.

⬐ kjs3

I actually thought that by now, in 2019, that people doing big data work understood that big data covered a vast number of processing use cases and saying "can it process big data" has about as much semantic meaning as "can it process blue". I will certainly keep that in mind from hereon out.

⬐ tempguy9999

Big data is mainly a marketing term to me. I've been involved in a Big Data project that wasn't because someone wanted to massage their CV with fancy tech. With current multicore machines and SSDs or cheap spinny disks you can hold and process incredible amounts of data.

So, serious question, how do you define big data? By some size threshold, or "it can't possibly be done on a single box", or something else?

And If I may, what is the BD work that you do (feel free to decline that one).

TIA

If you haven't watched it recently, this was always my favorite look at "MongoDB is web scale": https://www.youtube.com/watch?v=b2F-DItXtZs

https://m.youtube.com/watch?v=b2F-DItXtZs

That's the case for many engineering teams.

For those who haven't seen it: https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ NeoBasilisk

still one of my favorites

But MongoDB is web scale! https://www.youtube.com/watch?v=b2F-DItXtZs

I dearly wish he'd make more videos. I also loved 'MongoDB is web scale' - https://www.youtube.com/watch?v=b2F-DItXtZs

Pretty sure it was already the “hip” thing to make fun of mongodb before 2012. Who doesn’t remember this classic?

https://youtube.com/watch?v=b2F-DItXtZs

Oh, absolutely it has! Even if its query planner is... suboptimal[1]. It's even held up as an examplar of stability[2] :D

[1]: Yes, I'm nitpicking.

[2]: https://www.youtube.com/watch?v=b2F-DItXtZs

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ collyw

I had conversations that went like that.

reference is to: https://www.youtube.com/watch?v=b2F-DItXtZs

I will never stop loving this. Here is a youtube link in case anyone can't/won't run flash on that site.

https://www.youtube.com/watch?v=b2F-DItXtZs

Obligatory: MongoDB is web scale.[0]

[0]: https://youtu.be/b2F-DItXtZs

> "Most applications today run on a database technology that was introduced in the 1970s," Ittycheria said. "In the '70s, I was using a rotary phone to have a phone conversation. So people are looking for a modern, scalable and flexible platform."

It's like a weird version of the Turing test where you have to decide whether someone's speaking seriously or in jest when they talk about NoSQL.

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ Jach

I'm continuously amazed when I hear about mongodb in use. Not over say postgres, I get that there are nosql advantages, but over literally any other nosql option.

https://en.wikipedia.org/wiki/Poe%27s_law is the term you're looking for...

⬐ dplgk

I can say that CouchDB makes Mongo look like an absolute dreamboat.

⬐ hamandcheese

Which NoSQL would you prefer over mongo?

⬐ programmarchy

Couchbase has a pretty nice mobile sync system, with clients for both Android and iOS.

⬐ fermuch

The problem I found with couch is handling user permissions at database level. Except for that detail, it's one of the best databases I've ever used.

⬐ programmarchy

With the sync gateway you get pretty fine grained control. Every create or update goes through a sync function you can define in JavaScript which can deny requests. You can control which users see what by assigning the document to channels. So a user can only pull down documents if that document is on a channel they have access, too.

The issue I've found with the sync gateway is that the queries you can perform are more limited than what you can do directly on a Couchbase store e.g. joining data is difficult.

BTW, Couchbase should not be confused with CouchDB. Similar name and might have some history in common, but Couchbase is more fully featured.

⬐ beamatronic

Agree, and also I would distinguish between Couchbase Enterprise Edition versus Community Edition.

⬐ HodGreeley

Just want to note Couchbase is not just for mobile. Couchbase Server is an enterprise-class document-oriented (like Mongo) db with a dynamic query language that's a superset of SQL. (FD: I work for Couchbase).

⬐ revel

I started a company and chose couchbase. We haven't launched yet, so I've been using the community version and cobbling stuff together.

In this context: please, for the love of God, improve your documentation and dev tools. I love the underlying technology but my experience getting a basic service up and running has been pretty mixed. One of the reasons mongodb has been so successful is that there's 50 bajillion articles showing you how to hack up some crappy code that roughly does what you need it to do. Some of those articles are less bad than others, but in a pinch you can find something to at least get you on the right track. Couchbase doesn't have that deep well of experience to draw on. If you run into a problem, or experience strange behavior, it's up to you to figure out what's going on. That would be ok, but even the official documentation and standard dev tools are not good enough. To get people to adopt couchbase you need to do more to get them started.

Javascript-specific whining: it's particularly frustrating to find an official ODM like ottoman and discover that half the features don't work and that there are tons of bugs that haven't been fixed for about a year. These aren't minor bugs either; some of them stop you using headline features. Forget full text search, N1QL is mostly unusable! Check out ottoman bug #153 for details there.

⬐ milkytron

Cassandra is a pretty good alternative if you're looking for excellent write speeds and scalability.

⬐ rosser

We're using Cassandra at scale (~1M concurrent users at peak). It is, in our experience, an absolutely terrible piece of software.

The development community doesn't seem to care about fixing bugs, and when they do fix things, they reliably introduce new, often devastating defects. "Move fast and break things" is unforgivable at the persistence layer. We're stuck using an ancient, unsupported version, because it is, per our empirical testing, the least bad. But we made that choice, and for now we're stuck with it.

Please, let our pain be a lesson to you. I am totally willing to be someone else's "That's how you get ants..." on this point. Our emoji for it in Slack is a burning poo. It's that bad.

EDIT: phrasing.

⬐ gerbilly

>We're stuck using an ancient, unsupported version, because it is, per our empirical testing, the least bad. But we made that choice, and for now we're stuck with it.

Which is this least bad version, if you don't mind my asking?

⬐ rosser

2.2 causes us the least pain.

⬐ KGIII

I am not a DBA, obviously. I did have to work as one, but that was a lot of years ago. So, pardon me if this is a dumb question.

You say you're stuck with it. Can't you just change a few things, shut down for a little while, export your data, and insert it into a new DB? I know it is hard to do it with writes happening at the same time, but it seems like you could freeze it as read only, extract the data, and put it into a DB that has been prepped ahead of time and tested.

I know we did this more than once BUT it wasn't with public facing data and we were able to migrate while still working on existing data. We just couldn't add more data while doing so.

⬐ rosser

No, it is not that simple. "We just couldn't add more data while doing so" is, by itself, a deal-breaker.

⬐ munk-a

I work on more mature projects so I've never used a pure NoSQL environment but I do enjoy working with a split cache/persistence layer approach combining a RBMS (preferring PostgreSQL) behind a cache layer (preferring libmemcached).

⬐ xchaotic

Marklogic has a much better design when it comes to durability of writes and ACID in general. Disclosure: I work for Marklogic.

⬐ gilbetron

Druid works far better for our uses.

⬐ newman8r

I've been enjoying using ArangoDb

⬐ aorloff

Aerospike for low-latency small row sizes, HBase for everything else.

⬐ Jach

Literally any other. I'm unaware of any alternative so unreliable.

But I do think http://www.scylladb.com/ is great.

⬐ qohen

Literally any other. I'm unaware of any alternative so unreliable.

There was a post on LinkedIn last year, "MongoDB: The Frankenstein Monster of NoSQL Databases"[0], by the CTO of SlamData, which I think produces an analytics package for MongoDB or the like.

The piece is an interesting dive into why MongoDB is what it is (at least as of March 2016) -- and given his connection to MongoDB and its employees -- being a partner of theirs -- it's quite eye-opening (and I'm surprised with almost 200 comments in this thread that no one posted it previously):

Much like Mary Shelly’s Frankenstein monster, MongoDB’s data access layer is sewn together from ragged pieces that don’t fit together. Pieces that were never designed to fit together.

The result, depending on your point of view, is either an Enterprise-grade NoSQL database destined to supplant Oracle, or an unholy abomination of nature, deserving of an angry mob bearing torches and pitchforks.

Let me dissect this creature so you can decide for yourself.

[0] https://www.linkedin.com/pulse/mongodb-frankenstein-monster-...

⬐ qohen

Another post[0] by the same author complements the one I posted above -- this is more recent and is focused more on MongoDB's moves as a company, relating what he saw at MongoDB World 2017):

You might expect a database company to announce improved scalability, reliability, or performance. Or perhaps announce some of the countless features that users have been requesting for years, which are collecting dust in MongoDB's ever-growing issue tracker.

These are sane, logical expectations for a database company, so you'd be forgiven for being shocked at MongoDB's announcements that the company is investing massively into every product category except database technology.

Yet, to those who know MongoDB's troubled history, these announcements come as no surprise. In fact, I even predicted the launch of Stitch just over a year ago, at the last MongoDB World.

MongoDB didn't start as a database company, it's never acted like a database company, and in my opinion, it doesn't really have the DNA of a database company.

If his assessment in this next snippet is correct, one wonders about those who invested in this IPO:

MongoDB has no ecosystem. There are no analytics tools for MongoDB (except SlamData [N.B. this is the author's company] ), no backup software, no recovery software, no data integration software, no query optimization software, no data management software, nothing. Zilch.

There's just MongoDB.

In hindsight, this is an inevitable consequence of a company without database DNA trying to build and monetize a database. MongoDB couldn't figure out how to build an Oracle-sized empire on a database—partially, I'd argue, because they couldn't figure out how to build a database—yet they have to hit their sales quotas.

If you can't sell the database at scale, you end up trying to build and sell an ecosystem around the database. Slowly, bit by bit, MongoDB went after their early partners, trying to put them all out of business to drive a few million here and there.

The result is a "database company" that sells everything under the sun, including database management tools, data exploration tools, cloud hosting tools, cloud hosting services, and soon, BaaS (Bubble makes a return!) and BI software. Everything except, you know, a database.

[0] https://www.linkedin.com/pulse/mongodb-world-2017-lonely-sto...

⬐ fanbouey

MongoDB has the BI Connector https://www.mongodb.com/download-center#bi-connector

⬐ bad_user

In general NoSQL solutions are optimized for certain use-cases at the detriment of others. If you're looking for a general purpose database, then RDBMSs like PostgreSQL are your best bet.

So that said, when comparing NoSQL solutions, ending up with an apples versus oranges comparison is almost inevitable.

⬐ pishpash

NoSQL is optimized for no solutions. You do all the optimizations yourself. Only then are you allowed to speak of "optimized for certain use cases at the detriment of others."

⬐ qaq

The funny thing is PG will outperform most NoSQL solutions as NoSQL store for vast majority of use cases.

⬐ bad_user

Well, I for one think PostgreSQL is overrated.

The number one reason for why people want NoSQL is horizontal scaling and for that PostgreSQL is terrible, with all available solutions being hacks that don't work.

⬐ kod

Citus works just fine.

⬐ qaq

what % of people have workload beyond something that PG on i3.16xlarge can handle?

⬐ KallDrexx

It's also about locality of your data. Having a global infrastructure means 150-200ms minimum latency per query if you have a system in India or Singapore with a database server in a US data center region. That adds up quick.

⬐ qaq

That is orthogonal concern to what is being discussed if you run geo distributed Mongo cluster you will either have slow queries or will compromise on data integrity.

⬐ KallDrexx

It's not orthogonal because it is harder to have master-master cross data center setups in relational databases even in scenarios where eventual consistency is acceptable (such as a use case for a piece of our infrastructure we have).

Sure MongoDB may not be the best fit for it but my point is more that these are scenarios where horizontal scaling is an important consideration makes more sense for some nosql solutions than for sql solutions. It's not just about single box performance.

⬐ hendzen

people who don't want to get woken up if Amazon decides to terminate said i3.16xlarge...

⬐ jlarocco

Missed the point completely...

⬐ acdha

I think you should reconsider that dismissal: everyone needs n > 1 instances for reliability while increasingly few tasks require more {CPU,RAM,IOPs} than a single server can provide. That means that a growing percentage of problems will require clustering for reliability more than performance, and that favors the easiest to manage since in most cases every option will be fast enough.

⬐ qaq

Not a fan of AWS or cloud in general but accepted that this the direction the world is going for now. The AWS instance type was purely for illustration purposes.

⬐ gnaritas

Horizontal scaling is often just as much about redundancy as it is scaling; no one server solution is a valid answer to a production application.

⬐ gaius

I always think this when people talk about scaling. You can buy off the shelf now boxes with 48 or 96 cores, 1-2T of RAM, internal bays for 10s of T of SSD or connect it to an AFA and get 100s of T. This is not even an exotic custom build system, just a commodity, and has been for years. Running a recent version of a conventional database on a box like this goes a very, very long way, with very little hassle because you don't even need to think about "sharding", and you can always add a hotstandby for offloading reads, or for redundancy in another DC, or whatever. Systems like this can quite happily bottleneck on the network before the database starts breaking a sweat.

Remember, sharding isn't scaling the database. Sharding is admitting your database can't scale so you're offloading the problem to another layer.

⬐ qaq

You can have 224 cores and 12TB RAM in a commodity Supermicro box.

⬐ gaius

Indeed. You would need one helluva Postgres workload to overload that. Not one in a million people genuinely have that requirement.

⬐ qaq

People tend to copycat what Google and the like are doing without giving much thought to the fact that in many cases the solutions they adapt are simply due to the fact that they don't have an alternative and not because it's "better".

⬐ fermuch

I second this. Mongo clusters are easy to manage, and the aggregation pipeline or MapReduce engines do wonders on a sharded cluster with tons of data. That's the selling point of mongo for me.

⬐ orf

> with tons of data

Define tons of data. I'll bet a beer that what you say can fit on a single postgres instance and even be small enough to run fine on a cheapish developer laptop.

⬐ nicholassmith

That's interesting to me as I've been looking at Mongo aggregations for a solution to a problem but couldn't find any research on the performance of them.

⬐ ComputerGuru

It doesn't matter how easy to is to scale if it doesn't do it right. You can win all the internet points when it comes to speed and ease-of-use but it doesn't matter if it comes at the cost of data integrity.

⬐ Diederich

> It doesn't matter how easy to is to scale if it doesn't do it right.

I would refer you to an earlier comment:

> In general NoSQL solutions are optimized for certain use-cases at the detriment of others

Your definition of 'right' is absolutely not the only definition of 'right'.

More to the point: there are a lot of problem spaces where the data integrity provided by MongoDB are more than sufficient.

⬐ ComputerGuru

Sorry, I meant "correct" and not "right," as the former is objective and the latter can be (though often isn't) subjective. MongoDB is not correct; in fact, it is provably incorrect. If your needs to don't require correctness, then by all means.

⬐ hiram112

I don't disagree with you for teams who know which technology is right and which isn't, for their specific use cases.

But more common than not, inexperienced devs are using Mongo and similar to store relational data, simply because they were sold the 'MEAN' stack and didn't realize that, while it's easy to get a quick prototype running, a year or two later you eventually need things like transactions and joins, and NoSQL is absolutely the wrong technology most of the time.

⬐ mrfusion

Most vehicles are rolling around on technology that was invented 10000 years ago.

(Is that the same idea?)

⬐ semanticfact

That's interesting.

⬐ blibble

was it not round?

⬐ JumpCrisscross

I guess it didn’t have a hollow core...

⬐ tdhz77

Firestone recalled those tires in 1999.

⬐ marcosdumay

Well, the SGDBs of the 70's were nothing like the ones around today.

⬐ bdcravens

Meanwhile almost all of those applications are written in languages 20+ years old (Ruby, Java, PHP, Javascript.. Python is almost 30)

⬐ gremlinsinc

while the languages may be old...they're constantly updated... php 7 is lightyears ahead of 5 and 4 was just a joke.. Python 3 same.. just because somethingn is new/old doesn't make it better/worst I mean - many devs swear by Vim which is like 40 years old.. is sublime text better because it's newer?

(I use sublime..but I admire those who've jumped into vim for the productivity boost that brings).

⬐ callalex

That’s exactly the point though. It’s not like we’re plopping a floppy from Oracle into every machine running a modern SQL db.

⬐ awj

> "Most applications today run on a database technology that was introduced in the 1970s. We built ours on ideas that were tried and discarded in favor of that technology."

Not entirely true, but nuance often precludes persuasion.

Many of the ideas behind NoSQL databases can be very valuable, given the right context, but there are a lot of good reasons relational databases have been the de-facto default over the last four decades.

⬐ protomyth

Worse, the lack of awareness in how a phone system works by picking the phone type as opposed to the actual network is a perfect superficial understanding of technology that dovetails into the video way too nicely.

⬐ quickthrower2

Yes dumb argument. That YouTube video was the first thing I thought of when I read the sentence.

Let's scrap everything that was invented in the 70's or before.

⬐ KGIII

I saw that.

⬐ cortesoft

Sounds closer to Poe's Law

https://en.wikipedia.org/wiki/Poe%27s_law

⬐ blauditore

I know there's quite some aversion to NoSQL around here, and generally I don't care much as I'm seldomly dealing directly with databases.

But recently, I've been exposed to a fairly big and complex SQL one with several references between entities and lots, lots of X_has_Y tables. This makes me think that with growing complexity (which seems to be a general trend), NoSQL databases seem more practical at some point, or at least something less rigid than classic relational ones. I'm not saying SQL is obsolete, but it seems like its domain of usefulness is shrinking.

⬐ parasubvert

The point is that the relational model is a fully general purpose mathematical model for managing data with integrity and with ad hock access.

Most other models are for high performance of specific access paths and punt integrity management to code, which is a throwback to the 70s. They’re glorified file systems and data structure caches. Mongo has a reputation for losing your data.

There are cases where scale and availability kills you and you need something like Cassandra or whatnot. But there are few as flexible and general as Oracle or Postgres.

⬐ andrewmutz

I don't think the HN community is averse to NoSQL solutions, but is averse to MongoDB in particular.

⬐ erik14th

Everyone here seems to love Postgres and hate on Mongo, I have no technical knowledge to compare the two, so IMO a lot of that love and hate is more towards the "project attitude", MongoDB is a suit, a sellout, with bullshit marketing and all Postgres is like some roots hippie, that cares a lot more about technical values and neglects marketing.

⬐ paulmd

MongoDB accrued a lot of bad-will due to some extremely questionable defaults, which remain defaults to this day. There's no question that you can write a fast database when there's no guarantee that data ever hits the disk, but developers tend not to like it when a database accepts their write and then silently loses data. It's also great for toy problems and 15-minute-demos... but then you inevitably run into its limitations and end up re-implementing a database in your app.

Even at its best, there is essentially no reason to choose MongoDB over Postgres with JSONB-type columns. They are essentially the same data model but Postgres gives you better guarantees of data consistency, plus a forward migration path to relational data when the day inevitably arrives when you need to model relationships between entities.

At this point Postgres is where most open-source RDBMS development work is concentrated. It's not only a solid codebase, it's piling up features pretty quickly and there are relatively few niches it doesn't fill at least adequately. All of these niches are covered by some commercial products built on top of Postgres (eg EnterpriseDB or CitusDB). It's pretty much a one-stop shop for application development. You can use it for everything from GIS to machine learning [0] pretty efficiently, and it pretty much will just do the right thing without you watching.

NoSQL really fits best around the margins, like as an auxiliary system for analytics. There is really almost no use-case where "user inputs data and we lose it" is an acceptable application behavior, so consistency is a business requirement for your master database whether you realize it or not. And consistency across a distributed system is hard so it almost always makes sense to sidestep clustering until the last possible moment. Buying more machine is cheap, replication/failover is a lot easier than consistency between distributed masters, and if you are really up against the wall there are those commercial products that can do this with Postgres.

If you want to make an analogy... Oracle is the suit, Postgres is the hardworking small business that is slowly but surely eating up Oracle's lunch, and MongoDB is a trustafarian with a hot-dog detector app. And that's why there's a lot of resentment towards MongoDB.

[0]: The 9.x series and 10.0 release have been absolutely jam-packed with new features, it's absurd how fast development is moving at the moment. One of my favorites... indexed cube queries. A cube is a data-cube type, an N-dimensional cube of data. One feature of this is distance queries, which have obvious applications in pattern recognition tasks (eg k-nearest-neighbor). One of the features in 9.6 is index functionality for these, so you can now do indexed KNN searches on your data...

https://www.depesz.com/2016/01/10/waiting-for-9-6-cube-exten...

⬐ VintageCool

I'm not sure if those were questionable defaults, or questionable design decisions which were the only option at the time, and now persist as questionable defaults.

I'm pretty sure that mmap was the only storage engine available for MongoDB for most of the hype period.

⬐ porker

> NoSQL really fits best around the margins, like as an auxiliary system for analytics.

I'd say it also fits well in two niches: document datastores (so long as there's some JOIN support, via referencing nested documents vs direct nesting) and graph stores.

I remember 10+ years ago working on storing nested sets in the RDBMS and it wasn't pretty. And the RDBMS schema for Magento 1, with key-value tables all over the place which NoSQL would have removed the need for.

⬐ orf

> so long as there's some JOIN support, via referencing nested documents vs direct nesting)

Which has its own problems. PG does this just fine, with a full battle-tested relational system to back it (and you) up.

> with key-value tables all over the place which NoSQL would have removed the need for.

Product X having a stupid schema is not a good basis for an argument for or against a particular product.

⬐ porker

> Product X having a stupid schema is not a good basis for an argument for or against a particular product.

What other way could Magento have implemented user-defined columns at the time, using a RDBMS? In 2009 when MongoDB was released, JSON columnstores were something to dream of and the alternative was storing serialised data in a BLOB field. That "stupid schema" did not have an alternative I can think of, except NoSQL.

⬐ paulmd

Can you define what "nested sets" means more specifically?

Postgres supports hierarchical/nested structures using the "ltree" column type. There is nothing stopping you from defining a primary key of (eg) "set1.set10.set100". There is also support for recursive views/etc to operate on these kinds of sets.

Again, if you have some kind of "sparse" column, it can make sense to put that into a JSONB column. This is effectively the same thing as attaching an unstructured document to a record for this use-case.

⬐ porker

Sure: http://mikehillyer.com/articles/managing-hierarchical-data-i...

Joe Celko popularised them. I've been unable to find when they were first introduced; but a search of my source code archive points to having written one ~14 years ago.

⬐ acdha

I think most of it was a reaction to the massive marketing push Mongo made. There were so many years of people hyping it as the answer to all of your data storage needs, followed by accounts of users hitting bugs or reimplementing most of a SQL database in code, and the promised performance benefits either didn't materialize or hadn't been necessary in the first place (“web scale” turned out to mean only dozens to hundreds of requests per second for many apps).

Meanwhile, Postgres was quietly plugging away adding new features and continuing to deliver solid performance for a wider range of workloads, including better performance on JSON document storage.

⬐ autokad

i got downvoted in another thread for saying nosql has a lot of advantages in reducing work like not having to worry about carefully creating indexes when joining billion row tables. someone replied you dont need indexes. i replied I dont know how you plan on running joins on billion row tables in postgress without indexes. got downvoted again.

I bet they are still waiting for that join....

⬐ acdha

From the sounds of it, you were downvoted for not recognizing this as a textbook scenario with decades of practice and engine improvements devoted to allowing you to balance query performance against your server budget.

Interestingly enough, many of those techniques are common in the NoSQL world as well — a billion records is enough to require thinking about data flow anywhere — but the difference is that you have to deploy them more frequently.

⬐ orf

If you're dealing with billion row tables and using mongodb you are way out of your league. Good luck.

⬐ autokad

mongodb is not the only nosql db :) I use hive/hbase

⬐ parasubvert

Whether you need indexes really depends on what you’re trying to do. Full table scans are more efficient if you are doing analytics on most of those rows in a time series. If you need 1 row in a billion, yes, you need an index.

⬐ autokad

if your doing a join on two size N tables, your O(n) cost for each record is N, thus you are doing N^2 lookups. basically reading the entire DB into memory each record.

⬐ parasubvert

No. Worst case a nested join of two tables M and N would be O(MN). But most real databases would have a merge or hash join which could bring this down to O(M+N) if the tables are already stored in a sorted indexed order. It is also rare the cardinality of these databases will be similar in an analytical query - usually you have one time series “fact table” and a bunch of dimensions to enrich that data.

If I’m doing analytics on a time series this gets even better with partition pruning and hash joins or bitmap indexes. And if I have a columnar database, that blows up this whole complexity argument.

My point is that, layout and indexes should never be assumed to be one size fits all. Keep in mind if I’m doing analytics I want to bring my time series data into memory at least as a stream, as I need to calculate / filter / transform the records. Not everything is about rendering a page on a website.

⬐ matwood

> less rigid than classic relational ones

Less rigid than what? The schema is going to exist somewhere...

⬐ nwatson

So you gonna do all those JOINs in code???

⬐ HodGreeley

Part of the point is you don't need so many joins.

⬐ mi100hael

I'd argue the opposite is true. NoSQL requires you to be more intentional with your schema design since you can only query keys. As a database grows with complexity, you can find yourself in a world of pain with NoSQL if you all of a sudden need to query something in a different manner. At least with SQL you have more options with WHERE clauses & indexes even if performance might not be top notch.

⬐ HodGreeley

Not sure of others, but not true with Couchbase. You have queries just as flexible as with SQL.

⬐ philliphaydon

One of the reasons I love PostgreSQLs jsonb type is you get the power of relational database and power of (nosql?) Document database at the same time. i haven’t run into any query I couldn’t write against a jsonb doc. Granted dot notation on json would be nicer than ‘prop’ > ‘prop’ > ‘prop’, but I don’t feel limited.

⬐ SilasX

"Most spaceflight trajectories are based on physics that was introduced in the 1680s. In the 1680s, we were using slide rules to multiply numbers."

⬐ baybal2

I wish I can upvote you 10 times

⬐ Florin_Andrei

Actually they were using logarithmic tables then.

⬐ tlb

Slide rules date to the 1620s. https://en.wikipedia.org/wiki/Slide_rule#History

⬐ Florin_Andrei

TIL. Thanks!

I like MongoDB! It's web-scale:

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ jinqueeny

Haha, I have watched this video for 10+ times! It's hilarious. Couldn't help it...

⬐ VirtualAirwaves

This is a great video, and accurately sums up MongoDB and NoSQL. I can't imagine why they need to raise money, why they spend so much money as it is, and why they think there's a big growth business in NoSQL.

⬐ SilasX

I miss XtraNormal, the videos have a special deadpan quality to them.

From what I've observed, the "crowd" is easily seduced by performance above all other concerns - correctness, security, science, etc. NoSQL was invented and gained popularity b/c it originally was easier to scale via sharding, and that was seductive enough [1] to give it industry momentum. While performance is a feature, it seemed many folks originally advocating NoSQL did not understand or appreciate the mathematical foundations of the relational model. A decade of experience seems to be driving zeitgeist back to scientific fundamentals.

[1]:https://www.youtube.com/watch?v=b2F-DItXtZs

Actually, MongoDB is web scale.

https://www.youtube.com/watch?v=b2F-DItXtZs

https://www.youtube.com/watch?v=b2F-DItXtZs

Did you read the article past the benchmark numbers? That's more-or-less exactly the point the author made by the end. To wit:

> The majority of web applications handle far less than 1000 requests per second. I’d go as far as to say that most web application developers are employed by a company whose entire webapp does far less than 1000 requests/second. Most of them do less than 1000 requests/minute.

> When your application is big enough to be doing 20,000 RPM, you will have anywhere from a half-dozen to even fifty engineers, depending on your application’s domain. A single software engineer costs a company at least $10,000/month in employee benefits and salary. So we’re choosing our frameworks based on saving one-third of an engineer per month? And if that framework caused your development cycles to slow down by even one third of a mythical man-month, you’ve increased your costs, not decreased them. Choosing a web framework based on server costs is clearly a sucker’s game.

> What does web-scale even mean? It's an old-ish meme, from this video making fun of MongoDB. [1]

1: https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ flavio81

The majority of web applications handle far less than 1000 requests per second.

To be fair, two points:

a. The benchmarks do rather easy stuff like fetching rows from SQL and then serving them. No heavier stuff (like compressing a file, etc).

b. On the provided benchmark, Ruby on Rails with Unicorn barely achieves 2000 request/sec.

It's web scale: https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ pdkl95

"If you're willing to suffer complex code for performance, why not just write an nginx module in C?"

"node.js is is the most bad-ass rock star tech to come out since Ruby On Rails"

https://www.youtube.com/watch?v=bzkRVzciAZg

...

"What was that last part again? I could have sworn you just said Lisp"

(Yes, performance and scalability are complicated. Different solutions will be appropriate for different situations.)

Come on, we all know why: https://www.youtube.com/watch?v=b2F-DItXtZs&list=PLmGAf2A36C...

⬐ None

None

⬐ reallydattrue

That made me lol!

This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex.

No wonder the whole javascript stack is a complete and utter mess!

That, i suppose, was meant to be a joke. see https://www.youtube.com/watch?v=b2F-DItXtZs and just replace mysql with pg.

⬐ combatentropy

Yes, it was a reference to that. My comment is currently sitting at -2. I'm still getting the hang of Hacker News' sense of humor. On Slashdot, I would be at +5. Ho hum.

Anyway, just to make my intentions clear, I love Postgres, have used it for over a decade, and have yet to use MongoDB or any other NoSQL database.

⬐ grzm

In general, strive for substantive and constructive comments on HN. If you've got that, a little humor added in can be appreciated. Comments that are submitted only for humor value (which you knew yours was, given your parenthetical addendum) are likely to be less appreciated on HN than on other sites.

Here's a recent thread where this has been discussed:

https://news.ycombinator.com/item?id=13760333

There are likely others that describe it better, but I don't have them at hand.

Edit to add: Of course there are. Here's some comments by 'dang on the topic:

https://hn.algolia.com/?query=author:dang%20humor&sort=byPop...

⬐ combatentropy

So what I want to know is how the members of Hacker News act as one. Presumably these are the same people who appreciate humor on other sites, upvote it, and participate in it. But on Hacker News they somehow all know to downvote it. It's eerie.

⬐ rimliu

It's rather sad what set of behaviour is considered "professional".

⬐ grzm

I'm not sure I follow. People often behave differently depending on the context (e.g., work, home, school, out with friends), so acting differently on different sites doesn't seem very surprising. Also, different sites have different, though perhaps overlapping, populations, so the "average" behavior or culture is going to be different.

One example (mentioned in the HN guidelines) is Reddit. There are people who frequent both HN and Reddit, yet it's clear that members on HN—even those who use Reddit as well—don't want HN and Reddit to be the same. As such, they behave differently on each site. That's not to say one is better than any other: they're just different.

And HN members don't act as one—just as they don't act as one on any other site. If they did, your comment would have been downvoted to the point that it was flagged dead (given it's likely been seen by hundreds, if not thousands) or not flagged at all. Given it's current shade of gray, I suspect you've received only a few downvotes. (Edit to add: I see you did say it is at -2.)

Does that make sense? Or am I misreading you?

Anyway, this is quite off-topic now. I posted these in the hopes of providing a bit more insight into the HN community. I hope they've been more helpful than frustrating.

⬐ combatentropy

Thank you. No, you're not misreading me. I guess like you say, it could be worse than -2. (Although now it's at -3!)

I don't frequent Reddit. It sounds like it's so jocular that people come here to get away from it all.

I think the people downvoting this haven't seen [1] :)

[1] https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ bitwize

I think it's just that Hackernews doesn't find it funny.

You do realize that that wasn't intended to be a serious argument about the points of Ada vs. Rust... right? Have you seen "MongoDB is web scale" (https://youtu.be/b2F-DItXtZs)?

Obligatory "Mongo DB Is Web Scale" https://www.youtube.com/watch?v=b2F-DItXtZs

Everything needs to be web-scale:

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ mdekkers

That's hilarious

⬐ paulddraper

/dev/null is web-scale

http://engineering.wayfair.com/2013/04/devnull-vs-mongodb-be...

Sure, but I wouldn't exactly call them mature yet. I think MySQL is still much easier to trust than RocksDB. (I don't actually know much about RocksDB, but I remember that LevelDB had a bunch of data corruption issues. Hopefully RocksDB doesn't.)

To quote the famous video[1]: "Relational databases have been around since the fucking 70s and are some of the most mature technology you can find."

1: https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ ngaut

RocksDB is widely used in many of projects for years, i think it's quite mature. Not just TiDB and CockroachDB, MyRocks and MongoRocks are using RocksDB as storage engine too. Also there are plenty of other projects are using RocksDB. If you are worry about the adding new feature of RocksDB, yes, you are right, but all of the storage engine are changing rapidly(even InnoDB which is used by MySQL as default engine), Anyway, happy hacking :)

I'm not clear on why people would be exposing MongoDB on a public address?

This habit of database-like software (does Redis still not offer authentication?) treating authentication/authorization as a second-class citizen is odd.

I guess it has proven to be webscale though [1].

[1] https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ None

None

⬐ bdcravens

Redis has offered simple authentication for some time: https://redis.io/commands/auth

Redis generally isn't put forth a first-class member of a popular stack and featured in getting started tutorials in the same way that MongoDB has been.

Perfect dialog here. For anyone who doesn't "get this"... watch this video: https://www.youtube.com/watch?v=b2F-DItXtZs

https://www.youtube.com/watch?v=b2F-DItXtZs

Reminds me of https://www.youtube.com/watch?v=b2F-DItXtZs

"Shards are the secret ingredient in the webscale sauce, they just work.." :) https://www.youtube.com/watch?v=b2F-DItXtZs

Seriously. I was looking at DB usage statistics recently and was appalled MongoDB is still so popular. I thought it was done, nail in the coffin, when https://www.youtube.com/watch?v=b2F-DItXtZs came out 6 years ago, I haven't followed it much since then apart from the occasional post like this whose content is just "you thought it was bad already? haha it's worse."

And on that theme, then get ready for when your boss gets to the part about how NoSQL gives you:

"Rapid development. If you’re developing within two-week Agile sprints, cranking out quick iterations, or needing to make frequent updates to the data structure without a lot of downtime between versions, a relational database will slow you down."

Reminds me of this classic https://www.youtube.com/watch?v=b2F-DItXtZs

Mongodb is webscale https://m.youtube.com/watch?v=b2F-DItXtZs

> I'm not sure where this mantra that pgsql is slow comes from.

MongoDB factor is probably why from a few years back.

Basically MongoDB had defaults that threw data over the wall and didn't send acknowledgement back to the client, then relied on dirty pages flushing to disk in the background. Well it turns out if you don't actually write your data to disk, you get really nice benchmark results. ( Therefore the MongoDB is a webscale phenomenon with the obligatory famous video: https://www.youtube.com/watch?v=b2F-DItXtZs )

So a lot of people who didn't know how things work were impressed by sending data to /dev/null at high speed and saw postgres as slow and uncool. It was of course debunked since then but the vibe kind of hung around.

⬐ jfb

It's older than that. Postgres used to be compared directly with MySQL, back when ISAM was MySQL's "storage" engine. It was definitely slower -- of course it was doing more -- but it's really never been slow. But memes are difficult to kill.

I found reading it rather cathartic. It mirrors the vitriol that my C++ teachers had towards C, and it was a counterweight to the irrational exuberance they expressed towards C++ features like iostreams and inheritance.

I mean, I actually like C++. It's my language of choice. But around the early 2000s, the smugness was overwhelming. To me, this is the 'mogodb is webscale'[1] of yesteryear.

[1] https://youtu.be/b2F-DItXtZs

⬐ dunkelheit

Good point. I often hear about some perfectly valid piece of C++ code that it is "C-style". Like, so what?

Reading some eloquent critique of your favorite technology can encourage healthy attitude towards it and prevent turning it into holy cow.

"Learn X in Y minutes" is an awesome concept! Let me know if you ever decide to run with it. I guess I don't fit the CoffeeScript is a hipster language profile very well but I still enjoy writing code in it ;)

I basically agree with everything you've said (in both posts) ... you can be successful while wasting cycles. And if you're working on a low-volume and/or internal only application, you'll probably never face the limits of a modern server.

If you need to operate "at web scale" [1], or run into an uncommon (or common) bug, you'll need to know more about the frameworks and systems you're code relies on (e.g. MongoDB configuration for systems over 2GB [2]). Blog posts like the one referenced are completely unfair to those that developed MongoDB - read the manual and understand how MongoDB works OR use it at your own risk.

So I'll switch arguments and help you make your point. We have an application written in Oracle's Application Express - while we have extensive expertise in Oracle's database software, we have this one system which was completed for expediency's sake. It's kind of horrific but (mostly) works at the scale required. It would be financially foolish to dig into more deeply into ApEx for this one dead-end application. Everyone is happy.

[1] https://www.youtube.com/watch?v=b2F-DItXtZs (audio NSFW)

[2] http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...

⬐ logicallee

Thanks for the reply! (I didn't actually build those tutorials, i.e. learn x in y is another person's site, like I said I only spent a couple of hours on the concept of a site like this and the current tutorials are external. I did add the time analysis.)

I like your final example - and remember, you guys are Oracle experts: you're the most qualified people on the planet to learn ApEx properly from scratch, even though you haven't.

Now switch gears and imagine a college student who just has an idea for some cool project, but barely codes in any language. This describes the computing needs of 3 billion people. They're not qualified to quickly become experts and engineers at anything. But they still have a computer in front of them that does a trillion operations every few minutes. The gulf between using that to surf facebook or building... anything at all, even very poorly, is immense. (Like git that you can learn in 15 seconds, versus manually copying and renaming files for version control.) Thanks for the encouragement.

Author also made [1] and [2], FYI.

[1] https://www.youtube.com/watch?v=rRbY3TMUcgQ [2] https://www.youtube.com/watch?v=b2F-DItXtZs

Only made it about a third of the way through but it seemed like latter day Xtra Normal MongoDB/web scale shtick[1].

[1] https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ latchkey

Yea, too bad xtranormal is no longer around.

/dev/null is pretty darn fast, too. https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ thinkerer

I have to qualify that both are good for different purposes. But do go ahead and share your experiences with your stack used and performance.

The name of this makes me think of that MongoDB is web scale video[1]'s comment, 'Is /dev/null web scale?'.

[1] https://www.youtube.com/watch?v=b2F-DItXtZs

Obvious sarcasm (https://www.youtube.com/watch?v=b2F-DItXtZs)

Every time I see the word "Webscale" I think of this animation:

http://www.youtube.com/watch?v=b2F-DItXtZs

⬐ coldtea

The name was SPECIFICALLY picked to refer to that animation, it's an inside joke kind of thing.

And, after you've watched the original, check out "Erlang The Movie II: The Sequel", by Garrett Smith (aka gar1t): http://www.youtube.com/watch?v=rRbY3TMUcgQ

(You are likely familiar with one of gar1t's earlier videos, the classic "Mongo DB is Web Scale":

http://www.youtube.com/watch?v=b2F-DItXtZs

And he's got more on his YouTube channel: http://www.youtube.com/user/gar1t )

I don't like MySQL either. But it's probably the perfect choice. Or perhaps you think something webscale[1] is required?

[1] http://www.youtube.com/watch?v=b2F-DItXtZs

⬐ takluyver

If you're running an instance for yourself, or only a few users, SQLite would seem like the obvious choice. That avoids the hassle of running a separate database server.

⬐ mrmondo

I'm a fan of relational databases, just not MySQL. I've been administering MySQL, PostgreSQL and Oracle (not to mention a few others I'm too embarrassed to mention) for some time now and MySQL is a right pain in the ass. I'd rather see it use PostgreSQL. Love the video by the way!

http://grimoire.ca/mysql/choose-something-else

Reminds me of https://www.youtube.com/watch?v=b2F-DItXtZs

I'm guessing it's inspired by the video "MongoDB is Web Scale"

https://www.youtube.com/watch?v=b2F-DItXtZs

⬐ donretag

If /dev/null is fast in web scale I will use it. Is it web scale? Does /dev/null support sharding?

⬐ einhverfr

Yes, and it is much faster than MongoDB as there are no write locking issues.

⬐ aroman

Wow I can't believe that video has gone viral — I remember showing it around to a few friends 2 years ago or so when it only had a few thousand views. Ironically, MongoDB has since become my go-to DB of choice...

⬐ kansface

Have you read http://aphyr.com/posts/284-call-me-maybe-mongodb ?

Cool, time to make MongoDb web scale! :-)

http://www.youtube.com/watch?v=b2F-DItXtZs

⬐ kailuowang

That video is a bit unfair. I went to the MongoDB conference in NYC this year. None of the 10gen engineers made such silver bullet claims. They strengthened again and again that MongoDB is with significant limitations and scaling it requires a lot of effort and creativity from the client developer side.

That being said, I do wish they can use this investment to do more feature development than marketing.

⬐ junto

I know. I added the smiley face because I meant it tongue in cheek.

⬐ untog

That video is three years old, though.

⬐ ameoba

...but you can be sure that it's going to be posted every time MongoDB comes up. Somehow, people will vote it up as if it's still funny or adds something to the discussion.

⬐ creamyhorror

I like this bug report about pymongo (not the MongoDB core itself):

https://jira.mongodb.org/browse/PYTHON-532

-

Step 1. Use Mongo as WEB SCALE DOCUMENT STORE OF CHOICE LOL

...

Step 7. DISCOVER PYMONGO DOES NOT CHECK RETURN VALUES IN MULTIPLE PLACES. DISCOVER ORIGINAL AUTHOR SHOULD NOT BE ALLOWED NEAR COMPUTER

⬐ rpedela

One of the best bug reports I have ever read!

⬐ threeseed

Only if you're a 5 year old. It's immature, unhelpful and generally pretty pathetic.

⬐ _sh

Lighten up. The submitter identified exactly the line and commit that enabled crashing all mongo deployments with a 16-byte post (I presume, from the report). Considering the circumstances, I congratulate the submitter on being so whimsical.

  if (strcmp(buffer + position + 5, "$ref") == 0) { / DBRef */

Seriously?

MongoDB is web-scale

Edit: http://www.youtube.com/watch?v=b2F-DItXtZs

Well since his username is a slight variant on the parent's username, he's probably trolling and making reference to http://www.youtube.com/watch?v=b2F-DItXtZs

This is the famous video in which the benefits of the /dev/null storage engine are described in detail. Extremely enlightening!

http://www.youtube.com/watch?v=b2F-DItXtZs

tongue-in-cheek: http://www.youtube.com/watch?v=b2F-DItXtZs

Just pipe your data /dev/null to get that web-scale

http://www.youtube.com/watch?v=b2F-DItXtZs

But.. but.. MongoDB is web-scale: http://www.youtube.com/watch?v=b2F-DItXtZs

(Note how the video raises some of the same concerns as the blog post)

⬐ lucisferre

Because nothing adds to the cogency of one's argument like cartoon animals...

But wouldn't you rather pin your mission-critical data to a table and give it a night it will never forget? :-D

http://www.youtube.com/watch?v=b2F-DItXtZs

⬐ Luyt

And with transcript: http://www.mongodb-is-web-scale.com/

You know, Mongo DB is Web Scale: http://www.youtube.com/watch?v=b2F-DItXtZs

http://www.youtube.com/watch?v=b2F-DItXtZs

⬐ Luyt

A transcript can be found at http://mongodb-is-web-scale.com/

"Shards are the secret ingredient in the web scale sauce. They just work."