Hacker News Comments on
Michael Stapelberg: Why I wrote my own rsync
media.ccc.de
·
Youtube
·
123
HN points
·
1
HN comments
- This course is unranked · view top recommended courses
Hacker News Stories and Comments
All the comments and stories posted to Hacker News that reference this video.This is also available as a video, "Why I wrote my own rsync":
⬐ marciFor those who want to avoid youtube:⬐ kistanodCasually having 25 GBps internet speed... Remind me why I live in the country that has the biggest tech companies in the world + most venture capital, yet my ISP is currently offering 1 GBps max :) (I think I know the answer, yet this is painful to think about)⬐ bradknowles⬐ toolsliveSame country, but restricted to DSL speeds because the sole vendor who provides near-gigabit service in this neighborhood is Spectrum, whose service quality is so low that AT&T DSL is still a better option.Go two miles in any direction from our neighborhood, and you have multiple providers with gigabit or near gigabit speeds, including AT&T.
Regulatory capture indeed.
⬐ m463the answer is regulatory capture.⬐ mschuster91⬐ quickthrower2Usually I'd agree with you, but Switzerland has the advantage that 2/3rds of its population live in extreme density for European standards (426 ppl/km², [1]). The denser the population, the more cost-efficient the buildout of any infrastructure is.Additionally, Switzerland is all but drowning in money (in fact, they have to fight to keep the Swiss Franc affordable [2]) because everyone and their dog who can afford it parks their dark money there. That influx of wealth is reflected in the services and wages that the citizens get.
[1] https://www.avenir-suisse.ch/wie-dicht-ist-die-schweiz-besie...
[2] https://www.handelsblatt.com/finanzen/geldpolitik/schweizer-...
⬐ Nimitz14These are really half-baked thoughts.1. As you may know half of Switzerland is filled with mountains, so building infrastructure there is actually not that trivial (no, the high speeds are not just in the big cities, they're everywhere).
2. Dark money in Switzerland hasn't been a thing for a while. The wealth comes from sensible policies like a sovereign currency, low taxes and reasonable levels of regulation and bureaucracy. All that money also makes everything expensive, so the extra money kind of cancels out when it comes to doing stuff inside Switzerland.
⬐ kybernetikosAlthough the density thing is true, the entire population of the country of Switzerland is less than that of a large metropolitan area (e.g. London) so while it might not be possible to have Swiss level infrastructure in rural US areas it should be perfectly possible in major cities.⬐ vitusPut more concretely: the population of NYC proper (ignoring the broader metro) and Switzerland are pretty comparable (both around 8-9m people), but NYC is about 50x more dense (300 mi² vs 16000).⬐ Brian_K_White⬐ protomythBut the Swiss poulation is not evenly spread over it's land area.⬐ vitusWhile that's true, the two densest cities in Switzerland are Geneva (~12k / km^2) and Basel (~7k / km^2).NYC sits at about 11k / km^2. Unless 10Gbps internet and up in Switzerland is only available in Geneva, there's clearly more than just population density at play.
The funny part is that parts of the rural US have better internet speeds than the cities. Rural Electric Cooperatives do like their fiber. Carrington ND has a billboard proclaiming the availability of 10Gbe fiber to the home.Lucky to get 25Mbps max here :-(. And no, not rural. In a metropolitan area.⬐ JaggerJo⬐ akira2501germany?> that has the biggest tech companies in the world + most venture capital, yet my ISP is currently offering 1 GBps maxThese two facts are completely related. A wide array of competition is to be desired, not "biggest in the world."
⬐ nix23>A wide array of competition is to be desiredSwisscom the owner of the fiber-cable is mostly state owned, they have to rent it out for a fair price, important infrastructure should always be partially state controlled.
why not use unison as a starting point?⬐ binkHNWhile I enjoyed some of this video, I felt it was, at least, 20 minutes too long. It covered too much of his other projects, and it's title, perhaps, should be charged to note that.⬐ abdusco⬐ donpottHave to agree. His motivation was interesting but also could have been left out.⬐ heleninboodler⬐ NoneIn fairness, the title of the video is "Why I wrote my own rsync", not what's reported here.None⬐ hwdI have to say I have enjoyed every minute of the video. Just watching what Michael Stapelberg has achieved leaves me awestruck. With his subtle humor I find it really a joy to watch.Huh... You can achieve this trivially by signing up to Dropbox /s⬐ jamessbThe slides: https://docs.google.com/presentation/d/1XqWF7h8ScimDFbQgLcXP...The speaker Michael Stapelberg had a few blog posts about his home 25 Gbit/s fiber connection that have been discussed here previously
"My upgrade to 25 Gbit/s Fiber To The Home": https://news.ycombinator.com/item?id=31134534
"25 Gigabit Linux internet router PC build": https://news.ycombinator.com/item?id=27792738
⬐ e40I read through the slides and I still don't know why he wrote his own rsync.⬐ zamadatixIt's a bit easier to follow in the video since he talks from the slides more than presents them but as built up earlier in the deck (11-15) the router7 system us is written in pure Go, i.e. there is no other userspace, which comes with several neat advantages over a traditional system but he wanted to merge his routing box and server box as part of the upgrade needed to support 25G. Later (20-21) he talks about how all of the other software already had popular Go alternatives that could be easily dropped in except rsync and how there was a kludge way to get rsync on the box but it broke the advantages of doing the router7 Go only approach to do so and as a result he set out to implement enough of rsync to meet his needs.The journey and results are much more interesting than the why though. Router7 has some interesting diagnostic capabilities and workarounds specific to issues he's had from the ISP. Also the implementation of rsync itself is interesting in that it outperformed canonical rsync of the time doing the same task without "better performance" being a goal while writing it. There are limitations with this implementation though, it was only made to meet the particular use case not be a general rsync replacement and it doesn't have the latest protocol/hashing changes rsync has which allowed rsync to retake the performance lead since.
The short sum up would be "because he had an open router userspace written in pure Go and rsync didn't have a suitable pure Go implementation at the time". No taking over the world or anything like that just good ol' hacking for personal use.
⬐ porjoVideo is about this rsync implementation in Go: https://github.com/gokrazy/rsync