Wednesday, January 28, 2009

The cutting edge of VM design

The Java Virtual Machine generally stands out as the state-of-the-art, loaded with all sorts of crazy optimization tricks it would make your head spin. For this reason many languages (including brand new languages) are targeting it as their runtime, as opposed to writing their own virtual machine which I guess some people still like to do. However, I think the route the JVM has taken is one which will gradually wane in popularity as programmers begin to face a future with dozens if not hundreds of CPU cores at their disposal.

Designers of virtual machines will begin to undergo a realization which is already upon the designers of computer processors: focus less on doing one thing quickly and focus more on being good at doing many things at once. This is why I do not believe the state of the art in virtual machine lies in things like the JVM. Rather, I see the Erlang virtual machine as being the state of the art. Java simply was not designed for this future:

Image stolen from Ulf Wiger who took it from Joe Armstrong who borrowed it from Erik Hagersten

At first glance it may be difficult to appreciate what makes the Erlang VM interesting. Soft realtime garbage collection is nice but the JVM has hard realtime garbage collection. Erlang's JIT is comparatively slow at things like numerical computing and can't inline across modules the way HotSpot can inline methods across classes. It's a notoriously difficult beast for inexperienced system administrators who may see it start gobbling up CPU and RAM for no apparent reason, and the only way to probe what's doing is to get onto its shell and enter a bunch of commands in an esoteric functional language, horrors!

However, these problems are relatively minor when you look at what's on the roadmap for the Erlang VM. In the beginning of his book Programming Erlang, the language's creator, Joe Armstrong, lays out the dream of what his concurrency model hopes to achieve: your program runs N times faster on N CPUs. Erlang has more or less achieved this through the use of distribution. However, distributed computing is hard: suddenly you must consider the cost of sending messages across the network, dealing with latency, network outages, crashing servers, etc. While Erlang/OTP provides a great framework for distributed computing, many of the concerns of distributed computing become irrelevant when you're dealing with a single machine with vast numbers of CPU cores.

The Erlang virtual machine now supports an SMP scheduler which runs a hardware thread for each CPU core. This lets the VM distribute processes, its concurrency primitive, across multiple CPUs at once with none of the impacts distributed computing has to the programmer. It's a comparatively simple affair: make as many processes as you want and the scheduler will load balance them accordingly.

However, while this all seems nice and rosy the SMP scheduler is full of bottlenecks. It works quite well for two or four CPU cores, but as you add more and more you see diminishing returns. The same goes for the JVM, but while they're stuck with their fundamental design constraints, the Erlang concurrency model allows its VM designers considerably more freedom for optimization.

A recent presentation by super famous Erlang guy Ulf Wiger lays out the future of multicore programming with Erlang. The presentation goes through the present scalability weaknesses of the Erlang VM and how they plan on addressing them. You can take Ulf's word for it, or if you're lazy you can just keep reading and I'll try to break it down for you.

So, here's what it is: (sorry for ripping these images out of your presentation, Ulf! They rule!)

This image had a big Ericsson logo on it. Did I do bad by ripping it off? Oh well, if someone complains I'll make my own awesome version.

This is how the Erlang SMP scheduler works today. There are lots of schedulers, each running in a native thread, which all pull from a common run queue. Can you see the bottleneck? Look at all those arrows pointing to the same place! So what's the solution? Divide and conquer:

By giving each scheduler thread its own run queue, the only centralized ickyness becomes the logic needed to load balance processes across various schedulers. This whole thing is beginning to look a lot like how an operating system's kernel works, isn't it? Well great, so how did that all work out in terms of SMP scalability?

Uhoh, the diagonal line I was expecting appears to be a bit flaccid

The red line shows the present SMP scheduler with a single run queue, and the blue line shows the next generation SMP scheduler which will be released in the next version of Erlang. As you can see, adding multiple run queues improved the scheduler's performance, but clearly there is still a bottleneck as you add CPU cores. The graphs show the program actually slowing down with increasing number of cores. So what's the problem?

Aack, a bunch of arrows pointing at the same place again!

Erlang's memory allocator is presently a locking contention point and is limiting the SMP scheduler's scalability across increasing numbers of CPU cores. I can only assume from Ulf throwing this into his presentation that this is the next bottleneck the Erlang/OTP team at Ericsson intends to address after they've released the new SMP scheduler.

The moderate gains of the new SMP scheduler may not seem like something to be excited about, especially since the improvements in the benchmark weren't all that spectacular. So why am I excited? Because one by one the developers of the Erlang virtual machine are removing scalability bottlenecks and increasing the VM's performance. Due to Erlang's underlying concurrency model, the optimization potentials remain huge. And as the number of CPU cores available on a single chip continues to increase (this benchmark was run on a 64-core CPU) these sorts of optimizations will become increasingly necessary to leverage a CPU's full capabilities.

Architectures like the JVM aren't completely doomed in the multicore future. Approaches like software transactional memory can be used on the JVM (particularly with a language like Clojure) in ways that are better suited to certain types of concurrency problems than Erlang's shared-nothing process approach.

But overall, I find Erlang's approach to concurrency as one which makes things easier on the programmer while foisting more of the underlying complexity onto the VM itself. I think Erlang's approach to concurrency will generally make more sense to programmers than STM, especially if it has the right kind of face on it, which is what I hope to achieve with my language Reia, which targets the Erlang VM. And most of all, I see great potential for VM optimizations to improve the scalability on multicore CPUs, and hope one day the SMP scheduler is able to achieve Joe Armstrong's dream of your program running N times faster on N CPUs. Whatever approach the Erlang VM eventually settles upon will be studied for years to come.

50 comments:

Unknown said...

great post! thank you!

Wes Felter said...

Doesn't Azul run HotSpot on a 864-core system?

Unknown said...

switched blogs I see. I'm waiting till you're done with REIA to continue effing about with the erlang VM.

I want ruby on the erlang VM and you're as close as I'm getting.

Richard Minerich said...

I never realized Erlang had it's own VM tuned for parallelism. I'm going to have to spend some time reading about Erlang for sure.

Thanks for the post!

David Anderson said...

Just a note: if the bottleneck in the new Erlang VM is the memory allocator being slammed by many OS threads, the Erlang folks should consider using tcmalloc.

Tcmalloc the memory allocator used by Google in programs with many dozens (even hundreds) of OS threads competing for memory allocator time. It has a separate allocation pool for each thread that greatly reduces the need for hitting on the contented main heap allocator, and has a ton of smart logic to recycle memory in such a way as to remain thread-local as much as possible.

It probably won't make that floppy diagonal stand completely straight, but tcmalloc is known to keep chugging along at O(n)-ish long after glibc malloc has given up and died. And the allocator, along with cpu and heap profilers, are all open source too :-).

Stephan.Schmidt said...

If the Erlang VM is optimized for parallelism, why is the actor implementation only 1.5 to 2 times faster than fresh Java actor implementations?

(Or sometimes slower, http://www.codemonkeyism.com/archives/2008/06/21/interesting-picture-benchmarking-erlang-versus-java-concurrency/ )

Otherwise good article,

Stephan

--
Programming is hard - http://blog.codemonkeyism.com
http://twitter.com/codemonkeyism

sysprv said...

Take a look at the Dis virtual machine in Inferno.

Tim said...

I wonder why so many people are currently so excited about green threads. Are there any hard-numbers that show their benefits?

Not too many years ago, there have been a number of green thread implementations. For example, the first Java VMs had them. I also remember the first Linux thread libraries being user-space implementations. But eventually it was demonstrated that a good kernel-based thread implementation was not significantly slower than a user-space implementation, and common sense was that they were a bad idea that just adds complexity.

Has this changed on modern multi-core CPUs, and if yes, why?

Anonymous said...

Tim,

Erlang uses green threads in conjunction with systems threads so if you have 1000 processes (as they are called in erlang) on a machine with four cores they will be spread across the four cores.

An erlang process is a very lightweight thing; you can create thousands of them in under second or so and they consume very little memory (200 bytes of the top of my head). This coupled with the actor model creates a *very* powerful way of writing systems that will work across multiple core/cpus/machines.

Arkadin Agathos said...

Love your blog, hate the design. Light lettering on very dark background. Very difficult to read. Dark lettering on light background: much easier to read. 'Nuf said.

Unknown said...

دانلود بازی مهارت های موتور سوار دیوانه-Mad Skills Motocross 2
دانلود بازی آنلاین نظم و آشوب اندروید-order & choaos
تصاویر و عکس های تحصن معلمان در 27 فروردین ماه سال 94

Unknown said...

security system in delhi
cctv camera dealers in delhi
cp plus cctv camera online
hikvision camera online
cctv camera installation services in delhi
cctv camera installation services in gurugram
cctv camera installation services in gurgaon
cctv camera installation services in noida

Prateek Barn said...

avriq
avriq India

mariablack said...

Pc Optimization
Windows Installation
Data Recovery

shabnam praveen said...

Call girls in Kolkata
Call girls in Chandigarh
Call girls in Chandigarh
Call girls in Gurgaon
Call girls in Chandigarh
Call girls in Chandigarh

shabnam praveen said...

Call girls in Lucknow
Call girls in Guwahati
Call girls in Mumbai
Call girls in Jaipur
Call girls in Jaipur
Call girls in Jaipur
Call girls in Bangalore

Unboxthedigital said...

best english speaking institute in delhi
english speaking course near me
english speaking course in delhi
spoken english institute in delhi
english speaking institute in delhi
english coaching in delhi
best english speaking course in delhi
best institute for english speaking in delhi
english speaking classes in delhi
best english coaching in delhi
best english speaking institute in delhi with fees
english institute in delhi
best english institute in delhi
english speaking and personality development classes in delhi
public speaking institute in delhi
english speaking courses in delhi
best english classes in delhi
english coaching classes near me
english speaking course in rohini
english language institute in delhi
english spoken institute in delhi
spoken english course in delhiIgnou Projects
IGNOU MBA Project
Ignou Synopsis
Ignou MBA Report

Ignou report
Ignou MBA synopsis
Ignou MAPC Project Report
IGNOU MAEDU Project
Ignou MA Psychology Project
Ignou MESP 1 Project
Ignou MAPC synopsis
finance project for mba
Ignou MA Education Project
finance topics for mba project

Devi.Angularjs said...

Great Article. Thank you for sharing! Really an awesome post for every one.
Project Centers in Chennai

JavaScript Training in Chennai

Final Year Project Domains for IT

JavaScript Training in Chennai

Nannie Co Pam said...

IEEE Project Domain management in software engineering is distinct from traditional project deveopment in that software projects have a unique lifecycle process that requires multiple rounds of testing, updating, and faculty feedback. A IEEE Domain project Final Year Projects for CSE system development life cycle is essentially a phased project model that defines the organizational constraints of a large-scale systems project. The methods used in a IEEE DOmain Project systems development life cycle strategy Project Centers in Chennai For CSE provide clearly defined phases of work to plan, design, test, deploy, and maintain information systems.


This is enough for me. I want to write software that anyone can use, and virtually everyone who has an internet connected device with a screen can use apps written in JavaScript. JavaScript Training in Chennai JavaScript was used for little more than mouse hover animations and little calculations to make static websites feel more interactive. Let’s assume 90% of all websites using JavaScript use it in a trivial way. That still leaves 150 million substantial JavaScript Training in Chennai JavaScript applications.

geekymr said...

best folder lock software for windows here is the list of top 5 files and folder lock software for windows.
mcafee vs windows defenderwhich is a good choiche to make your computer safe

Susan said...

I really like your writing style, great information, thankyou for posting. Latest Sims 4 Downloads : Showing 9113 items. Customise your view with Compatible mode, Featured uploads, Favourites and more! sims 4 news

myslot said...

เบื่อสล็อต xo แบบเดิมๆต้องลองเกมสล็อตออนไลน์ slotxo สนุก สะดวก สบาย เล่นง่ายๆที่เว็บนี้

https://www.slot2xl.com/

Anaminka said...




After this brings up its clients get entertained well with its premium services, the Call Girls in Mahipalpur is liable to increase the level of fun by its quality services dissimilar to get anywhere. So, let us forget everything and take pleasure in what you wait for a long time. Check our other Services...
Call Girls in Vasant Kunj
Escorts in Mahipalpur
Escorts Service in Vasant Vihar
Russian Escorts in Dwarka
Female Escorts in Aerocity
Escorts in Vasant Vihar



slotxo said...

ไม่ต้องใส่สูตร โบนัสก็แตก ลองมาเล่นดูแล้วจะรู้ สล็อตออนไลน์ ที่ดีที่สุดต้องที่นี่เท่านั้น

https://www.slotxo369.com/

Mirana said...

This is a good thing for your life.

https://www.jackpotxo1.com/

nilila said...

Play slots games via mobile on the pussy888 website.
         Friends know that there are people in Thailand, more than 50% of countries that like to gamble online. There are many different types of gambling in our house, such as online football betting, online card games, lottery bets, which can be considered as well. Including online slots games that require real money to be played In today's gambling, it is considered very comfortable because friends can sit and relax and play at home without having to go out to the casino or casino again before. Lottery entries can also be placed on the website without going out with the dealer. Can you see that the technology has made online betting grow quite high?
Playing online slots is also a gambling. One game that can be played from everywhere. The good thing about online slots is that they are fun, fun, relieving and you can get a chance to win from the bonus or the jackpot of playing. In which friends, if wanting to play online slots on our website, can also play both on computers and mobile phones

Online gambling that can be played via mobile on the pussy888 website
1. Online slots
Sometimes it's up to Pussy888. The games on the 1st floor of the website are inevitable because of the online slots games on our website that have been accepted as the 1st floor of all websites in Thailand. Because in addition to having the most slots games to play, each game is clear and beautiful in Full HD, and the play system is world standard.
2. Online fish shooting games
Fish shooting games Sometimes you can say that fish shooting games are excellent, online gambling games in this era would not be wrong. Pussy888 has a lot of room to play fish shooting games. Who is really fascinated with the thrill of the ocean? Need to try playing fish shooting games on the Pussy888 website.
3. Online Arcade Games
Arcade games, even those who are still confused with the term arcade games, have to briefly explain that it's a cabinet game or press game that we used to play in the childhood or cabinet games located in the game zone in the mall. There are games that we probably know well, including racing games, monkey collecting games, banana games, motocross motorcycle games, these games Now, online gaming websites have improved to be able to be played on our website. On our website, every game that a lot of people have ever played has been accumulated at Pussy888.
4. Online casinos
Online casinos are also available to play on Pussy888. There is a live casino broadcast straight from the world's top casinos. There are all casino games such as Baccarat, Roulette, Black Jack, Dragon Tiger, Hi-Lo, bouncing gourds, crab, fish. Even if the Pussy888 website is named in terms of slots and online fish shooting games But the website also has a casino corner to play as well
5. Online lottery or Keno
Online lottery Or on the website pussy888 , called Keno, is a lottery ticket. Which the system will randomly select the sequence that comes up Including players must place bets By placing bets in a variety of ways, even more than direct lotteries, including straight numbers, high and low bets Evenly stab design Including a lot more
     Amateur Pussy888 can play online gambling games on many mobile phones, including most variously. Just one website is worth it. No need to go to find any website to apply further Can apply to receive free test credits Before thinking of deciding more real money

918 kiss said...

Amateur 918 kisscan play online gambling games on mobile phones today. Give away everyday Have fun at 918 kiss

joker slot said...

JOKER GAME
สูตร SA
SA GAMING
SEXY GAMING
HILO
ROULETTE
SEXY GAME
JOKERGAME
JOKER GAMING
ya ya ya hah aha

joker slot said...

Sexy Gaming

Sa game

Sa Gaming

Joker slot

sa-casino

Sexy Gaming

ufabet 72

บาคาร่า UFABET


ja ma mia

lnwslotgame said...

อยากรวยกดมา UFABET

AutuwalletBlog said...

It is good content.
autowallet
สล็อตออนไลน์
ออโต้วอเลท

joker slot said...

SLOT V
JOKER GAME
SEXY GAMING
winner55
SA GAMING VIP
สูตรเซียน

5555

michael obafemi said...

great post!
Thank you.
autoslot
สล็อตออนไลน์
ออโต้สล็อต

KingNintynineStar said...

I truly appreciate this post. I’ve been looking all over for this! Thank goodness I found it. You’ve made my day! Thx again
king99
I was recommended this blog by my cousin. I’m not sure whether this post is
written by him as nobody else know such detailed about my trouble.
king99
Hello! I could have sworn I’ve visited this web site before but after looking at
many of the articles I realized it’s new to me. Anyhow, I’m definitely pleased
I found it and I’ll be bookmarking it and checking back often!
jack88
jack88 ทางเข้า

Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with more information? It is extremely helpful for me.
รีวิว Review

I think this is an informative post and it is very useful and knowledgeable. therefore, I would like to thank you for the efforts you have made in writing this article.
ข่าว news
jack88 สล็อต


slotxo said...

เครดิตฟรี เล่น สล็อตออนไลน์ slotxo กับ

https://www.slotxo369.com/

nilila said...

blognewstime
pussy888gm

Wewallets said...

trick design slotxo

t4cetaku said...

Thank you for the good information and very helpful. That's very interesting. I like all the things you share and see your beautiful creations, greetings from Admin nantikanlah.com I hope you can visit and comment on my website :)

slotxo said...

slotxo ยอดเทิร์นจะมีการอัพเดทใหม่ทุกวันในช่วงเวลา 11.00น. – 12.00น.

nntfunny said...

Thanking for sharing Effective article. Please follow my portfolio. click here slotxo

Mr.slot said...


It's really progress It can create something like this >> slotxo

Unknown said...

Apply for PG SLOT, just make a minimum first deposit of 100 baht, get a 50% instant bonus up to 500 baht.

slotxo
pgslot

DIO1337 said...

Slotxo pc สล็อตออนไลน์ แจ็คพ็อตแตกง่ายที่สุดในไทย สมัครง่าย เล่นง่าย ได้เงินจริง

ISC123 สมัครไลน์

KingNintynineStar said...

king99 king99
jack88 slot jack88
รีวิว
https://topbetlink.com

KingNintynineStar said...

jack88 game jack88 download
jack88 สมัคร jack88
jack88 เกมสล็อต sagame
สมัครking99
jack88 ดาวน์โหลด
jack88 สล็อต

KingNintynineStar said...

jack88 โปรโมชั่น
jack88 สมัคร
league88 น้ำดี
sbobet ทางเข้า
sbobet เว็บตรง
maxstep2
sexygame
sagame66 เล่นง่าย

KingNintynineStar said...

หนังออนไลน์
หนัง ใหม่
sexygame66
BAOver
ufa191
ทางเข้า gclub มือถือ
sclub
mega888

KingNintynineStar said...

ace333
918kiss สมัคร
slotciti
168lottothai
sagame
sexygame

KingNintynineStar said...

lsm99
league88
jack88
sclub
ace333
slotciti
joker slot
slotxo
sbolotto88

slotxo said...

สมัคร slotxo ตอบสนองความต้องการผู้เล่นได้อย่างยอดเยี่ยม ไม่ว่าจะเลือกเล่นพนันเกมไหนก็เดิมพันได้อิสระ