Wednesday, September 28, 2011

Object Oriented Concurrency: A Deep Dive into Celluloid (Part 1)

There's a quote from Alan Kay I've often cited as a source of inspiration in my various projects, from Revactor to Reia to Celluloid:
"I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages"--Alan Kay, creator of Smalltalk, on the meaning of "object oriented programming"
This conception of objects likens them to things in the world which are very much concurrent. Every cell in your body is running at the same time, and cells communicate with each other using chemical signals. Computers on a network are all running at the same time, sending each other messages via network protocols.

In Alan Kay's conception of object oriented programming, it should be a natural model for concurrency. However, most languages that describe themselves as "object oriented" today bolt on concurrency as an afterthought, just a thin abstraction layer around the concurrency primitives provided by the operating system, such as threads, conditions, and mutexes. The hardware and OS people went through the herculean effort of abstracting hardware-level concurrency into threads, and from there, for the most part the programming language people haven't done much, at least in mainstream object oriented languages.

The exception lies in the functional language domain, in languages like Haskell, Clojure, and Erlang. Erlang, in part inspired by Alan Kay's language Smalltalk, has abstracted concurrency in such as way that users of the language are able to reason about it intuitively, something which can't be said for most users of threads. The key to Erlang's success, its secret sauce, lies in its message-based concurrency system, a system inspired by Smalltalk.

Smalltalk is generally considered to be the forerunner of all object oriented programming languages, but when it comes to taking to heart Alan Kay's conception of a message-based system, Erlang beats the mainstream object oriented languages hands down. Erlang has also incorporated the idea of communicating sequential processes, first seen in languages like Occam. By combining the ideas of Smalltalk with those of communicating sequential processes, Erlang created a system where concurrency is baked into the core of how the language operates, enabling fully isolated processes that talk to each other with asynchronous messages.

If this idea works so well in Erlang, can it be added to other languages in a way that makes sense?

From Actors to Concurrent Objects

Given Erlang was vicariously inspired by Smalltalk, an object oriented language, is there any reason why Erlang's ideas can't be applied to present-day object oriented languages to give them concurrent objects? The way Alan Kay describes objects they should inherently be concurrent, but the way we're used to working with objects is inherently sequential.

A whole slew of projects, including one of mine, have tried to implement Erlang's concurrency model in Ruby. None of them have been popular. In my opinion, this is because these projects (including my own project Revactor) have stuck far too close to Erlang, trying to write an Erlang DSL in Ruby instead of trying to reimplement these concepts in a Ruby-like manner. This makes trying to learn them pretty confusing, to the point where the best way to learn them is to first learn Erlang, then after you understand how Erlang works, you can translate in your head from Erlang into these Erlangy Ruby DSLs. Seems bad...

I read a blog post recently about how in the Smalltalk objects that end in "-er" (e.g. Manager, Controller, Builder, Worker, Analyzer) are "bad." For a long time I've thought much the opposite, that there are fundamentally two different types of objects: those which are inanimate and represent objects in the world, and those which are animate and active and manipulate the inanimate objects. I see the programming world as consisting of two very distinct things: actors, and the objects which they act upon. It's not that objects that end in -er are inherently bad, it's simply that OOP languages (including Smalltalk) have never contained the proper primitive for modeling their behavior.

Celluloid is a new kind of concurrency library. It allows you to model concurrent (i.e. thread-backed) actors in a system but still interact with them as if they were plain old Ruby objects. All you need to do to create concurrent objects in Celluloid is to include the Celluloid module in any Ruby class:

That's it! Just by doing "include Celluloid" all instances of this class you create will not be normal Ruby objects, but instead fully concurrent Celluloid-powered objects. These objects will continue to function just like any other object in Ruby. When you call methods on them they return values. If a method raises an exception, the method call aborts and the exception will continue to propagate up the stack.

I want to emphasize this because I think it's incredibly important: users of concurrent objects should not need to be aware that they're concurrent. They should work as closely as possible to any other plain old sequential objects.

Concurrent Objects in Celluloid

I'm going to help you understand how Celluloid works by first starting from a diagram of normal Ruby object interactions and building up from there. Let's start with a diagram of one method calling another:

Hopefully this diagram is easy to understand. Calls come in, responses go out, never a miscommunication! A call consists of the name of a method to invoke and arguments. The response consists of a value to return, or an exception to be raised. Object oriented dogma would have you think of a call as the caller sending a message to the receiver, which in turn sends a response back.

Now let's look at a concurrent object as implemented by Celluloid:
A quick overview: the caller and proxy objects function like normal Ruby caller and receiver objects in the previous diagram. However, the actor circle (i.e. the giant olive-looking thing) represents Celluloid's encapsulation of concurrency. Each actor runs in a separate thread, and exchange messages with proxy objects through their mailboxes, which are thread-safe message queues. Let's dive in!

All communication between regular Ruby objects and Celluloid's actors passes through a proxy object which intercepts the calls. This proxy is responsible for the central illusion that makes Celluloid's concurrent objects as easy to use as any other object, despite them being off in a distant actor somewhere that you must communicate with via messages through thread-safe mailboxes. The proxy object abstracts that all away, and gives you the familiar object-dot-method syntax that you know and love.

The proxy object translates the method call from a normal invocation into a Celluloid::SyncCall object. This object contains the method to be invoked and the parameters. This message is sent from the proxy to the recipient actor by way of its mailbox (which is an object of the Celluloid::Mailbox class).

Mailboxes work like thread-safe message queues, allowing several pending method invocations that the actor can process one-at-a-time. After a method has been invoked on the receiver object, the return value is sent back to the proxy in the form of a Celluloid::Response object. This value is then returned from the original method call made to the proxy.

This gives us the same behavior as a normal method call. However, normal Ruby objects aren't concurrent. To gain concurrency, we need to go beyond the normal method call pattern.

Adding Concurrency: Async Calls and Futures

Celluloid provides two methods for breaking out of the typical request/response call pattern:

  • Asynchronous calls: send a Celluloid::Call to a recipient actor, but doesn't wait for a response. The method call to the proxy returns immediately. To make an async call, add "!" to the end of any method.
  • Futures: allow the value a method returns to be obtained later, while that method computes in the background. To make a future call to an actor, use the #future(:method_name, arg1, arg2... &block) syntax. This returns a Celluloid::Future object, whose value can be obtained by calling Celluloid::Future#value
Let's look at these both to see how they actually work. First an asynchronous call, as we'd do it from Ruby code:


Even though ScarlettJohansson doesn't have an #oops! method, things are continuing to work as if you made two calls to the normal ScarlettJohansson#oops method, but no value is returned. Here's a diagram of what happens when you make an asynchronous call:


Asynchronous calls do not wait for responses, but forward a request for a method to execute, immediately returning back to the caller. No mechanism is provided to obtain a response to this call; async calls are a strictly fire-and-forget type of affair.

What if we want to invoke a method asynchronously but obtain the value the method returns at some point in the future? Enter futures!

Let's imagine we have a small army of Scarlett Johansson clones leaking photos on the Internet. Each one of them takes a little while drinking booze and fumbling around with their phone before they're actually able to post a nude photo. How can we tell them all to start doing this then reaping the results instantaneously when they're done? Let's use Celluloid::Actor#future to accomplish this task:

Our Scarlett Johanssons are now leaking photos to the Internet in parallel, and the wall clock time needed to have all of them get drunk and fumble around with their phone in order to post their nude photos is now barely more than the time each individual Scarlett took to post a couple photos. We're able to get the return value from ScarlettJohansson#oops after asynchronously requesting it be computed. How does this work exactly?


Celluloid implements futures through the Celluloid::Future object. Futures represent the result of a method which may or may not have finished executing before its result is requested. To obtain the result of a given future, call Celluloid::Future#value, but you need to be aware that Celluloid::Future#value will block until the method the future is associated with has finished executing.

Obtaining the value of a pending future is, like always, synchronized through a Celluloid::Mailbox object, which is used to provide a thread-safe interaction point for receiving the result of a method call if it hasn't already completed.

More to come!

I'd originally hoped for this blog post to contain a lot more, but honestly it's been sitting around half-finished for so long I thought I'd break it apart into a series.

In Part II, I'd like to cover what makes Celluloid truly unique (to my knowledge) among similar systems: "reentrant" method dispatch. This post will cover problems in existing RPC systems, including Erlang's gen_server, and discuss why Celluloid provides a more natural object abstraction. It will also explain why and how Celluloid uses Fibers to solve this problem, and the additional features that can be built upon Celluloid's reentrant method dispatch.

Part III (if I get that far) will provide a more in-depth look into some of the Erlangier features of Celluloid such as linking and supervision, and also discuss the roadmap for integrating some additional Erlang features into the Celluloid framework such as OTP-style applications and distributed actor systems. Stay tuned!

235 comments:

«Oldest   ‹Older   201 – 235 of 235
cookie-consent-popup said...

I would like to recommend this to my friends. Thank you for sharing this post.
GDPR COOKIE CONSENT

Sages Marketing said...

Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

Security Guard License
Ontario Security License
Security License Ontario
Security License

Thank you..

veramedhealthcare said...

Really nice, keep it up for more valuable information. Visit Veramed for Sensitive Skin Body Moisturizer Online and Pearl Glow Ayurvedic Skin Care Products at affordable price.
Buy Pearl Glow Herbal Skin Care Products

James Smith said...

This list of the best love quotes of all time is sure to make it easier to express your emotions and feelings of love to your partner. These famous inspirational love quotes and sayings will help you describe exactly how you are feeling with the most simple wording.


Today’s business banking decision makers don’t want much — just a simpler relationship with their commercial bank.

Academic Session - the time during which a school holds classes; "they had to shorten the school term"

The intensity of the tech market today is due largely to the digital revolution, so to survive, tech companies need to be built on a digital foundation.

We had a little delay in our posting schedule this week, but in honor of the holiday takeoff season, I thought I’d share an older Christmas post that got a lot of positive response a couple of years ago.

Well, SEO, social media and emailing are the main marketing tools and the best techniques to grow web traffic. But as you already use these methods, here are some other tools that might be pretty helpful.

I’ve recently been asked by a couple people who don’t have solid business idea that they want to work with us.

Self-care is essential to make you productive and able to take on any challenges. Self care advise of your mental health, love yourself and forgive yourself for any mistakes.

The vehicles total is updated to reflect the new addition(s), and the new vehicle(s) you added are now listed on the vehicles detail screen.

We have some amazing permanent blogger working in a VVIP property for collecting more information.

An law ordinance is a law enacted by a municipal body, such as a city council or county commission (sometimes called county council or county board of supervisors). Ordinances govern matters not already covered by state or federal laws such as zoning, safety and building regulations.

poker online said...

hanya dengan modal 10 ribu rupiah saja anda sudah bisa bermain di IDN poker situs judi poker online dengan permainan judi yang lengkap dan bisa menguntungkan semua pemain dengan uang asli, dengan pelayanan terbaik untuk bermain poker anda bisa bermain dengan lancar setiap hari tanpa gangguan lag atau yang lainnya.
http://menangpokersetiaphari.net/

https://www.mamacasinos.com/khatrimaza/ said...

Worldfree4u is an illegal torrent movie downloading website to download Worldfree4u latest Bollywood Movies,Worldfree4u

IDN Poker online said...

Mulai dari Texas Poker Online Pot Omaha hingga Ceme Keliling bisa anda mainkan hanya dengan modal awal sebesar 10.000 dan bisa dapat jackpot hingga ratusan ribu rupiah setiap harinya. kelebihan nya tidak sampai situ saja, ada banyak bonus yang bisa anda rasakan saat pertama kali mendaftar. Buruan klik linknya sekarang https://wilibe.com/situs-judi-idn-poker-online-uang-asli-deposit-paling-murah-cuma-10rb/

Unknown said...

There are plenty of benefits of hiring a white label digital marketing services USA. But, whether you’re just starting digital marketing or you’re looking for a new digital marketing agency. Look for the rating and reviews of the online marketing company.

Lenoxtons 20 said...

HP Stream 14 laptop will be apt for you. Powered by Intel Celeron N4000, dual-core processor, this device is a combination of style and functionality. This value for money laptop comes with BrightView WLED-backlit display, Office 365 personal for one year and 4GB DDR4-2400 SDRAM and 64GB eMMC.
HP Stream 14-inch Laptop
HP Pavilion Intel Pentium Gold 4417U
HP 2020 Newest 17.3 Inch Flagship Laptop
2020 hp 14 inch hd touchscreen
hp 15.6 hd touchscreen laptop

Seputarpoker said...

pilihlah agen slot online yang terpercaya agar anda bisa mendapatkan judi slot bonus besar dan keuntungan lainnya seperti fasilitas dan pelayanan terbaik.

Stainslev Pro said...

Pokerwan, situs judi online yang menyediakan daftar poker online terpercaya. Situs IDN Poker Online ini menyediakan layanan judinya selama 24 jam non stop. Di situs kami anda bisa memainkan game poker online dengan mudah, bahkan anda bisa mengakses poker android terbaru dengan cara mendownload aplikasi idn poker apk di situs kami. Situs poker online ini sudah mendapatkan lisensi resmi dari pihak PAGCOR dan BMMTestLab. Kami menjamin anda bisa memainkan permainan judi online yang bebas dari BOT sama sekali bossku.

agen resmi sbobet di indonesia said...

tinggalkan tempat main yang lama, langsung aja ganti main di agen sbobet online terpercaya berikut ini karena selain mudah dimenangkan, hasil kemenangan yang bisa didapatkan juga pasti di bayar http://jackpotbahagia.com/agen-sbobet-resmi-yang-menawarkan-minimum-deposit-terjangkau/

situs judi sbobet terpercaya said...

ada begitu banyak keuntungan yang bisa kamu dapatkan di http://onlinegamblingspecialreviews.com/agen-sbobet-online-dengan-fasilitas-unggulan-terbaik/

situs judi sbobet terpercaya said...

yuk daftar dan main di http://onlinegamblingspecialreviews.com/agen-sbobet-online-dengan-fasilitas-unggulan-terbaik/ dan dapatkan begitu banyak keuntungan menarik di dalamnya.

juragan88 said...

idnpoker situs judi kartu online yang memiliki banyak permainan judi terbaik yang bisa menguntungkan semua pemain dengan uang asli dan bonus besar setiap hari dengan mudah, dengan cara deposit di bank lokal terpercaya atau bisa juga pakai ovo dan gopay untuk mempermudah semua pemain untuk deposit dengan aman dan lancar.

riya singh said...

I enjoyed over read your blog post. This was actually what i was looking for and i am glad to came here!
Website: Imitation jewellery

Anirban Ghosh said...

You have provided finicky information for a new blogger so it has turned out to be really obliging. Keep up the good work!
SAP training in Kolkata
SAP training Kolkata
Best SAP training in Kolkata
SAP course in Kolkata

Agen Judi Online RESMI said...

Permainan situs slot online deposit pulsa ada yang asli dan palsu, maka dari itu cari tahu terlebih dahulu gimana situs yang satu ini sebelum bermain ya!

HOT AND SEXY INDEPENDENT ESCORTS IN LUCKNOW said...

THANK YOU FOR VISITING MY WEBSITE:-
russian escorts in gurgaon
housewife escorts in gurgaon
gurgaon escort services
gurgaon escorts
escorts in gurgaon
escort services in gurgaon
gurgaon call girls
call girls in gurgaon

IDN Poker online said...

bisa dapat 30jt rupiah hanya bermain idnpoker disini. Dan yang lebih enaknya lagi jackpot yang didapat utuh dan tanpa potongan sedikit pun.

arshtech said...

Nice post. Thanks for sharing. Please keep us updated for more. Please connect for digital services at affordable rates.
Mobile App Development Singapore
Freelance Web Development Singapore

agen resmi sbobet di indonesia said...

ga perlu pusing lagi kalah taruhan bola, karena setelah anda tau bagaimana cara menang sbobet yang mudah dipahami itu dijamin jadi bisa menang setiap hari

situs judi sbobet terpercaya said...

Bermain di Situs Sbobet Mobile Indonesia terpercaya, maka akan ada banyak keuntungan promo menarik dan fitur hadiah tanpa diudni

Seputarpoker said...

main slot online bisa pakai pulsa loh sekarang, cuma modal 25 rb pakai pulsa XL dan Telkomsel sekarang bisa langsung main.

centre99 said...

it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
chinese visa Singapore

Anu said...

Thanks for sharing this, I actually appreciate you taking the time to share with everybody.
Data Science Training In Hyderabad

duetqiu said...

Situs Judi QQ Online Terpercaya
Poker Online

duetqq.site said...

Situs Poker Online Terpopuler
BandarQQ

duetqiunet said...

Situs Poker Terpercaya
Poker QQ

PartaiQQlink said...

Situs Poker Terpercaya
Dewa Poker

DuetQQ said...

Situs Poker Online
Domino99

ListBandarQiu said...

Situs BandarQ Terpercaya
Judi Online

travel5212 said...

Amazing Article,Really useful information to all So, I hope you will share more information to be check and share here.thanks for sharing .
Website: Trip to Vietnam

Agen Judi Online RESMI said...

yuk sekarang main di situs slot online terbaru . kalian bisa menerima keuntungan yang berlimpah hanya dengan main di situs ini!

Seputarpoker said...

agen slot menyediakan fasilitas lengkap dan bahkan bisa bermain judi slot online langsung dari Hp kita loh.

«Oldest ‹Older   201 – 235 of 235   Newer› Newest»