Tuesday, March 22, 2011

Distributed systems and dynamic typing

There's a very good blog post by Robert Harper, a CMU CS professor, called "Dynamic languages are static languages." It's very enlightening and I strongly encourage you to read it. If I understand it correctly, which I freely admit I may not, the general idea is that dynamic languages can be thought of as static languages that have a single all-encompassing type. In that regard, dynamic languages are a proper subset of static languages. If you think I misinterpreted his post and I'm confused, please flame me in the comments.

In a previous post, called "Parallelism is not concurrency", he opines on a pet peeve of mine, namely how the terms parallelism and concurrency are nonchalantly and incorrectly interchanged. Parallelism applies to deterministic operations that operate on similar data in similar time. Some examples of parallel operations include rendering of 3D scenes on GPUs, or encoding/decoding blocks of compressed video. Concurrency, on the other hand, refers to the nondeterministic manner in which distributed systems operate, particularly ones where CPUs are separated over an unreliable network which can lose connectivity at any time.

In a third blog post, "Teaching FP to freshmen", Robert says he'll be teaching Standard ML to freshmen, and further argues he won't be teaching Object Oriented Programming because it's "both anti-modular and anti-parallel by its very nature, and hence unsuitable for a modern CS curriculum."

Three very interesting blog posts by the same person, clearly a well-educated computer science professor who knows more than I do. I mean, I like programming languages and I'm working on my own dynamically typed functional programming language, but Robert Harper wrote a book, one I certainly couldn't write. So who am I to judge?

Parallelism and Concurrency are Both Important

What I see missing from Robert Harper's writing is any attention paid to concurrency. He pays intense attention to parallelism, recognizes parallelism as important for the future, and strongly advocates functional languages as a great way to address the problems of parallelism. I have absolutely no bone to pick with him here and wish him great luck with his courses at CMU which address this problem domain.

However, my interests primarily lie in the realm of concurrent computing, and particularly in the area of distributed computing. In the area of distributed computing I find dynamic languages particularly important and think Robert Harper's article on static vs. dynamic languages omits some of the advantages of dynamic languages which make them work better in distributed systems.

This is a weighty subject, and one in which I don't think my own ideas alone make a particularly cogent argument. For some opening remarks, I will defer to Joe Armstrong, creator of the Erlang programming language, and his opinion from the book "Coders at Work" by Peter Seibel. As a bit of context to this quote, Joe is discussing the potential advantages that a static type system might confer upon Erlang. But then he gets to the disadvantages...
...the static type people say, "Well, we really rather like the benefits of dynamic types when we're marshaling data structures." We can't send an arbitrary program down a wire and reconstruct it at the other end because we need to know the type. And we have—Cardelli called it a system that's permanently inconsistent. We have systems that are growing and changing all the time, where the parts may be temporarily inconsistent. And as I change the code in a system, it's not atomic. Some of the nodes change, others don't. They talk to each other—at certain times they're consistent. At other times—when we go over a communication boundary—do we trust that the boundary is correct?
Type Systems and the CAP Theorem

There are two particularly sticky problems when it comes to the use of type in distributed systems. The first is the issue of serialization, or marshaling, of particular states. One way or another this is a solvable problem, both for statically typed and dynamically typed languages. I really don't want to delve too deep into this issue as it distracts from the larger point I'm trying to make, but in general, I think this is an easier problem to solve in dynamic type systems. I would also like to note that serialization formats which vomit their types all over the protocol and the codebase are outgrowths of static type systems. I'm looking at you, CORBA, SOAP, Protocol Buffers, and Thrift. On the flip side, systems which choose a minimal, semi-universal type system, such as JSON, BSON, BERT, and Msgpack, are all outgrowths of dynamic type systems. If I have a point to make here, I think it's that these systems are outgrowths of two very different ways of thinking about the same problem.

Marshaling is still a very important topic in distributed systems. Erlang largely abstracts this problem away from the programmer, allowing distributed nodes to exchange Erlang "terms" between processes on distributed nodes in the exact same way one would exchange messages between two processes located on the same host. The process of serializing that data, transmitting it over the network, receiving it via TCP, decoding it, and sending it to the appropriate Erlang process, is completely transparent to the end user. This is an extraordinarily powerful abstraction.

While statically typed languages can attempt to marshal data in a minimalistic JSON-like type system, this typically isn't the case. Instead, statically typed languages generally seem to prefer to vomit their types all over the protocol. The boilerplate code needed to marshal/unmarshal particular types can be generated automatically by a declaration of the types and methods of a particular protocol, such as the WSDL files used by SOAP. Again, users of statically typed languages could reduce the state of particular entities in their system to one which could fit into a minimalistic type system, but for static languages this is still a manual process, or one which requires manual code generation. In a language like Erlang which is built from the ground up to be distributed, dynamic, and operate around a minimalistic type system, serialization and deserialization can happen completely automatically.

Why is this important in a distributed system? Because, to paraphrase Joe Armstrong, distributed systems are messy. Imagine an Erlang-like distributed system that's statically typed. In order for such a system to work effectively, all nodes in the system must have the exact same code loaded and therefore have a universal consensus on what the types in the system are. This has huge implications on the theoretical properties on such a system. In order for a distributed system to agree on the nature of all types, it must be consistent.

However, if you're familiar with the CAP theorem, you may recognize the inherent problem this may cause. The CAP theorem gives you three options: a consistent fault-tolerant system, a highly available fault-tolerant system, or a consistent highly available system which breaks at the first sign of a fault. Only two of these options provide the consistency needed to ensure universal agreement on the types in the system such that automatic marshaling/unmarshaling of static types is even possible. In a distributed system, you either must give up universal agreement on the types, or sacrifice availability.

To quote Joe again, distributed systems are composed of parts which are "growing and changing all the time" with "parts may be temporarily inconsistent." While there aren't any guarantees that distributed systems built around dynamic type systems will work, inconsistent statically typed systems with disagreements about types are guaranteed not to work. Dynamic systems not only provide the possibility that your system may continue to function with different versions of the code loaded at the same time, but the ability for the programmer to plan for this contingency and offer ways to mitigate it. It's possible this may result in errors, but it may work, whereas incompatible type definitions are universally guaranteed to create errors. In a distributed environment, dynamic type systems provide extensibility, whereas static type systems actively seek to preclude it.

Something I've seen time and time again in systems like SOAP and even Thrift and Protocol buffers is attempts by programmers to escape the constraints of the type system, which almost universally fall into proprietary ways to store key/value pairs. One SOAP API I'm working with now provides "Maps" with "Entry" tags that have a key attribute and an associated value. Another implementation provides an array of "NameValuePair" objects. These solutions seem ugly, but in my opinion, their intentions are not. These are people seeking to extend running systems without the need to completely redefine the protocol. That's very much a practical concern.

Distributed Applications Must Be Flexible

In order for distributed programming to work effectively, nodes need to be able to call functions on each other without the need for programmers to write custom marshaling/demarshaling code for each type. The marshaled data needs to work extensibly, so that nodes running different versions of the code can still talk to each other in a forwards and backwards compatible manner.

Protocols will change over time. Older versions of the code need to work with a newer protocol, and vice versa, older versions of the protocol need to work with newer code. Nodes should be upgraded as practicality dictates. Perhaps your system administrator begins an upgrade and you lose access to a datacenter, because a janitor at your upstream ISP spilled a bucket of mopwater all over their datacenter's raised floor and caused a huge electrical disaster. Now your distributed application is running two different versions of the code, and it's suffered a network partition. Does this mean it should just break when the network partition is fixed?

Erlang has shown us it's possible to recover from these kinds of conditions. Even when we can't change code atomically across the entirety of a distributed application, it should still continue to operate without breaking. Distributed systems should be able to grow and change all the time without rigid boundaries.

42 comments:

IT said...

Java Training Institutes Java Training Institutes Java Training in Chennai | Java Training in Chennai | Dot Net Training in Chennai JavaScript Training in CHennai JavaScript Training in CHennai | | Single Page Application Development

Unknown said...


Thank you for the auspicious writeup. It in fact was a amusement account it. Look advanced to more added agreeable from you! However, how could we communicate? capital one card login in

shina said...

Surveillancekart | CCTV System
Pestveda | Pest Control
Termite control
Surveillancekart | CCTV System
cctv installation services
best home security camera system

shabnam praveen said...

Call girls in Kolkata
Call girls in Chandigarh
Call girls in Chandigarh
Call girls in Gurgaon
Call girls in Chandigarh
Call girls in Chandigarh

shabnam praveen said...

Call girls in Lucknow
Call girls in Guwahati
Call girls in Mumbai
Call girls in Jaipur
Call girls in Jaipur
Call girls in Jaipur
Call girls in Bangalore

Unknown said...

fortnite mod candy crush saga mod apk pubg apk

Unknown said...

weight loss supplement
weight loss supplement
Sleeping pills

naveen said...

AppVN is an outsider application store for Android. Not at all like Google Play, this application store gives you some one of a kind highlights, for example, the office to download certain official premium applications for nothing.
https://www.appvn.one/
Appvn
Appvn Apk

dadyar said...

در ملک مشاع هر کدام از مالکین می توانند نسبت به مالکیت و سهم خود تصمیماتی از قبیل اجاره دادن و یا فروش (فروش ملک مشاع) اقدام کند. به این امر در اصطلاحات حقوقی تصرف حقوقی می گویند. اما زمانی که پای اجرایی کردن تصمیم شخص برای فروش و یا اجاره به میان می آید دیگر پای تصرف معنوی در میان نیست، بلکه تصرف از نوع تصرف مادی است.
در تصرفات حقوقی تصمیم برای فروش ملک مشاع، به لحاظ قانونی ممکن است. اما هرگونه تصرف مادی و تحت تصرف در آوردن هر بخشی از ملک مشاع بدون اجازه دیگر شرکا قانونی نمی باشد. زیرا همه ی شرکا در تمام ملک سهم دارند و بخشی برای هر یک مشخص نگردیده است.
برای واضح تر شدن موضوع به این مثال توجه داشته باشید: تصور کنید امید و مرتضی هر دو به صورت مشاع مالک ملکی باشند. اگر هر یک از این دو نفر بخواهند سهم خود از ملک را بفروشد، این حق را دارد. اما اگر پس از فروش ملک، مالک جدید و یا شخص دیگری از شرکا که سهم خود را نگه داشته بود بخواهد در ملک تصرفی داشته باشد، مثلا بخواهد در ملک زندگی کند، این اقدام بدون کسب اجازه از دیگر مالکان و شرکا قانونی نمی باشد.

اگر هر کدام از مالکین بخواهند تصمیمی را که نیاز به تصرف مادی دارد را اجرایی کنند باید ملک مورد افراز قرار بگیرد. اگر بدون اذن دیگر مالکین در ملک مشاع تصرفی صورت پذیرد دیگر مالکین می توانند علیه متصرف دادخواست خلع ید به دادگاه ارائه کنند. سپس برای فروش و یا افراز ملک همانگونه که شرح خواهیم داد اقدام کنند.
فروش ملک مشاع
فروش ملک مشاع
افراز و فروش ملک مشاعی
بنا به موضوعات و مشکلاتی از این قبیل که در ابتدا به آن ها اشاره کردیم این معضلات باعث به وجود آمدن اختلافات زیادی بین شرکا می شود و حتی این موضوع همیشه باعث افت قیمتی املاک مشاع شده است. به همین خاطر اکثر صاحب خانه ها و شرکای املاک مشاع سعی در خاتمه دادن و حل این مشکل به فکر افراز و فروش ملک مشاع می افتند.
برای فروش ملک هیچ یک از شرکا مشکلی نداشته و می توانند اقدام به فروش کنند. اما اگر قصد مشخص کردن سهم خود از مال مشاع را دارند که در قوانین و حقوق به آن افراز می گویند، به اتفاق شرکا توافق کرده و سهم هر یک را مشخص می کنند. پس از توافق، برای رسمی و قانونی کردن افراز باید همه ی شرکا و مالکین نزد دفتر اسناد رسمی رفته و افراز نامه ای تنظیم کنند.
اگر در میان مالکان شخصی که دارای مشکل ذهنی است و یا کودکی که به سن قانونی نرسیده باشد حضور داشته باشند و یا حتی یک نفر از مالکان در جلسه دفتر ثبت حاضر نباشد افراز صورت نخواهد پذیرفت. در این مواقع افراز دیگر در دفتر اسناد رسمی صورت نمی پذیرد و به دلیل جلوگیری از ضایع شدن حقوق این افراد دادگاه برای افراز ملک مشاع تصمیم گیری می کند.
بهتر است بدانید: اگر مال مشاع غیر قابل افراز (تقسیم) باشد، در صورتی که طرفین توافقی بر سر مال نداشته باشند تنها دادگاه می تواند در مورد فروش و مزایده آن تصمیم گیری کند. یعنی اگر یکی از مالکین قصد فروش مال را داشته باشد باید دادخواستی مبنی بر فروش مال مشاع غیر قابل افراز به دادگاه بدهد. دادگاه نیز برای مال مزایده برگزار می کند و کارشناسی را جهت قیمت گذاری برای قیمت پایه در مزایده می فرستد.

r said...

Aishika Roy Book Kolkata Escorts & Kolkata Call Girls 9874341001 is a very charming girl working as Independent Escort in Kolkata.

Model Escort in Kolkata

College Escort in Kolkata

Escort in Kolkata

Vip Escort in Kolkata

Celeberty Escort in Kolkata

Celeberty Escort in Kolkata

Celeberty Escort in Kolkata

Model Escort in Kolkata

College Escort in Kolkata

Escort in Kolkata

Vip Escort in Kolkata

Celeberty Escort in Kolkata

Celeberty Escort in Kolkata

Celeberty Escort in Kolkata

webspace said...

We are Webspace Inc. organization working as the Best Digital Marketing Company in USA and we give many services to our client that is website designing, website development, Search Engine Optimization, E-commerce web Designing, Software Development, Google Adword and Mobile Application.
Web Development company in Los Angeles
web design New York
web development New York
online marketing New York
ecommerce web development New York
internet marketing New York
SEO company New York
seo company USA
Web development company
Web development company California
Web development company Los angeles
Professional Web Design Services USA
Website Design Comapny
Web Design Company
webiste design services
Web Development Company in USA
Web Development Services in USA
website development company in usa
Ecommerce Website Development Company in USA
Ecommerce Website Development Services
custom ecommerce development
Ecommerce Website Development Company In Usa
CMS Web Design Services USA
CMS Website development company In Usa
Digital Marketing compnay in Usa
Online Marketing Services
Digital Marketing Company Usa
Seo Comapny In usa
Professional Software Development Company USA
software development company
custom software development company
custom software development In Usa
App Development Services USA
Mobile App development Company
Mobile Application Development Services

Amber Collins said...

You have brought up a very fantastic details , appreciate it for the post.

www.gumawawebsite.blogspot.com

cute iphone xr cases cheap said...

the newest collection of aixonne website, the most beautiful jewelry of 2020, various kind of necklace for women, and all of them have discounts with unbelievable percent, best prices ever, check it out

salma said...


الرائد افضل شركة تنظيف و غسيل خزانات بالمدينة المنورة تنظيف خزانات بالمدينة المنورة وتقوم اياب تقيمها بالاشعه الفوق البنفسجية

تنظيف خزانات

customized gifts for him said...

floral cushions john lewis
floral designs for pillow cases
vintage pillow person

rima said...

siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service

gyaan said...




With our Call Girls in Gurgaonand escorts services, you may really improve stage of fun by its quality services different. Sure! This assists its people to victory over all the gloom and depressive disorders from your lifestyle and appeases your delicate wishes specifically.So, you may do a trip at web page and can get your wish suggested with an outstanding woman escort. Check our other services also...
Call Girls in Gurgaon
Call Girls in Gurgaon
Call Girls in Gurgaon
Call Girls in Gurgaon
Call Girls in Gurgaon
Call Girls in Gurgaon
Call Girls in Gurgaon

Raj Maan said...

thanks for useful info!!!
123movies

Lorriel Sims said...

type systems aimed at dealing with consistency and safety properties related with dynamic reconfiguration and communication in Open here for more complex distributed systems.

bandarq228 said...

situs dominoqq yang dapat menghasilkan uang dengan mudah hanya di http://180.215.13.115

Darren Demers said...

In order for distributed programming to work effectively, nodes need to be able to call functions on each other without the need for programmers to write custom marshaling/demarshaling code for each type. The marshaled data needs to work extensibly, so that nodes running different versions of the code can still talk to each other in a forwards and backwards compatible manner. afghani cap online , vintage choker necklace , tiger embroidery patch , Tribal Babay Banaris Dress

aditya said...
This comment has been removed by the author.
Aerocity Service said...

Paharganj Escorts
Dwarka Escorts
Russian Escorts Noida
Russian Escorts Delhi
Karol Bagh escorts
Delhi Escorts Service
Escorts in Delhi
Call girls in Delhi
Delhi escort
Aerocity Escorts

Escorts Service Kolkata said...

escorts
escorts
escorts
escorts
escorts
escorts
escorts
escorts

R1se Hluoluo said...

Bila kamu hidup jauh dengan orang-tua menjadikan menghubungi satu kegiatan rutin. Sempatkan diri 10-15 menit untuk menghubunginya, https://aksesorisfashion.com/ dapat pagi atau malam hari di saat rileks mereka. Ini supaya mereka rasakan quality time bercakap dengan kamu.

Aktivitas kemungkinan merintangi Anda untuk teratur berbicara dengan orang-tua Anda. Kemungkinan Anda berasa itu ialah hal yang lumrah, karena orang-tua tentu pahami aktivitas Anda yang tidak dapat ditinggalkan.

opbestcom said...


Looking forward to reading more. Great article post. Fantastic. Thanks so much for the blog. Much obliged.

Please Visit My homepage ➤ 대구오피

(freaky)

jasonbob said...

a bathing ape
golden goose sneakers
supreme clothing
curry
kyrie 7

Dev Kumar said...

I am sure it will help many people. Keep up the good work. It's very compelling and I enjoyed browsing the entire blog.
Business Analytics Course in Bangalore

Courses said...

I like to view your web site which is very useful and excellent resource and truly adored reading your posting. Thank you!
Data Science Course in Gurgaon

Courses said...

Really this article is truly one of the best in article history and am a collector of old "items" and sometimes read new items if i find them interesting which is one that I found quite fascinating and should be part of my collection. Very good work!
Data Scientist Course in Gurgaon

Technical Knowledge said...

You have done a great job and will definitely dig it and personally recommend to my friends. Thank You.
Data Science Online Training

Technical Knowledge said...

Very great post which I really enjoy reading this and it is not everyday that I have the possibility to see something like this. Thank You.
Best Online Data Science Courses

Tech Institute said...

A good blog always contains new and exciting information and as I read it I felt that this blog really has all of these qualities that make a blog.
Data Science Institutes in Bangalore

Anamika said...

Things always happen at the right time and direction through which all the benefit can be done from this.
Massage Service in Delhi

pinoy logo said...

Pinoy Replay is a party for a blueprint of the chance of the web.

izspa said...

good blog body to body spa in Elecrtonic City

IDN Poker said...

link alternatif poker deposit pulsa terbaik di tanah air.

Sruthi Karan said...

Thanks for the Valuable information and it is really helpful information for me. Once again Thank you so much for sharing with us.
Virginia Spousal Support Calculator
Spousal Support in VA

johnhardy said...

Some points you have mentioned about dynamic typing is very useful and helpful. I hope you will share more good updates. Now it's time to avail HYDRATING SUN SCREEN LOTION for more information.

Rubi said...

Thanks for sharing an informative blog. It was really awesome blog. Keep sharing the wonderful article.
strangulation charge in virginia

peter parker said...

Distributed systems and dynamic typing are great tools for creating powerful and reliable applications. Protect youtube channel from hackers They enable seamless integration of distributed components while also allowing for quick development and prototyping. They are also incredibly versatile, allowing developers to create everything from simple web applications to complex enterprise solutions. Truly a great combination of technologies.

Softcrayons said...

"Excellent post! Well-written, insightful analysis with practical examples and engaging style. Informative and enjoyable to read. Looking forward to more content from you. Keep up the great work!" I am a trainer at Softcrayons Tech Solution Pvt Ltd providing Google Analytics Training Noida at an affordable price. Join our training program today to learn from the best in the industry!