Wednesday, November 25, 2009

The new Reia: now without rainbow-farting Domo-kuns

Well, what can be said: the buzz about Reia has died down, and several people are confused about the state of the project. Is it going forward? Is it being actively developed? What's this Peridot thing?

I recently attended RubyConf and was asked by many people about the state of Reia. My answer was always "wow, I should really blog about that" to avoid repeating the same things to people over and over (after all, the Ruby community emphasizes DRY, right?). Well, here is that blog.

The State of Reia: Forget Peridot

To put it bluntly, Reia is undergoing a ground-up rewrite. This does not mean I am abandoning the previous codebase. Far from it. Instead I am rewriting the parts of it which are problematic, and pulling in code from the previous implementation where it makes sense.

When I talk to various people who are implementing languages for fun, it seems most people are interested in producing one-off experiments with little attention to developing them into "real" languages someday that might actually be used in production by a lot of people. This certainly makes sense, and that's how I started with Reia. However, buzz grew, and so did my investment in the project. Every indicator I've been given has shown me that Reia is something a lot of people are potentially interested in, so half-assing it isn't going to cut it.

The new Reia roadmap calls for reaching complete feature parity with Erlang with as minimal an implementation as possible, then making it rock solid. At this point, while Reia will lack many of the "dream features" of the previous implementation, it will be generally usable as an alternative to Erlang. Once new language features become available existing programs can be modified to make use of them. After all this is done syntactic sugar can be added, and finally, the concurrent object model.

Initially I had thought of splitting off these additional features into a separate language, which I called "Peridot", but after long and careful consideration, this doesn't make sense. The new Reia will start as an alternative syntax for Erlang (with destructive assignment) but will grow to include all of the features I had originally envisioned.

What Was Wrong with the Previous Implementation?

Why rebuild Reia from the ground up? Can't the existing implementation be salvaged and molded into something usable?

There are a number of problems with the existing implementation. Some stem from my lack of knowledge of Erlang itself when I started. Some of them stem from my lack of knowledge of best practices when writing a language compiler in Erlang. Others stem from the organic way the language evolved. But above everything else, the problems stem from one feature I deemed absolutely necessary: eval.

I like eval lots! If nothing else, it exists for one reason: so Reia can have an interactive shell (i.e. a read-eval-print loop, a.k.a. REPL). I spend lots of my time hacking on Ruby code by interacting with it from irb, the interactive Ruby interpreter. I have a very hard time working with languages that do not provide some form of interactive interpreter nowadays.

The biggest problem with implementing eval is that you have to write your own implementation for your language. In the previous version of Reia I tried to sidestep that by using erl_eval, the Erlang metacircular interpreter, as my eval implementation. Unfortunately, to facilitate this, I ended up implementing the entire code loading process in a way which shoved everything to erl_eval. The result ended up looking something like this:

the previous wonky ass Reia compiler

When code entered the system, it was first parsed and converted to Erlang by reia_compiler (the Domo-kuns). For module and class declarations, the code was compiled down to Erlang bytecode (the rainbow farts) which were in turn injected into the toplevel Erlang AST. In other words, the toplevel scope of any Reia file was never compiled, but simply stored as expressions, and where module/class declarations existed, instructions to load the compiled module (which itself was injected directly into the AST) were issued. This provides somewhat Ruby-like semantics for module/class declarations: they're "loaded" at the time they're declared.

The resulting Erlang AST, complete with compiled class/module fragments, was then shoved through the Erlang metacircular interpreter, erl_eval (seen in the diagram as the tornado). As you might guess, compared to compiled Erlang the metacircular interpreter is reaaaaaaaaally slow.

Once everything was said and done, the resulting class/modules were loaded into the Erlang code server, pictured here as a hungry Joe Armstrong going *nom* *nom* *nom*.

Making Reia Faster


As you may guess, an as-yet-unstated goal of this rewrite is to improve the speed of code-loading. Previously, Reia could not have a large standard library, because it took so long to load code. Furthermore, implementing a mechanism for caching compiled bytecode was impossible due to the API structure.

The new code-loading API directly compiles everything, including any code which is eval'd. This not only makes everything significantly faster but also facilitates the possibility of caching and also various bugs surrounding the eval implementation. From what I've gathered elsewhere, most compiled languages generally ditch any form of metacircular interpreter and implement eval by compiling temporary modules.

Doing this in Erlang is hard, because certain expressions in Erlang create things which exist beyond when code is being evaluated, namely processes and lambdas (a.k.a. funs). This was a vexing problem to me for quite some time, but after talking with Robert "Hello Robert" Virding, one of Erlang's creators, I believe I've come upon a workable solution, even if it's a bit of a hack.

Reia will implement its own "garbage collector" process for eval'd code, which periodically checks if all the lambdas/processes created by a particular eval call are no longer in use. If so, it will remove the temporary module. If not, then it will continue to wait. It is not the greatest solution in the world, but it will get the job done.

This means no Reia code will ever go through erl_eval. Everything is compiled. This will make code loading of all sorts, and eval, much faster. There are no longer any rainbow farting Domo-kuns.

What About Neotoma?

When I originally began my rewrite of Reia, I was attempting to redo the parser using Neotoma, a Parsing Expression Grammar (PEG) tool for Erlang, similar to Ruby's Treetop.

I eventually shied away. This had little to do with Neotoma itself, but my own inability to understand PEGs, and the fact that my own inability to understand them was a roadblock in continued development. Because of this, I switched back to more familiar tools: leex and yecc, the Erlang equivalents of lex and yacc.

Neotoma has come a long way and become better than ever. I am still considering using it. I think it would be a great tool for solving a lot of problems that aren't presently solved, like handling Reia expressions within interpolated strings. This is something I might pursue when I am confident that development otherwise is coming along at a steady pace, but at this point, switching to Neotoma is a lower priority for me than developing a rock-solid core language.

Where's the Code?

If you're interested in checking out the latest Reia codebase, it's available on this branch on Github:

http://github.com/tarcieri/reia/tree/minimalist

If you're looking at master, and wondering why it hasn't been touched in months, it's because I'm hacking on the new branch, not the previous implementation.

The new implementation is not generally usable. I am still working out the nasty details of implementing a compiled eval, as well as implementing cleaner compiler internals.

But hey, if you're interested in Reia, check it out and let me know what you think.

29 comments:

Roger Pack said...

glad to hear about reia. I know the scala guys had some difficulty getting eval to work in a compiled language, perhaps some lessons could be learned there?
Thanks again.
-r

Anonymous said...
This comment has been removed by a blog administrator.
Unknown said...

cctv camera dealers in delhi
cp plus cctv camera online
hikvision camera online
cctv camera installation services in delhi
cctv camera installation services in gurugram
cctv camera installation services in gurgaon
cctv camera installation services in noida
cctv camera security systems
cctv installation services
security systems for home

Sania Kumari said...


Call Girls in Mumbai
Call Girls in Puri
Call girls in Cuttack

Call girls in Bhubaneswar
Call girls in Gurgaon
Call girls in Gurgaon

Archana The best Call Girl said...

Escort services in Mumbai
Escort services in Puri
Escort services in Cuttack

Escort services in Bhubaneswar
Escort services in Gurgaon
Escort services in Gurgaon

Escort services in Jaipur
Escort services in Jaipur
Escort services in Jaipur

Archana The best Call Girl said...

********************************************************************************************************
Hi friends ! I'm Archana Kumari. I like to have bed relationship with different persons rather than with a single man for the whole of life. Not only is this my notion , but also is this fact that the secret organs are made for enjoyment. Please follow me on the links given bellow.

Call girls in Jaipur
Call Girls in Mumbai
Call Girls in Bangalore


Call Girl in Mumbai
Call Girl in Puri
Call girl in Cuttack

Call girl in Bhubaneswar
Call girl in Gurgaon
Call girl in Gurgaon

If you are interested in me and my ideology ,then you are invited to my bed. Just go ahead !
***********************************************************************************************************

Archana The best Call Girl said...

********************************************************************************************************
Hi friends ! I'm Archana Kumari. I like to have bed relationship with different persons rather than with a single man for the whole of life. Not only is this my notion , but also is this fact that the secret organs are made for enjoyment. Please follow me on the links given bellow.

Call girls in Jaipur
Call girl in Jaipur
Call girl in Jaipur

Call girl in Jaipur
Call Girl in Mumbai
Call Girl in Bangalore

Escorts in Mumbai
Escorts in Puri
Escorts in Cuttack

If you are interested in me and my ideology ,then you are invited to my bed. Just go ahead !
***********************************************************************************************************

Anjali Arya said...



Most Indian Men needs a young lady who is hot and in addition clever at the same time. Be that as it may, shockingly most elite girls are exhausting. Their service satisfaction level is so basic, yet can’t strike a discussion making their arrangement lacking something. As of now out Massage Parlor in Delhi They are a great listener, can chat on you on any point and give you such a delicate treatment. Check our other services...
Escorts in Lucknow
Call Girls in Lucknow
Escorts Service in Delhi
Call Girls in Aerocity




customized gifts for him said...

best baby gifts 2020
gifts for her valentines
valentine gifts for him romantic
best christmas gifts for mom
best gifts for dad 2020
best baby gifts 2020
rustic pillows for bed
decorative pillows for sofa
geometric pillow target
linen cushion covers uk

rima said...

siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service



siliguri escorts
siliguri escort
siliguri female escorts
siliguri escort service

Kelly Cain said...

Interpages

Guest Blogger

Guest Blogging Site

Guest Blogging Website

Guest Posting Site

Sajjad Ahmed said...

Nice Blog. Thanks for sharing.

Top Residential School in Dehradun

How Can Working Parents Help Their Children’s Education In School ?

LL said...

http://www.unlimitednovelty.com/2009/04/why-i-dont-like-scala.html

LL said...


There is a new mod that allows multiple new functionalities that were not available originally on the default version.

https://flosshype.com/fmwhatsapp/

Aradhya Mittal said...

With the Relaxable Mount-Abu Escorts service, you will get the best service through this agency. This agency provides exotic Female escort services to each of its clients by the best adorable Call Girls in Mount-Abu, and all the call girls do their best to make each of their clients happy and satisfied with their services.

Samuel D. Darden said...

I'd want to thank you for your time and work in creating this post. I want to learn this Ruby programming language because my game buddy in Granny 3 game ( here the website https://games.lol/granny-3/ for this game) told me that if I want to create unique computer games I should need to learn it. DO you have a free tutorial for this programming language, if you have please attached it here?


desktop version

Tech said...

Augurs Technologies is a Top-class "software development company" in Germany. Providing solutions that meet customer requirements and business expectations. We combine proven techniques, industry knowledge and our technology know-how from experienced software developers to deliver high-quality software solutions. In our work, we particularly value integrity and customer service. We are one of the best software development companies in Germany.
We plan smart, practical, cost-effective and instinctive software development, business process automation, e-commerce solutions for businesses that help streamline business processes and create revenue for startups and established businesses. The experience and know-how we have gained in the industry over the years enable us to provide solid and honest advice to our clients and produce highly effective results.
software development company in Germany

Shivani Singh said...

All the private requirements of the purchasers would be met and at an equivalent time satisfied while they plan to book the services of the Agra Escorts services. One can have the access of those ladies through the web services where their personal profile details are provided along side the pictures. You'd never end up during a confused state of mind while handling these busty individuals. The cooperation that these ladies would offer to you'd make every sensual nerve of you pacified to the premium levels.

R1se Hluoluo said...

Metode awal yang wajib dicoba pemain judi online buat bisa bermain judi bola sportsbook merupakan dengan mendaftar ke dalam agen bola sportsbook terbaik serta terpercaya.

Pemain judi online butuh melaksanakan proses catatan tersebut buat bisa membuat akun judi.

Pemain cuma bisa melaksanakan kegiatan judi bola sportsbook di dalam agen yang memanglah sediakan permainan tersebut. https://www.markasdomino.net/ Untuk para pemain judi pendatang baru, perhatikan opsi agen supaya bermutu serta terpercaya.

casinositeguide.com/카지노사이트 said...


Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man and Thanks for the post,Interesting stuff to read. Keep it up. Feel free to visit my website; 일본야동
일본야동
국산야동
일본야동
한국야동


Anamika said...

Things have changed and there are lots of queries that have solved all the solutions.
Massage in Mumbai

สล็อต พีจี said...

y8  เป็นผู้เผยแพร่เกมและก็ผู้พัฒนาเกม แพลตฟอร์ม Y8 เป็นโซเชียลเน็ตเวิร์กที่มีผู้เล่น 30 pg slot  ล้านคนและก็กำลังเติบโต พีจี สล็อต ออนไลน์กับพวกเราได้ทุกที่ทุกๆเมื่อนิยมได้เงินจริง

สล็อต เว็บ ตรง said...

Slotv9 สล็อตเว็บไซต์ตรง บริการ คาสิโนออนไลน์ สล็อตออนไลน์ ชั้น 1 เป็น เว็บไซต์สล็อตแตกง่าย สล็อตค่ายใหญ่ ใหม่ที่สุด โดยพวกเรารวมเว็บไซต์ pg slot

เว็บ เกม สล็อต said...

เว็บ เล่น เกม pg slot ด้วยระบบความปลอดภัย PG SLOT ที่ดียอดแถมยังเป็นเว็บไซต์ใหญ่ ป้อมอาจจะ มีความน่าไว้ใจโดยการการันตีว่าเว็บไซต์พวกเรานั้นเป็นเว็บไซต์ที่ถูกต้องตามกฎหมาย

John Hardy said...

Yes, I am agreed with this one. Riea is very good and useful for us. I must share this one with my good friends. Now it's time to avail https://quickshutterrepair.co.uk/ for more information.

johns said...

Your blogs are really good and interesting. It is very great and informative. This does not mean I am abandoning the previous codebase. Far from it. Instead I am rewriting the parts of it which are problematic, and pulling in code from the previous implementation where it makes sense bankruptcy lawyers in virginia beach. I got a lots of useful information in your blog. Keeps sharing more useful blogs..

neha said...

If Are you searching for a store that offers cell phone repair in Jonesboro and you are searching for an iPhone repair shop near me then Neha Wireless is the best option for you. we repair phones and sell new Phones at Affordable Prices Read More Visit Website:-

neha said...

if your iPhone is damaged or not working properly. so iPhone repair shop Neha Wireless store is available to serve you to repair your cell phone at Affordable Prices. Choose Neha Wireless Repair Services without hesitation and get your iPhone fixed within 30 minutes.:-

neha said...

Are you looking to buy an iPad or want to repair your old iPad which is broken and the display is gone? best iPad repair shop Neha wireless store is available to fix your iPad with all parts at Affordable Prices Read More Visit the Website:-