Friday, August 10, 2012

Debunking the Node.js Gish Gallop

A programmer who once a Ruby on Rails enthusiast switches to Node.js and thinks it's awesome, then proceeds to write a blog post about why Node is the bee's knees and Rails is crap. Attention is drawn to the changing nature of web design, from web pages with server-generated HTML to single-page JS-heavy apps written using Backbone, Ember, etc. Stop me if you think that you've heard this one before...

This is an argument I keep hearing over and over, and as far as I'm concerned it's nothing but a Gish Gallop of completely specious arguments, but I really worry... I worry because I keep hearing it over and over, and the fact that I keep hearing it over and over makes me worry that people are actually believing it. I don't know why I keep hearing it over and over. I'm not sure if people are running into problems, reading some of the prevailing "wisdom", and coming to the same conclusion or what. This really makes me sad, because whenever I read the posts like this, I do feel my previous passion for these same ideas, but for me that was half a lifetime ago, and my opinions have changed. I have been down these roads, over mountains, blazed my own trails, and then realized how stupid I was...

How do you defeat the Gish Gallop? I don't really enjoy doing this, but as far as I can tell there is no other way: we must go through the arguments one by one and show why they are completely ludicrous. So here we go...

In case you were confused, Rails is AWESOME for JSON APIs and single page applications

I love client-heavy HTML5/JS apps. I don't want every page on the web to be one, but there are many applications that can benefit a ton from keeping all of their state in the browser. In general: if you can do something without having to go across the network to do it, you will provide a better user experience, bar none.

The primary thing these applications crave are awesome JSON APIs (and Websockets... stay tuned). So why should you use Rails for a JSON API? Isn't Rails designed for HTML/JS pages? What benefit does Rails give you for building JSON APIs? And isn't Rails really slow?

Well no, I've been through this before. If you are building API-only applications with a single-page HTML5/JS frontend, you should definitely check out Rails::API. Rails::API completely eliminates any ActionView-centrism you may be worried about in Rails, and gives you awesome tools for building JSON APIs, like ActiveModel::Serializers. But that alone can't express what Rails brings to the table, so here as list of features Rails provides which are useful for JSON APIs, courtesy the Rails::API README:

Handled at the middleware layer:

  • Reloading: Rails applications support transparent reloading. This works even if your application gets big and restarting the server for every request becomes non-viable.
  • Development Mode: Rails application come with smart defaults for development, making development pleasant without compromising production-time performance.
  • Test Mode: Ditto test mode.
  • Logging: Rails applications log every request, with a level of verbosity appropriate for the current mode. Rails logs in development include information about the request environment, database queries, and basic performance information.
  • Security: Rails detects and thwarts IP spoofing attacks and handles cryptographic signatures in a timing attack aware way. Don't know what an IP spoofing attack or a timing attack is? Exactly.
  • Parameter Parsing: Want to specify your parameters as JSON instead of as a URL-encoded String? No problem. Rails will decode the JSON for you and make it available in params. Want to use nested URL-encoded params? That works too.
  • Conditional GETs: Rails handles conditional GET, (ETag and Last-Modified), processing request headers and returning the correct response headers and status code. All you need to do is use the stale? check in your controller, and Rails will handle all of the HTTP details for you.
  • Caching: If you use dirty? with public cache control, Rails will automatically cache your responses. You can easily configure the cache store.
  • HEAD requests: Rails will transparently convert HEAD requests into GET requests, and return just the headers on the way out. This makes HEAD work reliably in all Rails APIs.

Handled at the ActionPack layer:

  • Resourceful Routing: If you're building a RESTful JSON API, you want to be using the Rails router. Clean and conventional mapping from HTTP to controllers means not having to spend time thinking about how to model your API in terms of HTTP.
  • URL Generation: The flip side of routing is URL generation. A good API based on HTTP includes URLs (see the GitHub gist APIfor an example).
  • Header and Redirection Responses: head :no_content and redirect_to user_url(current_user) come in handy. Sure, you could manually add the response headers, but why?
  • Caching: Rails provides page, action and fragment caching. Fragment caching is especially helpful when building up a nested JSON object.
  • Basic, Digest and Token Authentication: Rails comes with out-of-the-box support for three kinds of HTTP authentication.
  • Instrumentation: Rails 3.0 added an instrumentation API that will trigger registered handlers for a variety of events, such as action processing, sending a file or data, redirection, and database queries. The payload of each event comes with relevant information (for the action processing event, the payload includes the controller, action, params, request format, request method and the request's full path).
  • Generators: This may be passé for advanced Rails users, but it can be nice to generate a resource and get your model, controller, test stubs, and routes created for you in a single command.
  • Plugins: Many third-party libraries come with support for Rails that reduces or eliminates the cost of setting up and gluing together the library and the web framework. This includes things like overriding default generators, adding rake tasks, and honoring Rails choices (like the logger and cache backend).
Rails has an unquestionably awesome feature set even if applied exclusively to JSON APIs, and this guy is taking it completely for granted:
"So your Rails server becomes an API, and your web site, like the iOS app, is the client. It's a clean separation of responsibilies, but given what Rails was designed to do, it's like having a horse rider climb on top of an elephant."
The design of Rails, as of Rails 1.2, provided clean abstractions for using the same code to provide server-generated HTML views and "REST" APIs in multiple serialization formats. This was a big deal at the time, and "the time" was 2 years before Node even existed. Fast forward 4 years and Rails 3 has been rewritten with an emphasis on modularization, allowing you to strip out the components you don't use and build lightweight stacks with only the things you need. Rails::API provides convention over configuration for a lightweight JSON-oriented stack.

But let me back up a little bit...
"The view in MVC is not just HTML and CSS; it's the presentation logic, and the presentation logic needs structure. With this need, client-side frameworks like Backbone, Spine, and Ember have come into the picture."
So I hear this guy Yehuda Katz worked on both Ember and Rails. You may have heard of Ember, it just won Throne of JS's framework of choice (Backbone won in the "library" category). But appeal to authority aside, what does using Ember and Rails in combination actually get you?

A problem I am certain you have run into is the manual nature of serializing JSON. Exactly how should you translate from a domain object into a JSON representation? What if the client wants to avoid repeat requests by eagerly loading other domain objects which are associated with the one you want to retrieve and including them in the JSON result? And wouldn't it be great if there were a single canonical representation for all of this that a standardized domain object abstraction running in the browser could automatically consume for us, so we don't have to manually write a bunch of JSON serialization and deserialization logic for everything in our system?

Can we put JSON on Rails? Yes we can: it's called ActiveModel::Serializers and Ember Data. All that glue code you've been writing over and over for serializing and unserializing JSON? Stop that. Seriously. You have better things to do than deal with the idiosyncrasies of whether you should wrap a particular array in an object or return a literal string or number as opposed to an object for future proofing. You are wasting your time with this minutiae and chances are the ActiveModel::Serializers representation is better than the one you are using. Let's take a look at why. 

The defining characteristics of the ActiveModel::Serializers JSON representation is that it explicitly avoids nesting objects within objects, instead preferring to keep the resulting structure flat and using IDs to correlate the relationships between data in the structure. Here is an example of a "post" object which includes comments and tags, taken from the ActiveModel::Serializers README:
  "post": {
    "id": 1,
    "title": "New post",
    "body": "A body!",
    "comments": [ 1, 2 ]
  "comments": [
    { "id": 1, "body": "what a dumb post", "tags": [ 1, 2 ] },
    { "id": 2, "body": "i liked it", "tags": [ 1, 3 ] },
  "tags": [
    { "id": 1, "name": "short" },
    { "id": 2, "name": "whiny" },
    { "id": 3, "name": "happy" }
There are multiple nested relationships in this document: the post has many comments, and comments have many tags. And yet we don't see duplication of comment or tag objects. We don't have to worry about which version of a repeated object is canonical, because there are no repeated objects. Objects within the resulting document are deduplicated and referred to symbolically by their ID. Using this JSON structure we can represent arbitrarily nested relationships between objects in the most efficient manner possible and completely avoid any problems with inconsistencies between duplicated versions of objects present in the document. This representation of JSON just makes sense, and perhaps you too have standardized upon it. Better yet, if you use this representation, then with very little effort on your part Ember Data can automatically consume it.

If you use Ember and Rails, you can abstract away JSON and save yourself the headache of writing custom serialization code. I'm going to say: score one for Rails and single page applications. Maybe you have some Node thing that can do that too, I don't know, but seriously, if you think Rails is bad for JSON APIs, you don't know Rails.

Moving right along, let's continue slogging through the Gish Gallop.

Node has nonblocking async I/O and Rails doesn't so Rails is slow!!!

Where to start with this one. Hmm, let's start here:
"When I think of Ruby and Rails' performance, I think of Ilya Grigorik."
Let me start by saying that Ilya is an awesome guy who has done a very thorough and nuanced survey of the many facets of Ruby performance over time. Taking any single thing he's said out of context and treating it like gospel is probably doing a disservice to Ilya. That said, let's see what thing Ilya said that this guy chose to single out and present out of context. Quoth Ilya:
"There is nothing about node that can't be reproduced in Ruby or Python (EventMachine and Twisted), but the fact that the framework forces you to think and use the right components in place (fully async & non-blocking) is exactly why it is currently grabbing the mindshare of the early adopters. Rubyists, Pythonistas, and others can ignore this trend at their own peril. Moving forward, end-to-end performance and scalability of any framework will only become more important."
So this is a line I hear out of Ryan Dahl a lot too. It's a line I used to believe.

Folks, I've been doing this stuff for awhile. I first discovered synchronous I/O multiplexing when I was about 15, which for me was half a lifetime ago, and since then I've been building network servers using this approach. I've built my own abstraction layers across select/poll/epoll/kqueue. I wrapped libev for Ruby in Rev/ and nio4r, the latter of which is a cross-platform abstraction for Java NIO on JRuby. I cannot express to you how much work I've invested in doing things the evented non-blocking way.

I don't think non-blocking I/O is a good fit for web applications that talk HTTP, although I think it can be a good fit for Websocket applications. I will get to my reasons later. But first, let's continue digging through the Gish Gallop:
"Ilya mentioned the framework/ecosystem that I now consider to be the threat to Rails: Node.js [...] The biggest thing I noticed was the difference in performance. It consumed less memory than Ruby, and it served more requests per second than Sinatra or even Rack."
I have a huge pet peeve, and that's when people talk about performance without numbers. I tried it and it was faster. I tried it and it was slower. If you really want to make a point about the performance of a particular thing, can you at least pretend you're using science?

I hate to do this, but I think I have to destroy your god. Let's see how Ilya's software stacks up to mine on a crappy "hello world" web server benchmark. First, the numbers for my web server Reel:

# httperf --num-conns=50 --num-calls=1000

Ruby Version        Throughput    Latency
------------        ----------    -------
JRuby HEAD          5650 reqs/s   (0.2 ms/req)
Ruby 1.9.3          5263 reqs/s   (0.2 ms/req)
JRuby 1.6.7         4303 reqs/s   (0.2 ms/req)
rbx HEAD            2288 reqs/s   (0.4 ms/req)
Let's compare to Ilya's web server Goliath, as well as Thin and Node.js:
Web Server          Throughput    Latency
----------          ----------    -------
Goliath (0.9.4)     2058 reqs/s   (0.5 ms/req)
Thin    (1.2.11)    7502 reqs/s   (0.1 ms/req)
Node.js (0.6.5)     11735 reqs/s  (0.1 ms/req)
All of these servers, including mine, are using non-blocking evented I/O. Is that remotely relevant? No. That's just a coincidence.

My web server is faster than Ilya's. So by Gish Gallop logic, Ilya must be wrong about everything. There must be no reason to use Ilya's web server. Let's write everything in Node since it won the benchmark.

There's a huge problem here: Goliath does things that Reel, Thin, and Node's HTTP server don't do. The reason it's slower isn't because Ilya sucks and is clueless about performance. The reason is that Goliath has features which these other web servers don't, which makes it an apples to oranges comparison. (I guess scumbag me for putting them all in a big list on the Reel web page)

The same can be said of Rails: it probably isn't ever going to have better latency through the entire stack  than any Node.js framework, but the latency of the Rails stack is probably going to be a lot less than your application logic, and that's still going to be a drop in the bucket compared to the network latency to a given user.

Celluloid solves every single problem you're whining about better than Node

Node has a lot of problems, and I'm not just talking about the audience it attracts. Let me start by saying this: many of the things I have built in Celluloid are based off of technologies originally developed for Node. My web server Reel uses the Node HTTP parser, and it's quite likely that the next iteration of nio4r I develop will be based off of libuv.

All that said, let me start with Node's fundamental problem: callback-driven I/O. Celluloid::IO is one of many systems, including Erlang and Go, that demonstrate that "nonblocking" and "evented" I/O are orthogonal to callbacks. Celluloid uses Ruby's coroutine mechanism to provide a synchronous I/O API on top of an underlying nonblocking system. However, where systems like Node force you to use nonblocking I/O for everything, Celluloid lets you mix and match blocking and nonblocking I/O as your needs demand.

If you have ever worked in a language like C(++) or Java, you probably know an amazing property of sockets: you can mix and match blocking and nonblocking I/O, even over the lifecycle of a single socket. Perhaps you will handle incoming sockets in a nonblocking manner at first, but if they make a complex request, you might change the socket to a blocking mode and hand it off to a worker thread.

Celluloid::IO makes this handoff completely transparent: simply by giving the socket to another Ruby thread which isn't a Celluloid::IO actor, it will automatically switch from nonblocking to blocking mode completely transparently.

But let's talk about Node's real fundamental problem, one that is extremely difficult to solve in any callback-driven system: flow control. Unfortunately the Node.js community has adopted the phrase "flow control" to mean "building abstractions around managing callbacks", however the phrase "flow control" has a very specific definition relating to the rates at which data is transmitted between systems.

In general, callback-driven systems can't manage flow control effectively. The most notable pathological case is the producer-consumer problem, whereby a slow consumer might force a system like Node to unboundedly buffer data from an unchecked producer. There's a clear and simple solution to this problem: make all I/O synchronous. Using coroutines that provide blocking-style APIs, you can easily compose producer/consumer problems in a manner that doesn't result in unbounded writes to a buffer, because simply by virtue of a virtual blocking API, the rate at which data is transfered from producer to consumer is kept in check.

But what about WebSockets?

Ruby has had some pretty awesome albeit overlooked and therefore stagnant solutions for WebSockets for awhile, like Cramp. I've been working on web-based push technologies for half a decade now, and explored a multitude of solutions including Comet, XMPP/BOSH, RabbitMQ long polling, and my own XHR long polling systems which I originally built around *gasp* threads nearly 3 years ago at this point.

Well, I'm quite happy to say that Reel now supports WebSockets. I certainly don't want to say that my recent spike is anywhere as mature as WebSockets in Node or their surrounding ecosystem. Instead, I think the API that Reel provides for WebSocks is simply better by design. If you managed to catch tenderlove's recent blog post on streaming live data, you may understand that all previous APIs you may have encountered in both systems like Rails or Node for streaming data were really obscuring the one API that truly makes sense for this use case: a socket.

WebSockets are in many ways similar to 0MQ sockets (which are used in DCell via Celluloid::ZMQ). WebSockets provide a framing mechanism which provides a message-based transport instead of the typical stream-based transport provided by TCP. That said, when processing message sequences, callbacks become extremely problematic, because you must reconstruct the state of the current request from the point of each incoming message. Callbacks work well for e.g. a chat protocol where there is no state relationship between messages, but as soon as there is you are effectively stuck building a finite state machine to manage the processing of each incoming message.

This is madness. There's a much better and much more straightforward solution to this problem: just use the goddamn stack. In order to do so, you need to provide a "blocking" API, but this isn't orthogonal to using nonblocking I/O. Celluloid::IO, Go, and Erlang all let you build concurrent, multithreaded, and potentially multicore systems on top of coroutines spread across multiple native threads.

That said, native threads are cheap nowadays and they're only getting cheaper. On most Ruby VMs a native thread will cost you about 20kB of RAM. If you want you can just build blocking I/O systems completely out of native threads without using any sort of evented I/O, and these systems can scale up to tens of thousands of connections.

Don't believe the hype

Node provides a limited subset of what Ruby can do, and it can be done better with Ruby. Node does not have a web framework of the same caliber as Rails. Node doesn't have threads, which in Ruby will spare you from Node's callback soup. Finally, there's the elephant in the room: JavaScript is a terrible, terrible programming language compared to Ruby. We're forced to use JavaScript in the browser, but on the server, we can choose the best language for the job.

Ruby on Rails remains the best-in-class web framework, and while there are arguments to be made against it, the ones I hear coming out of confused Node.js detractors do not hold water.


Haora said...

Even though I agree with you regarding your arguments on favor or Rails, I would do without your entire paragraph.. saying that "Ruby on Rails remains the best-in-class web framework" is a bit of a HUGE generalization, you're implying that it's the silver bullet of web frameworks.. that's never good.
Also, your hate for Javascript, is it really justified? I have a limited experience on the web, since I've only been working with it for the last 10 years (almost), but Javascript is a language that has received a lot of hate from people who don't really take the time to learn it properly... are you one of them?

Other than that, it was a great post, and the work you did to debunk all the gish gallop was great, keep it up and keep posting.

Xagile said...

Suggesting that the reason to use Node.js is performance is bullshit. I agree with you there.

However, I happen to like JavaScript about as much as I like Ruby, and there are advantages to working in the same language on both sides: like you can share modules, and you only have to know one language. Plus, if you really don't like JavaScript you can use CoffeeScript pretty much everywhere with just a few npm modules.

I think that Rails is an excellent platform and I plan to try out some of the other things you suggested. However, I also think that Node.js is pretty useful. It just needs some time to mature.


SubStack said...

In general, callback-driven systems can't manage flow control effectively. The most notable pathological case is the producer-consumer problem, whereby a slow consumer might force a system like Node to unboundedly buffer data from an unchecked producer.

It seems that you've never heard of streams in node. The great thing about streams is that they can handle the tcp backpressure problem very elegantly. Just do:


in an http server handler and the response handler will tell the readable stream to slow down if the remote client can't download the data fast enough.

The streaming api is all built with callbacks so I find your assertion that callbacks can't handle the backpressure problem to be really odd considering how obsessed the core node team is about doing backpressure correctly.

It will be even easier to write streaming abstractions with correct backpressure behavior in node 0.9.

MoJava Cafe said...

"However, where systems like Node force you to use nonblocking I/O for everything, Celluloid lets you mix and match blocking and nonblocking I/O as your needs demand."

Node does not force one to use non-blocking IO for everything.

Tony said...

@SubStack: The Streams API is great, except not everything uses it, thus you encounter the problem of inconsistent I/O semantics... the very problem Node was supposed to solve by making everything nonblocking.

@MoJave Cafe: sure, except you don't have additional threads to perform I/O in like you do in Ruby, so if you block, you lose concurrency and block the entire process.

Ege said...

You can write blocking code and put it on a forked process.

Node.js projects are easy to start with and becoming easier to maintain as the platform itself and the frameworks on top of it matures while being flexible enough.

Kevin McCaughey said...
This comment has been removed by the author.
Xagile said...

I think Rails is ideal for interacting with objects that are backed by structured data. There are all kinds of applications for that. But there are times when you don't need that or want something different. I like Sinatra a lot too.

Roger Pack said...

I would says rails forever if it weren't so very slow...

Tony said...

@Ege with multiprocess systems you run afoul of one of two problems:

1) Serialization is expensive
2) Multiprocess shared memory is hard, ugly, broken

Regarding #1, if you can avoid crossing process barriers and stay within a shared heap, or manage your own memory in the same process, you have the fastest solution available.

Regarding #2, the semantics of SysV shared memory systems are complicated and ugly. I know there are a lot of fans of e.g. python multiprocessing but that's significantly more complex than using threads + Jython (or JRuby)

Peter Moberg said...

Great article! Totally agree that choosing Node.js for performance reasons is wrong.

My biggest problem with Node is callback driven way you have to write your applications. It really obfuscates the code and makes it unnecessary complicated to read and understand the code.

I blogged about it a while back:
Callbacks are polluting your code

Gui said...

Pretty much all the bad things you said about Node.js are wrong.

- JSON serialization is a reason Rails kicks ass. Really? Pretty much any framework can do all the things you mentioned in countering the first "argument" against Node (and that argument has nothing to do with the comparison between Node.js and Rails, btw).

- Performance of node.js is better? Yeah, all your arguments actually confirm that. That is so contradictory if we remember the title of this amazing post.

- "All that said, let me start with Node's fundamental problem: callback-driven I/O." That just has nothing to do with reality. Node.js actually allows you not to worry about the stupid concept of flow control. And even if you want to do that foolishness, Node.js will allow you to do that PRETTY EASILY.

You should be ashamed of yourself and delete that post. For some unbelievable reason your post ranks well at Google.... and it's because of clueless people like you that more and more people give Rails up and choose Node.js.

Guess you just have no clue about programming fundamentals. Here's one: REST is analogous to a messaging system. So just worry about the GODDAM MESSAGES... oh, and btw, messages are async.

Pastor Joe said...

Meh. Those who think JavaScript is a terrible language typically don't understand the difference between OOP and functional programming. They are two entirely different paths.

The author would do well to stick to his very well stated arguments in favor of Rails. Overtly generalized mud slinging does nothing to move the web forward. It only portrays the author as ignorant when he is not; and divides the larger web community which thrives on openness and collaboration.

In short fanboys kill innovation.

a ji o ji suno ji said...

LOL,they are cute my friend! One Pugs is one of my daughter's FAVorite breeds (the rest of us here likes em too ;) ). Moonlight (the Grand Daughter of Spot the Wonderdog,an Austrian Shepard we still love and miss) is the same way when one of us gets in (especially one of the kids from school).
german Shepherd Puppies For Sale

Mark Selby said...

Creating API's in Rails is a terrible idea unless performance, thus user experience and server costs don't matter.

ActiveRecord is the biggest problem. In many use cases using find_by_sql will yield a 10x speedup - and there's no reason not too for API's - this endpoint serves this data should always be achievable in one query.

A lot of the movement to NoSQL databases is the result of non SQL knowing people using ORM's to access databases inefficiently. Trouble is, you really begin to lose the structure to the data, and then the client code becomes more and more *if's* and *then's*.

Writing API's in Node is really easy - get this data and send it out.

"Callback hell" is just developers who are trying to apply a procedural mindset to evented code. In practice, it makes the code much more modular and maintainable. As soon as you hit your "callback hell" you realise that you're doing it wrong and that a module is needed.

Do this, let me know when you're done. Nothing hard about that.

The trick to all of it is to get your data right. Get the data right and much of the server code isn't needed. Unfortunately Rails works against that mindset, even encouraging people *not* to learn things like efficient querying over "we must do everything in Ruby and add more and more servers."

Been there, seen it, done it.

If you aren't comfortable with Javascript, what sort of web developer are you anyway?

Caner Çakmak said...

In the coming versions of Rails they should provide us easier and faster ways to interact with js code by using ruby. We all use coffescript and love it but it's still js. That may be something like Opal. Something that enables use to play with client side using just ruby.

Tech is advancing and I feel myself *uckin inclined to use JS alternatives. But I don't want it! I don'T want to use more and more JS! It's 2014 and yes we know many are switching to Node for several reasons. It's obvious that Rails is losing it's popularity and JS may takeover however the magic wasn't Rails and won't be.

The magic is Ruby. I've never heard a person not liking ruby but heard many hating js!

So If we are forced to use js we have to sort it out. And we have to sort it out efficiently and effectively. To that day I'll continue writing Ruby.

Thanks for this great post

Fire Dragon said...

Làm sao để gửi hàng đi miền tây? Nếu đây là điều bạn đang thắc mắc thì hãy đến với chúng tôi. Chúng tôi là công ty chuyên nhận vận chuyển hàng. Các dịch vụ của chúng tôi hiện đang được rất nhiều ủng hộ. Và đây là những dịch vụ tiêu biểu được nhiều sử dụng của chúng tôi: giao hàng nhanh, giao hàng nhanh tphcm, nhận ký gửi hàng hóa, dịch vụ giao hàng thu tiền cod, ship hàng nội thành, gửi hàng về miền tây, chuyển hàng về đà nẵng, dịch vụ chuyển hàng. Nếu bạn đang cần vận chuyển hay sử dụng dịch vụ giao hàng nội thành hãy liên hệ với chúng tôi nhé.

Nhà Đất Xinh said...

Nếu bạn đang muốn đăng tin bán nhà hay bán đất hoặc bạn muốn mua nhà hay đất thì hãy đến với chúng tôi rao vat mien phi, với chất lương hàng đầu chúng tôi sẽ giúp các bạn , đăng tin và xem các khu vực nha dat quan go vap, ban dat quan 9, nha dat quan thu duc , nha dat quan binh tan , nha dat quan tan phu , nha dat quan tan binh và các khu vực khác trên toàn quốc với uy tín và hiệu quả cao khi bạn đến với chúng tôi.

Toeic Ece said...

công ty Song Đing cung cấp dịch vụin bang ron tai tphcm
với dịch vụin hiflex
cao cấp được nhiều người tin dùng.Đặt biệt chúng tôi còn cung cấp dịch vụtreo bang ron tphcm

Toeic Ece said...

công ty Song Đing cung cấp dịch vụin bang ron tai tphcm
với dịch vụin hiflex
cao cấp được nhiều người tin dùng.Đặt biệt chúng tôi còn cung cấp dịch vụtreo bang ron tphcm


You can watch & download drama online for free at DRAMAFIRE. We update full korean drama, hongkong drama... engsub

Bui Son said...

Chiến lược được hoạch định theo quan điểm phát triển là đi thẳng vào các công trình hiện đại, kết nối giữa các lĩnh vực van tai container và kết nối quốc tế, giảm chi phí gui hang ve Ha Noi. Mục tiêu là vận tải phải đáp ứng được phát triển kinh tế, trong đó có chú trọng xuất nhập khẩu. Song, chúng tôi đang gặp rất nhiều thách thức”, Thứ trưởng Đông nói và cho biết, trước hết là thách thức về huy động nguồn lực phát triển kết cấu hạ tầng giao thông để chuyen hang den Thanh Hoa, trong điều kiện nguồn vốn ngân sách đầu tư rất hạn chế. Đến 2020 nhu cầu phát triển các dự án ưu tiên cần khoảng 50 tỉ USD. Tuy nhiên, vốn có thể lượng chuyển hàng từ Huế được mới khoảng 30%.

Thách thức thứ hai là phải giảm được chi phí vận tải, trong khi chi phí chuyển hàng đi Bình Dương đang quá cao làm ảnh hưởng đến cạnh tranh, xuất nhập khẩu. Theo đánh giá của WB, chi phí logistics của Việt Nam hiện chiếm trên trên 20% GDP, trong đó chi phí chuyển phát nhanh đến Mỹ chiếm 50-60%. "Về lĩnh vực này, chúng tôi cho rằng phải tăng cường kết nối vận tải, đẩy mạnh phát triển các vận tải chi phí thấp, chẳng hạn như Hàng hải, Đường thủy nội địa. Gần đây, Bộ GTVT có đề xuất với WB hỗ trợ lập báo cáo hàng năm về logistics và cơ sở dữ liệu phục vụ cho kết nối các phương thức chuyển hàng nhanh đi Singapore.

Đây là các thông tin quan trọng giúp cơ quan quản lý nhà nước hoạch định và giám sát thực hiện chính sách liên quan đến GTVT, thương mại và nâng cao năng lực cạnh tranh dịch vụ chuyển hàng nhanh sang Đài Loan, cũng có thể đưa lên cổng thông tin điện tử để giúp các doanh nghiệp định hướng trong phát triển kinh doanh, đây cũng chính là những vấn đề mà cong ty van tai noi dia chúng tôi hiện đang yếu", Thứ trưởng nói

hai dao said...

Hiện tại tôi đang hoc nau an cũng như là học pha chế tại TPHCM ở trường NETSPACE. Khá là tốt nên nếu bạn nào có ý định thì đi học chung nhé

Giải Trí said...

Mình cũng đồng tình với suy nghĩ này, cảm ơn bạn đã chia sẽ.

Bạn đang làm bên lĩnh vực gì, mình có thể hợp tác chứ?

Bên mình chuyên về lĩnh vực van tai hang hoa, cung cấp dịch vụ gui hang nhanh di Ha Noi, gửi hàng ra Quảng Ninh giá rẻ, xe gui hang ra Bac Giang, gui hang ra Lang Son gia re, gui hang di Thai Nguyen gia re, xe gửi hàng đi Bắc Cạn, gửi hàng ra Cao Bằng giá rẻ, gui hang nhanh ra Tuyen Quang.

Nếu bên bạn chuyên về lĩnh vực thang máy gia đình Mitsubishi, thang may tai khach mitsubishigiá thang máy bệnh viện thì bên mình sẽ chiết khấu tốt cho bạn nhé.

Đình Tâm Lê said...

cách làm bánh tráng trộn
cách làm bánh bông lan
trân châu nhân cùi dừa
cách làm pizza trứng
cách làm bánh bao nhân thịt
cách làm bún đậu mắm tôm
cách nấu sườn xào chua ngọt

Trần Ly Ly said...

sức khở mỗi ngày mai
tin tức xã hội
tạp chí chuyện xã hội
tin tức thẩm mỹ viện
tin tức cuộc sống hàng ngày
nâng mũi ở thu cúc có đẹp không
giá thu nhỏ đầu mũi ở thu cúc
nâng mũi ở thu cúc giá bao nhiêu
thu nhỏ đầu mũi ở thu cúc

ThanhPhong Land said...

chung cu the golden palm
chung cư the golden palm
du an the golden palm
dự án the golden palm
can ho the golden palm
căn hộ the golden palm
mat bang the golden palm
mặt bằng the golden palm
the golden palm
golden palm
bang gia the golden palm
bảng giá the golden palm
chu dau tu the golden palm
chủ đầu tư the golden palm
golden palm le van luong
thông tin nhà đất
mua chung cu the golden palm
golden plam lê văn lương, mua
mua chung cư the golden plam
chung cu le van luong
chung cư lê văn lương
golden palm hdi
golden palm ceninvest

ThanhPhong Land said...

mat bang toa A5 an binh city
mat bang toa A6 an binh city
mat bang toa A7 an binh city
mat bang toa A8 an binh city
mặt bằng tòa A1 an bình city
mặt bằng tòa A2 an bình city
mặt bằng tòa A3 an bình city
mặt bằng tòa A4 an bình city
mặt bằng tòa A5 an bình city
mặt bằng tòa A6 an bình city
mặt bằng tòa A7 an bình city
mặt bằng tòa A8 an bình city

ThanhPhong Land said...

tổng quan dự án an bình city
gioi thieu an binh city
giới thiệu an bình city
bang gia an binh city
bảng giá an bình city
tien ich an binh city
tiện ích an bình city
mua chung cu an binh city
mua chung cư an bình city
mat bang toa A1 an binh city
mat bang toa A2 an binh city
mat bang toa A3 an binh city
mat bang toa A4 an binh city

ThanhPhong Land said...

chung cu an binh city
chung cư an bình city
du an an binh city
dự án an bình city
can ho an binh city
căn hộ an bình city
mat bang an binh city
mặt bằng an bình city
an binh city
an bình city
an binh city pham van dong
an bình city phạm văn đồng
chung cu pham van dong
chung cư phạm văn đồng

chu dau tu an binh city
chủ đầu tư an bình city
vi tri an binh city
vị trí an bình city
tong quan du an binh city

ThanhPhong Land said...

Đèn Chiếu Sáng
den chieu sang
den led
đèn led
đèn pha led
đèn pha led
den pha led
den duong led
den duong led
chóa đèn
choa den
choa den chieu sang
chóa đèn chiếu sáng
cột đèn
cột đèn trang trí
cột đèn chiếu sáng
cột đèn sân vườn
đèn rainbow
đèn maccot
đèn master
đèn ps05
day cap dien
dây cáp điện
cáp đồng
cáp đồng ngầm
cáp đồng treo
cáp nhôm
cáp nhôm vặn xoắn
dây nhôm trần
dây đồng trần
đèn cao áp
den cao ap
bóng đèn cao áp

Duong Qua said...

Cùng đồng môn với nhau rồi, bên bạn có van chuyen hang ghep di Ha Tinh, van chuyen hang ra Nha Trang, tim xe van chuyen hang hoa Khanh Hoa không. Bên mình có lắp đặt thang máy Mitsubishi 250kg, lap dat thang may tai khach benh vien, lap dat thang may nang hang cần giá chuyển hàng từ sài gòn ra Vinh để cung cấp cho dự án giu xe thong minh, vậy bên bạn có cuoc van chuyen duong bo rẻ không?

John Alert said...

JavaScript Training in CHennai

JavaScript Training in CHennai JavaScript Training in CHennai JQuery Online Training JQuery Online Training

John Alert said...

Enjoyed your approach to explaining how it works, hope to see more blog posts from you. thank you!

Angularjs Training in Chennai | Backbone.JS Training in Chennai | Bootstrap Training in Chennai | Node.js Training in Chennai | Typescript Training in Chennai

John Alert said...

AngularJS Training in Chennai AngularJS Training in Chennai Node.js Training in CHennai Angular 2 Training in Chennai Angular 2 Training in Chennai Node.js Training in CHennai Node.js Training in chennai MEAN Developer Training in Chennai

EmberJS Training in Chennai EmberJS Training in Chennai KnockoutJS Training in Chennai KnockoutJS Training in Chennai
BackBoneJS Training in Chennai BackBoneJS Training in Chennai

Kits Online Training said...

Fantastic article post.Really thank you! Awesome.
Sap Fiori Implementation and Configuration

Sap Tutorial

SCCM 2012 Online Training

Selenium Testing Tool

SQL Server DBA Tutorial Pdf

Tableau Tutorial

Automated Testing Tools

Essay Writing Service Reviews said...

Excellent post about new and relevant topic. Programming and techno logic field have great demand in the recent days it is essential to discuss about it. Research paper writing service reviews also plan for this type of discussion.

jatin tyagi said...

Very good write-up. I definitely appreciate this website. Continue the good work!
Devops Online Training
Adobe cq5 Training
Dell Boomi Training

Huyana Tran said...

tại địa bàn TP Hồ Chí Minh, ViettelCargo chuyên cung cấp dịch vụ chuyển phát nhanh trên toàn bộ các quận huyện: chuyển phát nhanh tại quận 1 TP Hồ Chí Minh chuyển phát nhanh tại quận 2 TP Hồ Chí Minh chuyển phát nhanh tại quận 3 TP Hồ Chí Minh chuyển phát nhanh tại quận 4 TP Hồ Chí Minh .
chuyển phát nhanh tại quận 5 TP Hồ Chí Minh chuyển phát nhanh tại quận 6 TP Hồ Chí Minh chuyển phát nhanh tại quận 7 TP Hồ Chí Minh .

Kisk Joko said...

Giám đốc đảm trách Nghiên cứu CBRE tại Singapore và khu vực Đông Nam Á, Desmond Sim nhận định: “Hiện vẫn còn phổ quát rào cản và hạn chế vận chuyển hàng về Bắc Kạn mà những nước trong khối ASEAN cần phải vượt qua”. đầu tiên là khả năng điều hành yếu kém của chành xe vận chuyển đi Thanh Hoá, nguồn cung nở rộ sở hữu thể dẫn đến biến động giá thuê mặt bằng cong ty van chuyen di Bac Giang. Điều này sẽ gây bê trễ hoặc thậm chí ngăn cản việc mở rộng của các nhà bán sỉ.

Thứ hai là việc thiếu nguồn lao động với tay nghề cao cũng là thách thức, với thể gây trở ngại trong việc vận tải hàng hoá Hà Nội, vận chuyển hàng đi Bắc Ninhvan chuyen hang ra Cao Bangchuyen hang di Dien Bien Phu và mở mang của các nhà cung cấp công nghiệp có giá trị cao.

Thứ ba là sự chênh lệch lớn về chuyên môn giữa các nước thành viên cũng giảm thiểu những tác động hăng hái của buộc phải tự do hóa cần lao ASEAN.

Cảm ơn bạn nhiều nhé.

Tibco Online Training said...

Nice post..
AWS Online Training

Salesforce Online Training

Võ Minh Tú said...

Hướng dẫn cách làm sữa chua

Anand raj said...


Anand raj said...
This comment has been removed by the author.
Anand raj said...

Well! Job Oriented Course with internship

Phương Thảo said...

Great idear. your sharing is helpful to us. we hope more article from you.
vận chuyển hàng hóa Bắc Nam

Unknown said...

I’m in love with this web site. I have visited this site so frequently.
I discovered this web site on the internet. I have gotten a good stuff of information. Many thanks. keep updating me..
Acer Contact Number

Email Support Phone Number

Unknown said...

It's an uncommonly nice post. Genuinely refreshing to examine it till the end thanks divide such a wonderful post

Windows Support Number UK

PrinterTechnical Support

Unknown said...

Great article you can go through on my blogs PEGA Online Training
Workday online training

Unknown said...

Nice post. Thanks for sharing this.Keep it up.We provide Hotmail and Outlook Customer Service.
if you want any solution related to Hotmail and Outlook than visit here:
Hotmail Support Number

Avinash Kanugula said...

What is Trending Now:
Xiaomi Redmi 4 Smartphone | Specifications | Price | Features
8 Interesting Uses of AirPlane Mode Probably you Don’t Know
Sony Xperia XZ Premium Launched In India
10 Synthesis Useful Office Software on Android
Do you Take Notes to Study? Science has Good News for you

Sumit Yadav said...

This post shares some good quality content I want to know from where do you get all this information. And thanks for sharing it here.
Dell Customer Support Number

Gmail Support said...

This Post has some good quality content which is a treat to read out.
Gmail customer support

Router Support said...

nice post.
Tenda Router Customer Support Number

Sarah jack said...

I am feeling so cheerful to examine your blog post,really dazzling post you have shared and I got so many information from your blog,Thanks
Lenovo Support Number

Hotmail Account Support Number

Faheem said...

pest control services
termite control services

pest control services in gurgaon
pest control services in gurugram

pest control services in delhi

pest control services in noida

pest control services in east delhi
pest control services in north delhi
pest control services in south delhi
pest control services in west delhi

Anosh Alassi said...

شركة تنظيف بأبها
شركة تنظيف فلل بأبها
شركة تنظيف منازل بأبها
شركة تنظيف شقق بأبها
شركة تنظيف مجالس بأبها
شركة تنظيف موكيت بأبها
شركة تنظيف كنب بأبها
شركة تنظيف سجاد بأبها

Aliya said...

Surveillancekart | CCTV System
Pestveda | Pest Control
Termite control
cctv installation services
best home security camera system

not responding said...

I love devotinal songs. You have shared a very good quality list of new songs .I really like at. Thanks Google

Indra Dhawa said...

I like the bloger page. Outlook Support

Indra Dhawa said...

Are You looking for best Outlook service?
Outlook Support Number

Fuzzy International said...

I am delighted to see this post.Thank you for Sharing!
cashew nuts supplier in Dubai
A4 paper supplier in Dubai

Unknown said...

I am feeling so cheerful to examine your blog post,really dazzling post you have shared,Thanks

visit here

visit here

visit here

visit here

visit here

visit here

visit here

visit here

visit here

visit here

Tilly Taylor said...

Hello there, just became aware of your site via Google, and discovered that it’s truly useful. I’ll be happy if you continue this in future.

visit here

visit here

visit here

visit here
visit here

visit here

visit here

visit here

visit here

visit here

shina said...

cctv camera dealers in delhi
cp plus cctv camera online
hikvision camera online
cctv camera installation services in delhi
cctv camera installation services in gurugram

Saanvi Sharma said...

This is also a very good post which i really enjoyed reading. It is not everyday that i have the possibility to see something.
Female Escorts Services in Gurgaon
Affordable Escorts Services in Gurgaon

kiya said...

Thank you for your post, I look for such article along time, today I find it finally. This post gives me lots of advice it is very useful for me.
Independent call girl

Mehar Kaur said...

I look at this write-up definitely about the being similar to latest and former technological innovation, it’s amazing content. Hot Female Escorts in Gurgaon Minna | Escort Service in Gurgaon By AntaKusuma |Nisha Roy Independent Escort in Gurgaon |Top Escorts Service in Gurgaon, Gurgaon Call Girls |Gurgaon Comapanion Service By Rimjhim Khanna | |Independent Gurgaon Escort Service |Keira Gurgaon Call Girls | |Beautiful Call Girls in Gurgaon

Ajay Bhati said...

A debt of gratitude is in order for sharing so implication article with us. I concur with your thought completely.I am anticipating another awesome article from you.

Mehar Kaur said...

Hi Guys I have best classified two ads. this is very popular ads in Delhi and Gurgaon. On this ads get more visit peoples. thank you so much for visit with you. visit my classifieds ads: Free Green Classified Ads, Latest Free Classified Ads 2018 | Jabraa Classified Ads, New Classified ads 2018

shagufta kanwal said...

COREL DRAW X8 Keygen + Crack
Corel Draw x8 Keygen is truly well known and worlds most excellent graphics software. Corel Draw x8 Keygen can create your pictures amazingly with greatest editing tools and latest supportive stuff.

shagufta kanwal said...

you can dowmload here more and more..........
ccleaner crack
CCleaner is the number-one tool for cleaning your Windows PC. It protects your privacy online and makes your computer faster and more secure....

Aurora HDR 2018 With Full Crack
Aurora HDR 2018 is a handy photography editor which has got some advanced tools for creating some stunning portraits as it relies on the innovative technology.

Wow, Its really wonderful to read this article,you can dowmload here more and more
Window 10 pro product licence key
Window 10 pro product licence key operating system brings you the most innovative tools and security features that will not only make your device secure but also enables you to perform your daily task in most creative and productive way....

shagufta kanwal said...

Wow, Its really wonderful to read this article,you can dowmload here more and more..........
ICare Data Recovery Pro 8 crack
ICare Data Recovery Pro 8 crack is the world’s most popular software which can recover all deleted files, documents, photos, mp3 and zip files and damaged disks. Besides this, it can restore data after formatting hard disk drive, external hard drive, memory card and USB flash etc.

Wow, Its really wonderful to read this article,you can dowmload here more and more
Auslogics boostspeed key
Auslogics boostspeed key is another great program for optimizing your computer. Naturally, you can remove garbage from hard disks and the system registry, optimize installed applications on your computer, work with hidden Windows settings and so on.

Wow, Its really wonderful to read this article,you can dowmload here more and more
Microsoft office 365 product key
Microsoft Office 365 is a complete set of tools and softwares present in an Office setup file. This is just the combined version of some online as well as offline office tools by Microsoft. Just in case of this archive, It is combined Microsoft office 2013 and everything in it, plus some online libraries and synchronization features added into one bundle.

ijazz jazz said...

Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.


ijazz jazz said...

Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.



I believe there are many more pleasurable opportunities ahead for
individuals that looked at your site.

aws training in bangalore

aws training in chennai

Devayani Kaur said...

I really experienced studying this useful material, big fan. Keep up amazing execute and please tell me when can you post more material or where can I research more on the subject..
Roshni Khanna |
escort in dwarka |
aerocity escorts |
gurugram escorts service |
female escorts in aerocity |
independent escorts in gurgaon |
delhi escorts

Amar G said...

artificial intelligence training in bangalore

Delhi Escorts Services said...

gurgaon escorts
escort in gurgaon
independent delhi escort
independent escort in Delhi
model escorts in delhi
escorts in delhi
Goa Escorts
Escorts in Goa
Escorts in mumbai
mumbai escorts

dalip ku said...

this is very nice blog, is new directory of movers and packers gurgaon and we have some also Ltd company of packers and movers in gurgaon

Tania Sharma said...

Jaipur Escorts
Sexy Hot & Nude Photo
Delhi Escorts

keith kristoper said...

Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.
python training in bangalore