
By Bill McColl | Article Rating: |
|
October 18, 2010 08:15 AM EDT | Reads: |
29,808 |

Over the past few years, Hadoop has become something of a poster child for the NoSQL movement. Whether it's interpreted as "No SQL" or "Not Only SQL", the message has been clear, if you have big data challenges, then your programming tool of choice should be Hadoop. Sure, continue to use SQL for your ancient legacy stuff, but when you need cutting edge performance and scalability, it's time to go Hadoop.
The only problem with this story is that the people who really do have cutting edge performance and scalability requirements today have already moved on from the Hadoop model. A few have moved back to SQL, but the much more significant trend is that, having come to realize the capabilities and limitations of MapReduce and Hadoop, a whole raft of new post-Hadoop architectures are now being developed that are, in most cases, orders of magnitude faster at scale than Hadoop.
The problem with simple batch processing tools like MapReduce and Hadoop is that they are just not powerful enough in any one of the dimensions of the big data space that really matters. If you need complex joins or ACID requirements, SQL beats Hadoop easily. If you have realtime requirements, Cloudscale beats Hadoop by three or four orders of magnitude. If you have supercomputing requirements, MPI or BSP
The one area where MapReduce/Hadoop wins today is that it's freely available to anyone, but for those that have reasonably challenging big data requirements, that simple type of architecture is nowhere near enough.
Published October 18, 2010 Reads 29,808
Copyright © 2010 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Bill McColl
Bill McColl left Oxford University to found Cloudscale. At Oxford he was Professor of Computer Science, Head of the Parallel Computing Research Center, and Chairman of the Computer Science Faculty. Along with Les Valiant of Harvard, he developed the BSP approach to parallel programming. He has led research, product, and business teams, in a number of areas: massively parallel algorithms and architectures, parallel programming languages and tools, datacenter virtualization, realtime stream processing, big data analytics, and cloud computing. He lives in Palo Alto, CA.
![]() Apr. 27, 2018 01:00 AM EDT Reads: 3,882 |
By Pat Romanski ![]() Apr. 27, 2018 12:15 AM EDT Reads: 4,026 |
By Yeshim Deniz Apr. 27, 2018 12:00 AM EDT Reads: 1,178 |
By Elizabeth White Apr. 26, 2018 11:30 PM EDT Reads: 7,204 |
By Elizabeth White ![]() Apr. 26, 2018 11:00 PM EDT Reads: 2,935 |
By Elizabeth White ![]() Apr. 26, 2018 10:00 PM EDT Reads: 5,384 |
By Pat Romanski ![]() Apr. 26, 2018 09:30 PM EDT Reads: 4,370 |
By Liz McMillan ![]() Apr. 26, 2018 09:00 PM EDT Reads: 22,894 |
By Elizabeth White ![]() Apr. 26, 2018 09:00 PM EDT Reads: 5,269 |
By Elizabeth White ![]() Apr. 26, 2018 07:45 PM EDT Reads: 6,131 |
By Liz McMillan ![]() Apr. 26, 2018 07:15 PM EDT Reads: 6,092 |
By Liz McMillan ![]() Apr. 26, 2018 07:00 PM EDT Reads: 14,007 |
By Elizabeth White ![]() Apr. 26, 2018 06:00 PM EDT Reads: 4,745 |
By Pat Romanski ![]() Apr. 26, 2018 05:00 PM EDT Reads: 7,357 |
By Liz McMillan Apr. 26, 2018 03:30 PM EDT Reads: 2,871 |
By Pat Romanski Apr. 26, 2018 03:30 PM EDT Reads: 1,625 |
By Elizabeth White Apr. 26, 2018 03:00 PM EDT Reads: 7,907 |
By Elizabeth White ![]() Apr. 26, 2018 02:00 PM EDT Reads: 4,454 |
By Pat Romanski ![]() Apr. 26, 2018 01:15 PM EDT Reads: 9,093 |
By Elizabeth White ![]() Apr. 26, 2018 01:00 PM EDT Reads: 1,418 |