At very high level we can say that SQL (structured query language) is by design targeted at structured data only but most of initial applications in Hadoop deal with unstructured data such as text.
Following are some more detailed comparison of Hadoop with SQL databases on specific dimensions:
SCALE-OUT INSTEAD OF SCALE-UP
- Scaling commercial relational databases is expensive because to run a bigger database you will need to buy a bigger machine.
- Hadoop is designed to be a scale-out architecture operating on a cluster of commodity PC machines, Adding more resources means adding more machines to the Hadoop cluster, Hadoop clusters with ten to hundreds of machines is standard.
KEY/VALUE PAIRS INSTEAD OF RELATIONAL TABLES
- In RDBMS, data resides in tables having relational structure defined by a schema.
- Hadoop uses key/value pairs as its basic data unit, which is flexible enough to work with the less-structured data types. In Hadoop, data can originate in any form, but it eventually transforms into (key/value) pairs for the processing functions to work on.
FUNCTIONAL PROGRAMMING (MAPREDUCE) INSTEAD OF DECLARATIVE QUERIES (SQL)
- Under SQL you have query statements; under MapReduce you have scripts and codes.
- MapReduce allows you to process data in a more general fashion than SQL queries. For example, you can build complex statistical models from your data or reformat your image data. SQL is not well designed for such tasks.
OFFLINE BATCH PROCESSING INSTEAD OF ONLINE TRANSACTIONS
- Hadoop is designed for offline processing and analysis of large-scale data. It does not work for random reading and writing of a few records, which is the type of load for online transaction processing.
- Hadoop is best used as a write-once , read-many-times type of data store. In this aspect it is similar to data warehouses in the SQL world.