Bigdata SQL: Approaches to Reduce Latency on SQL Queries

The larger the data size and larger the I/O, the longer is the time spent in  scanning the data to get to the right data required to fulfill the query. Much thought, research, and innovation has gone into optimizing the storage layer to build optimizations in reducing the footprint of the data set. Below, we discuss some optimizations that can be performed
at the storage layer, to reduce the I/O.

When thinking about performance improvements, there are three types of performance considerations to keep in mind:

1. Write performance—how fast the data can be written

2. Partial read performance—how fast you can read individual columns within a data set

3. Full read performance—how fast you can read every data element in a data set