3.1. General

Version 2.1 by Wai-Tim Ng on 2018/07/31 07:21

Index

  • Big data
  • Data cube
Big data is data sets that are so voluminous and complex that traditional data-processing application software are inadequate to deal with them. Big data challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy and data source. Data sets grow rapidly - in part because they are increasingly gathered by cheap and numerous information-sensing Internet of devices such as smart phones, Earth observation sensors, software logs, etc.
A data cube efficiently stores a multi-dimensional array of values. Typically, applied in contexts of big data (i.e. time-series, very high-resolution satellite data), exceeding the hosting computer's main memory. This data can be effectively queried and sliced to extract the required data.