Tuesday, August 6, 2013

Basic Meaning of Big Data

From last many days some students and my juniors are asking about basic of Big Data and how it is related to Cloud Computing. So I thought to write an article explaining meaning Big Data.

As the name suggest Big Data is related with huge amount to data which can not be processed by using simple methods and tools. For example, modern high-energy physics experiments, such as DZero1, typically generate more than one TeraByte of data per day. The famous social network Website, Facebook, serves 570 billion page views per month, stores 3 billion new photos every month, and manages 25 billion pieces of content. Main Question is how to process this much of data in less time? Data collected from these sources is very loosely linked to each other so making decisions from this data is very complex and time consuming. Our today's conventional databases cannot process data if they don't know exact relation between terms. Now a days organizations collect data form many different sources and methods. For example, an laptop company can collect data about a product from social networking sites such as Facebook, twitter etc. from many laptop related blogging websites and even from laptop selling online sites. Data collected from this much different sources can’t be process my conventional databases to make any proper decisions. This data is too big, moves too fast, or doesn’t fit the structure of conventional databases. To gain value from this data, we must follow a new approach. This new approach is known as Big Data.

Seeing the need of Big Data on March 29, 2012 American Government announced “Big Data Research and Development Initiative” and Big Data became the national policy for first time. Recently, the definition of big data as also given by the Gartner: “Big Data are high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable
enhanced decision making, insight discovery and process optimization”( I will explain three V’s of big data in coming posts in detail).

Basically technique used for getting useful information from very large unstructured data sets is known as Big Data. This data can be of any type. I hope you get very basic meaning of Big Data. Stay tuned i will post for detail about Big data architecture, 3 V’s and all in subsequent posts.

Rajinder Sandhu


  1. its gud and helps me to understand the basic..:-)

  2. Hello Rajinder Sandhu,
    Recently, I reading something about on Cloud provider virtual Environment but enable to simulate it can u help me out how to create a virtual environment to provider IaaS to their users/clients through vmware studio for a cloud provider.I shall be very thanksful to you

  3. hi Rakesh,
    If you want to create an cloud environment for providing IaaS service i recommend use openStack or CloudStack tool. Both are open source and you will find many installation guide on internet. OpenStack and CloudStack are not simulator they create actual cloud environment.
    However, VMware Studio is used to create virtual appliance capable of running on vmware hyper-visors. It is not used to create any kind of Cloud environment.

    Hope you get your answer..

  4. k. thanks for replying .
    how can we migrate a cloud storage data from 1 cloud to another in virtual environment i.e from 1 host provider to another ?