A long desired self-learning system to aid big data business social media consumption

Abstract

Organizations have developed Big Data capabilities to process enormous amounts of data – Petabytes and such. It is the lower end of the Volume-spectrum of Big Data that has not found enough easy to use, low infrastructure solutions that may require little learning yet may offer real time interaction possibilities and human friendly visualization. Typical media consumers such as airlines flyers have lagged in consumption of information, because Big Data business so far has failed to even acknowledge that the social media data average flyers can often have to deal with, has similar Big Data challenges. This experiment based research establishes and demonstrates a context-aware approach that can help flyers self-learn from blogs and their own social media data. It show-cases InContext, an in-browser Social Media data processor that has been tested with nearly 42,000 customer reviews expressed as nearly 0.5 million lines of text equivalent to nearly 17,500 A4 sized pages of text. InContext uses light-weight technologies for processing enormous amounts of Social Media data and carries out considerable Big Data tasks, real-time in an interactive manner without requiring the use of any (1) external dictionaries, (2) SQL or NoSQL like querying, and (3) libraries. InContext enables: (1) exploration of text data (2) keyword and phrase matching (3) data partitioning (4) data reduction (5) extraction of entities and dictionaries (6) extraction of relationships between entities (7) extraction of specific information in-context, say regarding food and drinks, in-flight entertainment or seating comfort and (8) comparison of flyer reviews with that specificity. This work has potential for significant impact on business. Especially the high usability and low resource requirements of InContext have implications for design of comparative apps by business. Also, by enabling flyers to consume more information – this work proposes to alleviate information load – thus increasing the availability of flyers to even more information from business. This work is novel, and we are not aware of any literature that addresses this issue.