InfoSphere BigInsights

What’s New in IBM InfoSphere BigInsights V2.0

New features that take the complexity out of Hadoop and enable everyone to get their hands on big data

IBM InfoSphere BigInsights V2.0 builds on top of open source Hadoop and augments it with mandatory capabilities needed by enterprises. IBM InfoSphere BigInsights v2.0, released November 16, 2012, contains significant enhancements that include:

New Big Data Tools: Visualization, Monitoring, Development

Business Analysts

• Centralized dashboard – allows business analysts to leverage a new charting engine and use BigSheets, a spreadsheet-like visualization tool, to get insights into their data, view analytic application results and monitor metrics
• BigSheets enhancements – allow business analysts to view BigSheets data flows between and across data sets to quickly navigate and relate analysis and charts
• Application linking - allows user to compose new applications from existing applications and BigSheets and invoke analytics applications from the web console, including integration within BigSheets
• New built-in applications:
  • REST data source App that enables users to load data from any data source supporting REST APIs into BigInsights, including popular social media services
  • Sampling App that enables users to sample data for analysis
  • Subsetting App that enables users to subset data for data analysis

Data Scientists

• Social Data Analytics Accelerator – allows for deeper analysis and richer insights, addressing key use cases such as customer profiling and lead generation using the Social Data Analytics Accelerator
• R integration – allows users to execute an ad hoc R job directly from BigInsights Web Console
• Text analytics enhancements – bring support for modular AQL (Annotator Query Language), improving the usability and development productivity of text analytics applications, via enabling & reusing common extractors, dictionaries, rules, tables

Administrators

• Monitoring – provides the ability to control all applications from a single page and a centralized dashboard view to visualize key performance indicators including CPU, disk, and memory and network usage for the cluster, data services such as HDFS, HBase, Zookeeper and Flume, and application services including MapReduce, Hive, and Oozie

Developers

• New and enhanced editors
  • New workflow editor - greatly simplifies the creation of complex Oozie workflows with a consumable interface
  • New Pig Editor - with content assist and syntax highlighting that enables users to create and execute new applications using Pig in local or cluster mode from the Eclipse IDE
  • Enhanced Jaql Editor - with extended support for Jaql syntax, extended content assist, and improved execution feedback
• Enhanced application development & deployment
  • Enhanced Text Analytics development - includes support for modular rule sets
  • Ability to publish new application types – including BigSheets Macro, BigSheets Reader, AQL module, Jaql module
  • Enhanced scope of development artifacts – during the deployment phase, this includes artifacts for Text Analytics, scripts for Jaql, Hive SQL, Pig, and BigSheets macros and readers
• New unified tooling environment for the Big Data Application Development Lifecycle - enables developers to sample data and define, test, and deploy analytics applications from the BigInsights Eclipse tools. Administer, execute, and monitor the deployed applications directly from the BigInsights Web Console

Enterprise Integration

• Data Explorer – bundled with InfoSphere BigInsights, allows indexing and “on the glass” integration
• Guardium – enables auditing, security and monitoring capabilities
• DataStage – enables data collection and integration
• Cognos Business Intelligence - bundled with InfoSphere BigInsights, brings Support for Hive; Business Intelligence capabilities
• Netezza - allows users to query and join Netezza data in InfoSphere BigInsights using UDFs
• Platform Computing Symphony - high performance, low-latency platform computing grid
• WebSphere - WAS 8.5 Liberty Profile – high performance secure REST access
• R - allows users to execute open source R jobs directly from InfoSphere BigInsights web console

Application Accelerators

• Social Data Analytics Accelerator – provides the capability to ingest and process large volumes of social media data to provide key insights that can be used to develop programs/applications such as customer retention, customer acquisition, lead generation, brand management, and campaign effectiveness
• Machine Data Analytics Accelerator - provides the capability to ingest and process large volumes of machine data, which can include IT machines, sensors, meters, GPS devices and more, to provide key business insights