ReadWriteEnterprise

Pentaho Corporation today announced that it has made freely available under open source all the big data capabilities in its Kettle v4.3 release, and has moved the entire Pentaho Kettle project to the Apache License Version 2.0. This is the same open source license that Hadoop and others use. We have covered Pentaho before here.

Hadoop seems to be on everyone's minds this year. It's certainly a hot topic for Forrester's James Kobielus, who's recently released several reports on Hadoop – including a best practices guide aimed at enterprises.

Hadoop is still pretty young, so Kobielus first starts with a couple of challenges facing early adopters. The big challenges, according to Kobielus? An immature market, evolving core specifications, a need for custom coding and a lack of a widely adopted "stack" for Hadoop.

The huge problem for online services is that traditional SQL database managers don't scale up when database sizes approach "exascale" - the tremendous and fast-growing repositories needed by services like Facebook and Twitter. There's nothing conceptually wrong with SQL, it's just that the underlying RDBMS architecture does not perform well with these tremendous workloads.

Simpler database constructs can handle bigger workloads, as long as the work they do stays more along the lines of simple storage and retrieval and doesn't get too analytical. Today, a new vendor named DataStax whose backers include Rackspace is launching a commercial rendition of an exascale database manager that marries an open source database manager project launched at Facebook with an open source distributed processing project started at Google.

The news this week about firing Carol Bartz, Yahoo's CEO, made us go into the Wayback Machine to recall the many good things that Yahoo has created over its life. While there are many that are lining up to take shots at the Yahoos (certainly justified, including this mention of Bartz here), there are still some things worth noting that came out of Yahoo or that were touched by the company.

This week Microsoft announced its intentions to bring Hadoop to SQL Server and Parallel Data Warehouse (PDW). PDW was introduced last year and brought centralized data warehousing capabilities to SQL Server.

"As a first step, we will soon release a Community Technology Preview (CTP) of two new Hadoop connectors - one for SQL Server and one for PDW," the SQL Server Team blog says.

It's hard to keep track of all the database-related terms you hear these days. What constitutes "big data"? What is NoSQL, and why are your developers so interested in it? And now "NewSQL"? Where do in-memory databases fit into all of this? In this series, we'll untangle the mess of terms and tell you what you need to know.

In Part One we covered data, big data, databases, relational databases and other foundational issues. In Part Two we talked about data warehouses, ACID compliance, distributed databases and more. Now we'll cover non-relational databases, NoSQL and related concepts.

This week Pentaho rolled out an update for Pentaho 4 that expands its open core business intelligence platform's native support for big data sources. Pentaho now supports Apache Hadoop, HPCC Systems, EMC Greenplum, HP Vertica, IBM Netezza, MongoDB and more.

Last month Twitter acquired social media analytics company BackType. Much of BackType's technology (such as ElephantDB and Cascalog) are already open source, and this week Twitter announced that BackType's Storm will be open-sourced at the Strange Loop conference in September.

Storm is a Hadoop-like system, but instead of running MapReduce "jobs" that eventually end, Storm runs never ending "topologies." It can be used for continuous computing, processing streams of data, etc.

Cloudera and Dell today announced an agreement to sell a complete Apache Hadoop solution. The package will include Dell hardware (including Dell PowerEdge C2100 servers and PowerConnect switches), Cloudera's Distribution including Apache Hadoop, Cloudera Enterprise and of course support. It should be available for purchase within the next 30 days. Details can be found on dell.cloudera.com.

Rumors have been circulating for the past few months that Yahoo would create its own Apache Hadoop commercialization company to compete with Cloudera. Today GigaOM's Derrick Harris reports that Yahoo will make an official announcement this week.

According to Harris, the spin-off will be called HortonWorks, a reference to the elephant-themed Dr. Suess book Horton Hears a Who.