Comments for Database Analytics Startup Aster Data Launches, Analyzes MySpace

Database Analytics Startup Aster Data Launches, Analyzes MySpace

2008-05-20T07:01:01Z

Grid-computing startup Aster Data Systems will officially launch today, three years after it was founded. Aster, which began in the Ph.D program at Standford, is a provider of "massively parallel processing databases" for organizations that have mammoth quantities of data that need to be stored and analyzed quickly. The Redwood City, California-based company is backed by Sequoia Capital, Cambrian Ventures, and First-Round Capital.

]]> Aster's nCluster software allows companies with large amounts of data to store it on commodity hardware and scale with one-click, adding new servers as the data set grows. The company's first major client is MySpace, which generates 100s of terabytes of traffic data from its 110 million monthly unique users. Mining that data to understand how customers use and interact with the site requires some pretty robust architecture.

Aster's solution for MySpace uses a 100-node cluster of off-the-shelf commodity servers that can capture and load 100% of the data and run complex queries quickly. "MySpace needed to analyze complete datasets - not just samples or summaries. Sampling would completely miss infrequently occurring but highly profitable patterns," according to Aster, which says that nCluster has allowed MySpace to work with all of its terabytes of data and avoid the need to sample.

nCluster works by splitting up the cloud into smaller bits that each have a specific task. "Loader" nodes load data from external sources (and export to them), while "worker" boxes keep data stored on local disks. A "queen" layer directs the entire operation intelligently routing queries to the proper node. The "loader" tier can scale independently as needed, say Aster. "This enables query load-balancing to eliminate hot-spots and increase performance, returning results in seconds or minutes versus hours or 'did not finish,'" writes the company in a case study.

The software reminds me of 3Tera's AppLogic (our coverage), which is a grid computing operating system that makes it easier for companies to deploy their own compute cloud on commodity hardware. nCluster is essentially the same idea, but with an eye specifically toward managing and querying massive databases.

Comment from Falafulu Fisi on 2008-05-20

2008-05-20T12:17:28Z

Excellent product and well done to the founders for developing it. Massive Data analytics of today is mostly done off-line and not on-line ie, to find or extract important features from the data. Parallel processing has an advantage in doing live (on-line) query of the stored features (which were calculated off-line) from the database(s). This is what Google is doing. Feature extractions is done off-line and indexed, which is the slowest part. Live query is done using on the feature database , not on the original database, but features do point to IDs in the original database for retrieval.

I would say that the top analytics of today that target the enterprises are SPSS, SAS, Oracle, KXEN. Aster Data Systems must battle these established data analytic vendors in that market.

I know (not personally) a few personnels who are top of the ladder in the domain analytic R&D (ie, machine learning, data-mining, etc) that consult to those vendors mentioned above, such as Prof. Amy Lanville (she is consulting to SAS), where her algorithm for solving the PageRank was adopted by Google in 2003. Langville's algorithm solves the PageRank faster than the one that Google used. PageRank can be solved using a variety of linear algebra algorithms, but Langville's algorithm was faster the the Google's own PageRank solver in 2003.

Another one is Prof. Vladimir Vapnik (Consulting to KXEN), who was a pioneer in the development of the popular SVM (Support Vector Machine), which is adopted in most state-of-art software of today, from speech processing, image processing, online recommendation engine, DNA sequencing, text-mining, and more... There are more top notch researchers in analytics than the 2 names I have mentioned where they are consulting to those leading vendors in enterprise analytics.

I have read some of the publications of those researchers and I think that Aster Data Systems must stepped up in order to compete with these well-established analytic vendors.

Comment from kaz on 2008-05-26

2008-05-26T07:19:14Z

Good article and good comment.