ReadWriteWeb

The rate at which data, or content, is being produced for the Web and being generated for businesses has outpaced the rate at which conventional databases are evolving to better manage it all. It's a fact of life that we perceive on a gradual basis every day, but that we haven't yet acknowledged to be as significant or dangerous a trend as it is: Data is getting slower. Networks are getting bigger as the cloud is getting broader, and data that was already difficult to manage is becoming impossible. Content management systems today continue to be based on the types of structured database systems about one or two steps more evolved than dBASE. We've known they would be insufficient for the task, but we've put off the problem of composing a new architecture.

It's already too late for major IT companies to start that new architecture from square one; if a company has any hope of addressing this colossal, underappreciated problem, it will need to acquire the architectural project in progress. This is what Hewlett-Packard announced yesterday that it intends to do: acquire a software firm whose core product aims to supplant everything we know about databases, both the SQL kind and the Google kind. In its place would come a clustered approach whose goal is no less than to be the central repository for meaning in the world.

And in exchange for this, HP is willing to let go of the promise of Palm.

It's hard to keep track of all the database-related terms you hear these days. What constitutes "big data"? What is NoSQL, and why are your developers so interested in it? And now "NewSQL"? Where do in-memory databases fit into all of this? In this series, we'll untangle the mess of terms and tell you what you need to know.

In Part One we covered data, big data, databases, relational databases and other foundational issues. In Part Two we talked about data warehouses, ACID compliance, distributed databases and more. Now we'll cover non-relational databases, NoSQL and related concepts.

The twentieth anniversary of the World Wide Web was celebrated worldwide a few days ago, which says something quite compelling about the state of the Web itself. As ReadWriteWeb Editor-in-Chief Richard MacManus personally verified two years ago by interviewing the guy who thought up the thing in the first place, the Web was established in 1989.

Historically, facts have been considered impediments to good stories. That is until recently, when not even the literary equivalent of blinking neon "FAIL" signs have stopped entirely apocryphal stories from propagating like mercury. Just ask any Internet Explorer user with a high IQ: When the opportunity for a juicy headline arises, such trifle things as facts, math, and common sense don't even amount to blips on the Web's radar.

It's hard to keep track of all the database-related terms you hear these days. What constitutes "big data"? What is NoSQL, and why are your developers so interested in it? And now "NewSQL"? Where do in-memory databases fit into all of this? In this series, we'll untangle the mess of terms and tell you what you need to know.

In Part One we covered data, big data, databases, relational databases and other foundational issues. In this section we'll talk about data warehouses, ACID compliance, distributed databases and more. In part three, we'll cover non-relational databases, NoSQL and related concepts.

agamemnon is a Python-based graph database built on pycassa, the Python client library for Apache Cassandra. In short, it enables you to use Cassandra as a graph database. The API is inspired by the Python wrapper for Neo4j, neo4j.py.

A few months ago we told you about a paper by Microsoft researchers, Erik Meijer and Gavin Bierman which argued that non-relational data stores will need to create a standardized database query language in order to achieve widespread adoption.

Today a new potential standard for document databases (and possibly other NoSQL databases) was announced: UnQL.

An Evans Data survey of 1,200 developers, 400 of which are enterprise developers, found that 56% of enterprise developers already use schemaless databases and 63% plan to use one in the next two years. Adoption is particularly strong in the APAC countries.

Among the general developer community, only 43% of respondents plan to use NoSQL in the next two years.

We've covered several programming languages now, but we haven't really gotten into database development yet. Although NoSQL is hot, SQL (structured query language) isn't going away. Different databases (such as Oracle, MySQL and PostgreSQL) use different variations, but the idea is to have a unified system so that a developer can move from database to database with a minimum of relearning.

If you're not quite sure yet what a relational database is or why someone would use one, check out our guide to database terminology. Otherwise, let's dive into some books for beginners, and one for experienced database developers.

Continuing our series of business apps for the iPad, today we look at database clients, including ones for managing FileMaker, MySQL, Oracle and PostGres databases.

Previously we looked at personal database apps. We've also looked at several iPad apps for IT professionals.

When we talk about the benefits of using a non-relational database management system, often referred to as a NoSQL database, we sometimes lose track of what a traditional database is still good for (for some background on what a relational database is, see our guide to data terminology).

In a blog post at DBMS2, database veteran Curt Monash explains when it's still best to use a relational database.