Scaling the database, cheaply and massively
Posted on May 21, 2008 by Chris DeBrusk
As a heavy user of databases, both in my smaller ventures like TravelGator and also in my consulting work for large financial services companies helping them figure out how to do nasty jobs like trade surveillance, I’ve spent hours (days?) trying to figure out how to load, analyze and extract data from large databases in an efficient manner (and before the market open the next day). Unfortunately, traditional databases like Oracle and MySQL make this job very difficult . They required significant of tuning to get queries to run fast and are completely inadequate for anything that is adhoc in nature. Even the industry leading ETL frameworks have all kinds of limitations when it comes to loading large datasets that require a lot of cleaning and normalization.
So it is fun to see that startups haven’t completely given up on trying to do the database better, even given the large players in the space like Oracle, SAS and others.. Aster has released a product that came out of a Stanford PhD project (why is it always Sanford…?) that is attempting to do for databases what Google has done for search - make the database scale horizontally to thousands of generic, off the shelf servers.
Apparently they are already working with MySpace to load and analyze 100 terabytes of data that they’ve collected from the users. If this thing works it might help move the database out of the Oracle era of small incremental improvements and into the future.
Thanks to Read Write Web for bringing them to my attention.
Sphere: Related Content» Filed Under Cloud Computing
Comments
Leave a Reply
Subscribe via FeedBurner