Storage efficiency

Storage efficiency is a big deal. I have learned a lot about read, write and space amplification over the past few years. Tiered storage has been a part of my work life via flashcache, but tiered storage for an LSM doesn't require flashcache. This is going to get interesting. We have more choices for SSD at a variety of price points based on write endurance and performance.  We have write-optimized database engines (RocksDB, Tokutek, WiredTiger) arriving for OLTP workloads. We can use this with commodity hardware and open-source DBMS solutions like MySQL and MongoDB. A lot of interesting work remains to put the pieces together while matching the quality of service we get from MySQL+InnoDB and I hope to use RocksDB as part of MySQL and MongoDB.

We are beginning to engage with the community. Yoshinori Matsunobu has a talk at the MySQL UC. I have a talk at the CloudDM workshop at ICDE. I expect more talks, hopefully a few conference papers and if you are local then there are RocksDB meetups.

One benefit from RocksDB compared to a b-tree is much better compression and much less write-amplification. I have observed this via linkbench and real workloads. There are many other ways that an LSM can reduce the IO demand from an application. But I don't want to give away too many details until Yoshi and I have done our talks.

I don't get marketing

MongoDB 3.0 is almost here. It includes the WiredTiger storage engine which is very impressive. Alas, now we are told that the original storage engine has performance problems. I am shocked. Who knew? Not anyone reading their documentation. I don't understand why companies play this game. Be honest with your users as this goes a long way to building a solid and educated community. I hold MySQL to the same standard. Lets us not forget that awesome manual section on atomic operations.

Don't let the marketing team have editorial control over the documentation team.

Edit: an era has ended, the section on atomic operations has been removed from the MySQL manual.