GDC 08: SQL Considered Harmful
Posted by Ben Zeigler on February 26, 2008
Update: Slides posted here.
Cryptic Studios didn’t have a huge presence at GDC this year, but a few of us gave some talks. One of them was “SQL Considered Harmful”, which was given by Shannon Posniewski. Shannon had a big part in the design of the system, and I and another engineer did a majority of the implementation. Shannon’s working on putting slides up, but until then there’s an great summary up at T=Machine. [NC]Anson has some commentary up, as does Jeff Freeman (cute image, need to use that somewhere). I posted thoughts similar to Shannon’s presentation in a blog post from a couple years ago, when we were starting development. Because I was working on the details of the system, here’s random responses to some of the comments on the talk:
- I had a bit of Object Database knowledge at the start of the project, while the rest of the team did not. Specifically, my dad worked on ObjectStore for a few years, and he’d taught me the fundamentals. After experiencing the failure of SQL, Shannon and the other software leads laid out a system where structures were stored hierarchically and optimized for quick read and write instead of random access. When I learned about the design, I recognized it as a basic Object Database, and the design of the system merged with existing Object Database theory.
- I actually did evaluate ObjectStore and Versant in the research stage of the project. This was my specific job, so Shannon wasn’t really involved. At the time, neither had a product offering that made sense for a company of our size and requirements (Versant has since started a push into MMOs, but I have no idea how good that specific product offering is). The biggest technical problem was that both systems were implemented in fairly-heavy C++ and imposed their own object model. In order to convert our code over to use it would require an extensive rewrite of our entire data back end (as Shannon said, we had a bunch of existing data manipulation and persistence code in proven use in CoH). Without having an existing data infrastructure, using an existing ODB probably makes sense.
- The CrypticDB isn’t live in production, but we are using it exclusively in development. We’ve been running all of our development and playtesting on the CrypticDB for the last year and a half, and functionally it’s working extremely well. We’ve done a bunch of synthetic scalability tests too, which are looking promising. Shannon just didn’t want to overstate the progress of it too much, but I’m as confident as you can be before you actually bring something live for beta. There’s nothing more I can say than check back after Champions launches :)
- Lastly, the benefits of having an integrated data pipeline from client to database are really exciting. When you start thinking of all kinds of data in your system as being essentially the same (and part of a web of related data), it opens up a lot of possibilities.