Only as strong as the weakest link
August 15, 2010 Leave a comment
I know, you’re still waiting on me to finish my Azure Diagnostics series. I’m working on it, but its taking longer then I wanted for me to do the due diligence on it. Meanwhile, I’ve been having discussions lately that I wanted to share.
Now I’m sure many of us have had to build a high performance website. We build the site, toss a load test at it, then tweak it until it can handle the load. If you need more performance, you may throw more memory, more CPU, performance tweak a few things. And if you’re lucky, the guesstimate that was used to come up with the projected load was either low or on a few occasions, accurate.
Problem is, as developers and architects, we’ve grown lazy. We’re too dependent on scaling vertically (bigger and more powerful hardware). We also too often fail to consider scalability in the early stages of architectural design. The prime example of this is database design. We think about creating an affective physical db schema, but how often do we design early on for scaling our database.
Yes, I’m talking about database partitioning.
The discussions I’ve been having lately usually start with “I need more than 50gb for SQL”. To which I usually respond “why?”. Do we have blob types stored in SQL? Can you move those objects to Azure Storage? Do you really need all 50gb in a single DB instance? Do you have a subset of that data that is mostly read only? Can it be moved to its own DB instance or better yet possibly be converted over to an in-proc cache system?
We need to rethink application data stores. Those with experience with data warehouses and operational data stores are already reading this and thinking ‘well duh”, but the fact of the matter is that the average developer still needs to change their mindset. To an extent, we need to reach back 20+ years to the days when the mainframe and multi-tenant systems were common.
The rules of DB’s have changed. And if we don’t change with them, we’ll continue to limit ourselves. To quote “The Matrix”, free your mind. Think about the DB not as an appliance but as yet another tool in your toolbox. If we don’t, it will remain the weakest link in our solutions.
On that note I’d like point you at a case study for TicketDirect out of New Zealand. Pay special attention to the “Migrating to Windows Azure” section. Where they talk about using small, sometimes temporary SQL Azure partitions to deliver a high performance, nearly on-demand type solution. Imagine selling out an arena for a U2 concert in 20 minutes and the hosting costs being less than a new netbook.
If reading this blog (or better yet the case study) excites you with new ideas, then I’ve accomplished my goal. Have fun thinking outside of the box.