Application Fabric Architecture
August 10, 2009 3 Comments
So, a couple weeks back I rallied on a bit about the need for health and activity monitoring in cloud computing solutions. I made the argument that solution architects cannot depend on the service providers to meet these needs. However, I didn’t offer any real solutions. Since that time I’ve started putting some time and thought into the creation (or location) of design patterns intended to help address these concerns within the confines of a cloud computing based solution. I wasn’t able to find anything that seemed to echo my own thoughts, so I’d like to give you my preliminary work on what I’m calling the “Application Fabric Architecture”.
As I started trying to find a solution to my dilemma, I looked for existing solutions. I wanted examples of systems that have to manage application systems spread across multiple instances but yet still be able to maintain them as a cohesive whole. The examples that really leaped out at me were those of the various Cloud Computing providers themselves. The ones I looked at all seemed to have this notion of a fabric. The fabric would contain the individual instances of the application(s) which were managed by a centralized controller. Furthermore, each instance had an agent that was responsible for monitoring the status of the instance and either taking direction from the controller or feeding information to it.
The result looks something like this:
Pretty straightforward I know. Also a bit of a “well duh” once I started visualizing it.
The Primary Components
The Application Fabric Architecture (if anyone has a better idea of a name, please let me know), is comprised of 4 primary components. The Controller, the Agent, the Application Node, and a Database (some type of persistent storage). The duties of each of these components, as well as how they interact, is what I’ve really been putting my thoughts too.
The Application Node is the simplest (in terms of architecture role) of all the components. The Node is the grunt that has the responsibility for doing the manual labor the application requires. This could be research computing, message processing, or just serving up web pages. Its not aware of and really doesn’t care about other instances or the architecture components beyond the what it needs to know to do its job.
The Controller is the brains of our architecture. In theory, its responsible for checking the status of all the application nodes and acting on that status. It can also potentially perform such tasks as starting/stopping nodes (if a suitable host provider API is available), serving up configuration data (such as security credentials), or even delegating workloads. In practice, well… I’ll get to that another day.
Since the Node is not directly aware of the Controller, we need a component that handles this communication and feeds information back to the Controller, this is the Agent. The Agent feeds information up to the Controller and in turn takes commands from it. Its responsible for registering the application node with the controller when it starts up, as well as trying to tell the controller when a node stops (as best it can).
Lastly, we need a Configuration Database. This data source does not have to be an actual database, but it does need to be some type of persistent storage (it could be a file, or even another process running elsewhere). The reason we need this is that because of the distributed nature of our application, we won’t always be able to have a direct connection between the Controller and the agents. So the database acts as an intermediary. Both the agents and the controller need to be able to reach it so they can then use it to relay messages.
Of course, this is just the beginning. The Agents and Controller could communicate via service calls, a relay bus, queues, or by dropping files onto a shared folder structure. Additionally, the functionality provided by the cloud computing hosting system will have a significant impact on what type of functionality will be available to the controller and its agents. But regardless of the details, its my belief that these basic component roles will still be present.
So once this is in place, what are some possible uses for this architecture? Well, this is where my mind starts getting blown. A simple example I quickly came up with is this…
- Controller calls host providers and triggers the startup of a new instance
- New instance starts and initializes the agent
- Agent registers the instance with the controller and requests configuration data
- Controller acknowledges the registration and responds with necessary configuration data (security credentials, endpoints, configuration settings, etc…)
- Agent receives data and stores it for retrieval by the Node (exposed via properties?)
- Node begins performing work
Of course, much of this will depend on the mechanisms for allowing the agents and controller to communicate as well as the options the controller has to give direction to the cloud host. This is also just the beginning of nearly endless possibilities. Possibilities I look forward to exploring over the coming months.
I know its not much yet, but its a starting point. In future updates, I’m going to look a bit more in depth at the tasks (command delivery, status updates, registration) and issues (bandwidth consumption, availability, connectivity) that we’ll need to address which this architecture. Over time, this might even expand into how to coordinate multiple controllers running in different geographical locations.
In the meantime, I’d really appreciate any feedback on what I’ve presented here. Its very much in an incubation state, but I think it has potential. The eventual goal is to drive this down to a series of class/object designs and maybe even some implementation templates (something akin to the Enterprise Library possibly). So it will definitely be long road with lots of work before its finalized.
Till next time and I look forward to hearing from you!