Placement Constraints with Service Fabric

I call this blog my notepad for a reason. Its where I can write things down if for no other reason then to help me remember. There’s the added benefit that over the years a few of you have found it and actually come here to read these scribbles and from what I heard, you sometimes even learn something.

That said, I wanted to share my latest discovery, not just for myself, but for the sake of anyone else that has this need.

Placement Constraint Challenges

As I mentioned in my post on Network Isolation with Service Fabric, you can put placement constraints on services to ensure that they are placed on specific node types. However, this can be challenging when you’re moving your application packages between clusters/environments.

The first one you usually run into is that the moment you put a placement constraint on the service, you can’t deploy it to the local development cluster. When this happens, folks starting trying to alter the local cluster’s manifest. Yes, the local cluster does have a manifest and no, you really don’t want to start changing that unless you have to.

So I set about to find a better way. In the process, I found a couple of simple, but IMHO poorly documented features that make this fairy easy to manage. And manage in a way where the “config is code” aspects of the service and fabric manifests are maintained.

Application vs Service Manifest

There are a couple ways to configure placement constraints. One of the ones you’ll find most readily is the use of code at publication. I’ll be honest, i hate this approach. And I suspect many developers do. This approach while entirely valid, requires you to write even more code for something that can easily be managed declarative via the service and package manifests.

If you dig a bit more, you’ll eventually run across the declaration of constraints in the service manifest.

 
<ServiceTypes> 
  <StatelessServiceType ServiceTypeName="Stateless1"> 
    <PlacementConstraints>(NodeTypeName==BackEnd)</PlacementConstraints> 
  </StatelessServiceType> 
</ServiceTypes> 

This IMHO, is perfectly valid. However, it has the unfortunate side affect I mentioned earlier. Namely that, despite hours of trying, there’s no easy way to alter this constraint via any type of configuration setting at deployment time. But what I recently discovered was that this can also be done via the application manifest!

 
<DefaultServices>
  <Service Name="Stateless1">
    <StatelessService ServiceTypeName="Stateless1Type" InstanceCount="[Stateless1_InstanceCount]">
      <SingletonPartition />
      <PlacementConstraints>(isDMZ==true)</PlacementConstraints>
    </StatelessService>
  </Service>
</DefaultServices>

I never knew this was even possible until a few days ago. And admittedly, I can’t find this documented ANYWHERE. It was by simple lucky I tried this to see if it would even work. Not only did it work, but I found that any placement constraint declared in the application manifest would override whatever was in the service manifest.

But this still doesn’t solve the root problem. But it does put us a step closer to addressing it.

Environment Specific Application Parameters

As I was trying to solve this problem, Aman Bhardwaj of the service fabric team sent me this link. That link discusses options you have for managing environment specific settings for service fabric applications. Most of the article centers around the use of configuration overrides to change the values of service configuration settings (that are in the PackageRoot/Config/Settings.xml) at publication. We don’t need most of that, what we need is actually much simpler.

The article discusses that you can parameterize the application manifest. This will substitute values defined in the ApplicationParameter files (you get two by default when you create a new service fabric application), In fact, the template generated by visual studio already has an example of this as it sets the service instance count. And since, as I just mentioned we can put the placement constraints into the application manifest… well I think you can see where this is going.

We start by going into the application manifest and adding a new parameter.

 
<Parameters>
  <Parameter Name="Stateless1_InstanceCount" DefaultValue="-1" />
  <Parameter Name="Stateless1_PlacementConstraints" DefaultValue="" />
</Parameters>

Note that I’m leaving the value blank. This is completely acceptable as it tells the the service fabric that you have no constraints. In fact, during my testing any value that could not be properly evaluated as a placement constraints will be ignored. So you could put in “azureisawesome” and it will have the same affect as leaving this blank. However, we’ll just keep it meaningful and leave it blank.

With the parameter declared, we’re next going to update the constraint declaration in the application manifest to use it.

 
<DefaultServices>
  <Service Name="Stateless1">
    <StatelessService ServiceTypeName="Stateless1Type" InstanceCount="[Stateless1_InstanceCount]">
      <SingletonPartition />
      <PlacementConstraints>[Stateless1_PlacementConstraints]</PlacementConstraints>
    </StatelessService>
  </Service>
</DefaultServices>

So “(isDMZ==true)” has become “[Stateless1_PlacementConstraints]”. With this done, now all we need to do is go define environment specific values in the ApplicationParameter files.

You’re most likely going to leave the Local.xml parameter as blank. But the Cloud.xml (or any other environment specific file you provide, is where you’ll specify the environment specific setting.

 
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/SampleApp1" xmlns="http://schemas.microsoft.com/2011/01/fabric">
  <Parameters>
    <Parameter Name="Stateless1_InstanceCount" Value="-1" />
    <Parameter Name="Stateless1_PlacementConstraints" Value="(isDMZ==false)" />
  </Parameters>
</Application>

With these in place, when you deploy the service, you simply have to ensure you specify the proper configuration file when you deploy.

image

This is of course the publication dialog for Visual Studio, but that parameter file is also a parameter of the Deploy-FabricAppliation.ps1 file that is part of the project and use to do publications via the command line. So these same parameter files can be used for unattended or automated deployments.

Off to the festival

And there we have it! Which is good because that’s all the time I have for today. My son is heading off to school this week which means my wife and I will officially be “empty nesters”. Today we’re taking him and his girlfriend to the local Renaissance Festival for one last family outing before he leaves. So I hope this helps you all and I bid you a fond “huzzah” until next time!

PS – I have place a sample app with many of these settings already in place on github.

Advertisements

9 Responses to Placement Constraints with Service Fabric

  1. SATHISH KRISHNAN says:

    Thanks Brent, this helped !

  2. Mike says:

    Give this man a beer if you see him on friday night!

  3. Nice one, helped me a lot.

  4. Lars Eirik Sivesind says:

    But one thing I don’t get: Every article I’ve found shows using publish profiles from Visual Studio. However, in production scenario I can’t use VS to publish services, I need to use powershell. How can publish profiles be utilized from powershell? I know I can go via Deploy-FabricApplication.ps1 which is part of the VS project, but this script contains so many things I don’t need that I don’t wanna touch it.

    • Brent says:

      The publishing profiles are a VS construct. More specifically, its part of the VS tooling for Service Fabric and doesn’t directly translate to anything about Service Fabric itself. You see something similar for applications being published to Web Apps or Virtual Machines.

      You shouldn’t in most cases have to touch the FabricApplication.ps1 script. You can just use it “as is” to do the deployment from the command line. I have a partner that’s doing just this from their build server. If you really wanted to break it down into its components parts, you’ll see what its essentially about parsing the profiles and calling the underlying methods to register the applications and services manually. These lower level calls are already fairly well documented and even encouraged by the PG as a way to deal with environmental customization at deployment.

  5. Sonia says:

    Hi Brent very nice blog. do you know how we can create a service fabric package in VSTS by excluding the Code folder so as to just package the manifest files and config folder? I am trying to search in google but didnt find anything yet.

    • Brent says:

      What specifically are you trying to accomplish? I ask because the manifest files are really just a visual studio tooling construct. They aren’t actually needed by Service Fabric. When you run the deployment powershell script included in the VS project, its actually parsing those files and using them to register your application and its services with the service fabric. Its only the code assets that are actually shipped to the service fabric cluster.

      • Sonia says:

        Hi Brent,
        What I was meaning in creating a diff package. But I got what I was looking for here – http://amolenk.com/2016/06/30/Creating-a-Service-Fabric-Diff-Application-Package-using-PowerShell/

        However do you know we we can change the application parameters after the package is deployed, without having to redeploy the package?
        For eg: If I have specified connection string in the Application parameters (cloud.xml) file and deployed the application. Then I realize that for some reason I need to change that connection string. So is there a way that I can only deploy the cloud.xml file or change it somehow on the cluster without having to re-deploy the package? Because re-deploying the package would mean changing the version of the package.

  6. Brent says:

    For what you’re trying to do I would suggest using the configuration package. Each service fabric service is made up of 3 packages: Code, Config, and Data. You can independently deploy and version each of these. For more info, take a look at this link: https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-cloud-services-migration-worker-role-stateless-service#configuration-settings

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: