How to warm up Azure Web App during deployment slots swap

Azure Web App deployment slots are used to help roll out new versions of an app without downtime or cold start activation. New version is typically deployed to a staging slot, then after testing and final verification it gets swapped into a production slot. During the swap operation the Web App’s worker process may get restarted in order for some settings to take effect. Even though the swap does not proceed until the restarted worker process comes back online on every VM instance, it may still not be enough for application to be completely ready to take on production traffic. This post explains how you can use the recently enabled Application Initialization Module to completely warm up your application prior to swapping it into production.

First of all it is necessary to explain the sequence of actions that happens when a staging slot is swapped into production. When the Swap button is clicked in Azure Portal or a corresponding management API is called:

  1. The App Settings and Connection Strings that are marked as “Slot” are read from the Production slot and applied to the site in the Staging slot. That causes the site’s worker process to be restarted for those changes to take effect and become visible as process environment variables;
  2. Then the site in the staging slot gets warmed up. To warm up the site an HTTP request is made to the root directory of the site to every VM instance where site is supposed to run. The warm up request has a User-Agent header set to “SiteWarmup”;
  3. After warm up has completed the host names for the sites in production and staging slots get swapped. Now the site that has been warmed up in the staging slot starts getting production traffic and the site that used to be in the production slot is now in the staging slot
  4. The site that is now in the staging slot gets updated with the App Settings and Connection Strings associated with the staging slot. That causes restart of that site, but it is not in production slot any more so restart is harmless.

Sometimes hitting the site’s root URL is not enough to completely warm up the application. For example it maybe necessary to hit all important routes in an ASP.NET MVC app or to pre-populate the in-memory cache. That is where the Application Initialization Module can help.

Let’s use a simple example to demonstrate how a Web App can be warmed up in the deployment slot during the swap operation. First let’s create a site and a staging deployment slot:

Next let’s set some slot settings on the App and its staging slot. These slot settings will cause the App’s worker process to restart during swap.

For the actual app code I used two simple PHP files: index.php and warmup-cache.php. The index.php is served when site’s root URL is requested. The warmup-cache.php is my “cache warmup” code that takes long time to run (emulated by sleep() command). In real application that can be the script that makes database queries to fill up the cache.

Finally I also have a web.config file which configures AppInit module:

<system.webServer>
  <applicationInitialization >
    <add initializationPage="/warmup-cache.php" hostName="appinit-warmup.azurewebsites.net"/>
  </applicationInitialization>
<system.webServer>

In the applicationInitialization section I can specify multiple URL paths that need to be requested in order to warm up my application. In my case I only need to hit one URL. Also notice that I can specify the host name to use for the warm up requests (this is optional and if not specified the “localhost” will be used as a host name).

The following steps are just for verification/debugging purposes. There is no need to perform them when using AppInit module. In fact enabling them for your production site may considerably slow it down.

To confirm that the warmup-cache.php is actually requested during the swap I will use Failed Request Tracing. It can be enabled from Azure Portal:

However, that will trace only failed requests. I need it to trace all requests. For that I add the following section to the web.config file:

<tracing>
  <traceFailedRequests>
    <clear/>
    <add path="*">
      <traceAreas>
      <add provider="WWW Server" areas="Authentication,Security,Filter,StaticFile,CGI,Compression,Cache,RequestNotifications,Module,Rewrite,iisnode" verbosity="Verbose" />
      </traceAreas>
      <failureDefinitions statusCodes="200-600" />
    </add>
  </traceFailedRequests>
</tracing>

The important attribute here is statusCodes which specifies that requests with status codes between 200 and 600 should be logged.

Now let’s run the swap command.

During the swap operation the application is restarted and the first warm up request is made to the root directory of the web app. That starts the AppInit module which makes a request to warmup-cache.php URL and waits until it completes. Only after that request competes the swap operation proceeds to the next step and swaps the host names so that the warmed up site starts getting production traffic. Because of that the swap operation takes longer time to complete.

After the swap completed we can analyze the Request Tracing logs to confirm that the warmup-cache.php URL has been hit prior to the swap. Note that we need to get those log files from the site in the production slot now.

Looking through the logs we can see the following:

  • The first warm up request is made to the web site. Notice the user agent value set to “SiteWarmup”:
  • Another log file shows that the appInit module has started and made an HTTP request to the warmup-cache.php around the same time. That request has the host name I specified in the web.config file. Also the user-agent is different.

    As expected that request took around 30 seconds.

That simple example demonstrates how to use IIS Application Initialization Module with Azure Web App deployment slots to ensure that the application in the slot is completely warmed up. The complete Web App code can be downloaded from the link below.

Azure Web App Deployment Slot Warmup Example

28,079 views

ruslany on September 30th 2015 in PHP, WAWS

PoorFairAverageGoodExcellent (8 votes, average: 4.38 out of 5)

11 Responses to “How to warm up Azure Web App during deployment slots swap”

  1. Gravatar ImageMickaël responded on 30 Sep 2015 at 1:28 pm #

    Great article, thanks!
    One question, though: is targeting a specific host name safe? Could that result in not all of your instances being warmed up – given you have more than one?

  2. Gravatar ImageVishnu Arunachalam responded on 02 Oct 2015 at 4:59 pm #

    Thanks for the details. This has been very helpful. I have one follow up question.

    After warm up when the host names are swapped, what happens to the traffic that is currently in the production slot? We are deploying a web app with very high traffic and want to make sure that the users do not have a bad experience when the swapping is done. Do some requests get dropped during the swapping process?

  3. Gravatar ImageSelim responded on 08 Oct 2015 at 5:24 am #

    Hi Rusany,

    Thanks for the very helpful article!

    We have tried this out and found out that during swapping, warm up requests are sent to the wrong slot.

    We are sure of this because we did some logging to db in the warmup page controller and the logs definitely belonged to the production code, not the development code.

    Could there be a bug?

  4. Gravatar Imageruslany responded on 30 Oct 2015 at 9:28 am #

    Hi Selim, that should not happen and if you see that happening, then this sounds like a bug. Can you let me know your site name where you can repro this problem? Just send me a private message here http://ruslany.net/contact/

  5. Gravatar Imageruslany responded on 30 Oct 2015 at 9:32 am #

    Hi Vishnu,

    No requests should be dropped during swap. If there are any requests that are currently being processed by the site in production slot then will still complete.

  6. Gravatar Imagemani responded on 16 Nov 2015 at 12:35 am #

    nice article about azure web app

  7. Gravatar ImageNick responded on 08 Jan 2016 at 10:49 am #

    I’m trying to find a solution to warming up when swapping to production so I’ve been reading trough your article but there are some strange things going on.
    I enabled the logging for all the status codes so I can see my warmup requests and they seem to respond with a 301 status code right away. Instead my site goes already in production and it’s down for a couple of minutes after the first request.
    Here’s the log: https://gist.github.com/nickdima/d01e3a6114eee1895d40
    Any ideas?

  8. Gravatar ImageJean-Sébastien Goupil responded on 12 Feb 2016 at 5:16 pm #

    I have been trying to install the applicationInitialization on our site.
    It does work, but I need to hit some pages that are behind a login page.

    In my /Cache URL request, I make a WebRequest to /Login, get the CSRF token, make a login request, get the cookie. Then I make 20 more requests with a WebRequest to the private pages.

    The problem here is I get the following error:
    Unexpected Error in Application: System.Net.Sockets.SocketException (0×80004005): An attempt was made to access a socket in a way forbidden by its access permissions 127.0.0.1:80
    at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
    at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)

    Also, the website will give me a YSOD, then a 502 from the server.
    After a while, the site becomes available but does did not warm up appropriately. Definitely crashed during my Cache warm up.

    In order to get the correct path to hit, I use the following:
    return Request.Url.Scheme + System.Uri.SchemeDelimiter + Request.Url.Host + (Request.Url.IsDefaultPort ? string.Empty : “:” + Request.Url.Port);

    But I suspect there might be a problem with this?
    I get the following:
    http://localhost

    Like Mickaël mentioned on your blog, I am afraid to use the “hostName” property in the WebConfig as it would not warm up my instances properly.

    Is there a way to achieve this thing without any problems?

    Thanks!

  9. Gravatar ImageSebastian responded on 08 Apr 2016 at 1:57 am #

    Hi,

    is it possible to create a “warm-up” for Java services? I am using an Azure API Service with Tomcat.

    Regards

    Sebastian

  10. Gravatar ImageCarlos Pedro responded on 16 Sep 2016 at 3:12 am #

    Hi,
    Do the initialization pages get hit when deploying to a single instance? I am trying to test this in my test/dev website but warm up never runs.

    Thanks,
    Carlos

  11. Gravatar Imageruslany responded on 27 Oct 2016 at 5:25 pm #

    Hi Carlos, the initialization page should be hit even if web app is on a single instance.

Trackback URI | Comments RSS

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

XML Markup: If You want to add XML code to the comment please XML encode it first, otherwise the code will not show up.

Recently Published Articles