RuslanY Blog » ASPNET

10 URL Rewriting Tips and Tricks

ruslany — Wed, 08 Apr 2009 22:34:21 +0000

This post describes some of the tips and tricks that one may find useful when solving URL-based problems for their web server or web site. Each tip/trick has a description of a problem and then an example of how it can be solved with IIS 7 URL Rewrite Module.

1. Add or Remove Trailing Slash

Many web applications use “virtual URLs” – that is the URLs that do not directly map to the file and directory layout on web server’s file system. An example of such application may be an ASP.NET MVC application with URL format similar to this: http://stackoverflow.com/questions/60857/modrewrite-equivalent-for-iis-7-0 or a PHP application with URL format that looks like this: http://ruslany.net/2008/11/url-rewrite-module-release-to-web/. If you try to request these URLs with or without trailing slash you will still get the same page. That is OK for human visitors, but may be a problem for search engine crawlers as well as for web analytics services. Different URLs for the same page may cause crawlers to treat the same page as different pages, thus affecting the page ranking. They will also cause Web Analytics statistics for this page to be split up.

This problem is very easy to fix with a rewrite rule. Having or not having a trailing slash in the URL is a matter of taste, but once you’ve made a choice you can enforce the canonical URL format by using one of these rewrite rules:

To always remove trailing slash from the URL:

To always add trailing slash to the URL:

2. Enforce Lower Case URLs

A problem similar to the trailing slash problem may happen when somebody links to your web page by using different casing, e.g. http://ruslany.net/2008/07/IISNET-Uses-Url-Rewrite-Module/ vs. http://ruslany.net/2008/07/iisnet-uses-url-rewrite-module/. In this case again the search crawlers will treat the same page as two different pages and two different statistics sets will show up in Web Analytics reports.

What you want to do is to ensure that if somebody comes to your web site by using a non-canonical link, then you redirect them to the canonical URL that uses only lowercase characters:

3. Canonical Hostnames

Very often you may have one IIS web site that uses several different host names. The most common example is when a site can be accessed via http://www.yoursitename.com and via http://yoursitename.com. Or, perhaps, you have recently changed you domain name from oldsitename.com to newsitename.com and you want your visitors to use new domain name when bookmarking links to your site. A very simple redirect rule will take care of that:

To see an example of how that works try browsing to http://www.ruslany.net/2008/10/aspnet-postbacks-and-url-rewriting/. You will see in the browser’s address bar that “www” is removed from the domain name.

4. Redirect to HTTPS

When a site that requires SSL is accessed via non-secure HTTP connection, IIS responds with HTTP 403 (Unauthorized) status code. This may be fine if you always expect that your site visitors will be typing “https://…” in the browser’s address bar. But if you want your site to be easily discoverable and more user friendly, you probably would not want to return 403 response to visitors who came over unsecure HTTP connection. Instead you would want to redirect them to the secure equivalent of the URL they have requested. A typical example is this URL: http://www.paypal.com. If you follow it you will see that browser gets redirected to https://www.paypal.com.

With URL Rewrite Module you can perform this kind of redirection by using the following rule:

Note that for this rule to work within the same web site you will need to disable “Require SSL” checkbox for the web site. If you do not want to do that, then you can create two web sites in IIS – one with http binding and another with https binding – and then add this rule to the web.config file of the site with http binding.

5. Return HTTP 503 Status Code in Response

HTTP status code 503 means that the server is currently unable to handle the request due to maintenance. This status code implies that the outage is temporary, so when search engine crawler gets HTTP 503 response from your site, it will know not to index this response, but instead to come back later.

When you stop the IIS application pool for your web site, IIS will return HTTP 503 for all requests to that site. But what if you are doing maintenance to a certain location of the web site and you do not want to shut down the entire site because of that? With URL Rewrite Module you can return 503 response only when HTTP requests are made to a specific URL path:

6. Prevent Image Hotlinking

Image Hotlinking is the use of an image from one site into a web page belonging to a second site. Unauthorized image hotlinking from your site increases bandwidth use, even though the site is not being viewed as intended. There are other concerns with image hotlinking, for example copyrights or usage of images in an inappropriate context.

With URL Rewrite Module, it is very easy to prevent image hotlinking. For example the following rewrite rule prevents hotlinking to all images on a web site http://ruslany.net:

This rule will rewrite a request for any image file to /images/say_no_to_hotlinking.jpg only if the HTTP Referer header on the request is not empty and is not equal to the site’s domain.

7. Reverse Proxy To Another Site/Server

By using URL Rewrite Module together with Application Request Routing module you can have IIS 7 act as a reverse proxy. For example, you have an intranet web server and you want to expose its content over internet. To enable that you will need to perform the following configuration steps on the server that will act as a proxy:

Step1: Check the “Enable proxy” checkbox located in Application Request Routing feature view is IIS Manager.

Step2: Add the following rule to the web site that will be used to proxy HTTP requests:

Note the http:// prefix in the rewrite rule action. That is what indicates that this request must be proxy’ed, instead of being rewritten. When rule has “Rewrite” action with the URL that contains the protocol prefix, then URL Rewrite Module will not perform its standard URL rewriting logic. Instead it will pass the request to Application Request Routing module, which will proxy that request to the URL specified in the rule.

8. Preserve Protocol Prefix in Reverse Proxy

The rule in previous tip always uses non-secure connection to the internal content server. Even if the request came to the proxy server over HTTPS, the proxy server will pass that request to the content server over HTTP. In many cases this may be exactly what you want to do. But sometimes it may be necessary to preserve the secure connection all the way to the content server. In other words, if client connects to the server over HTTPS, then the proxy should use “https://” prefix when making requests to content server. Similarly, if client connected over HTTP, then proxy should use “http://” connection to content server.

This logic can be easily expressed by this rewrite rule:

9. Rewrite/Redirect Based on Query String Parameters

When rewriting/redirection decisions are being made by using values extracted from the query string, very often one cannot rely on having the query string parameters always listed in exact same order. So the rewrite rule must be written in such a way so that it can extract the query string parameters independently of their relative order in the query string.

The following rule shows an example of how two different query string parameters are extracted from the query string and then used in the rewritten URL:

With this rule, when request is made to page.asp?p2=321&p1=123, it will be rewritten to newpage.aspx?param1=123¶m2=321. Parameters p1 and p2 can be in any order in the original query string.

10. Avoid Rewriting of Requests for ASP.NET Web Resources

ASP.NET-based web applications very often make requests to WebResources.axd file to retrieve assembly resources and serve them to the Web browser. There is no such file exists on the server because ASP.NET generates the content dynamically when WebResources.axd is requested. So if you have a URL rewrite rule that does rewriting or redirection only if requested URL does not correspond to a file or a folder on a web server’s file system, that rule may accidentally rewrite requests made to WebResources.axd and thus break your application.

This problem can be easily prevented if you add one extra condition to the rewrite rule:

Setup IIS on Server Core – Windows Server 2008 R2

ruslany — Fri, 27 Feb 2009 01:20:24 +0000

This post has been updated from its original version to correct the installation instructions, which have changed since the time the post was written in Feb 2009.

With the addition of .NET Framework to Server Core in Windows Server 2008 R2 the Server Core installation option became even more appealing for those who want to use a very low footprint server for hosting their applications. Availability of .NET framework provides the following great benefits:

ASP.NET support – you can now use Server Core to host your ASP.NET applications.
IIS Remote Management – Server Core does not provide any user interface other than command line. But if you prefer to use IIS Manager UI to administer IIS, you can now use IIS Remote Manager to connect to IIS on Server Core and perform all the management tasks from within familiar UI of IIS Manager.
PowerShell – Windows Server 2008 R2 includes IIS PowerShell snapin, which is also available on Server Core.

This post describes how to setup and configure IIS on Server Core in Windows Server 2008 R2. Specifically the following tasks are described:

Using oclist and ocsetup commands
Basic Installation of IIS
Installing ASP.NET
Installing PowerShell and IIS snap-in
Enabling IIS Remote Management

Using oclist and ocsetup commands

oclist command can be used to list the available and installed roles and services on the server. The output of the command looks similar to below:

It is important to understand how oclist renders component dependencies. For example, in the above screenshot the oclist output shows that IIS-FTPExtensibility is dependent on IIS-FTPSvc, so in order to install IIS-FTPExtensibility it is first necessary to install IIS-FTPSvc.

ocsetup command can be used to install and uninstall individual roles and services.

Basic Installation of IIS

To perform basic and default installation of IIS on Server Core run the following command:

start /w ocsetup IIS-WebServerRole

After the command has run you can run oclist | more and check which IIS components have been installed.

Installing .NET Framework

If you plan to use ASP.NET or IIS Remote Management then it is necessary to install .NET Framework first. To install it use the following commands:

start /w ocsetup NetFx2-ServerCore
start /w ocsetup NetFx2-ServerCore-WOW64

Installing ASP.NET

To install ASP.NET on Server Core run the following commands in exact same order as listed below:

start /w ocsetup WAS-NetFxEnvironment
start /w ocsetup IIS-ISAPIExtensions
start /w ocsetup IIS-ISAPIFilter
start /w ocsetup IIS-NetFxExtensibility
start /w ocsetup IIS-ASPNET

Installing PowerShell and IIS snap-in

First, install the PowerShell by running the command as below:

start /w ocsetup MicrosoftWindowsPowerShell

Next, launch the PowerShell by running this command:

\windows\system32\WindowsPowerShell\v1.0\powershell.exe

After running this command you should see a PowerShell prompt.

In order to enable the IIS snapin it is necessary to change the script execution policy by running this command:

Set-ExecutionPolicy RemoteSigned

Restart PowerShell for the policy changes to take effect. After re-entering the PowerShell import the IIS snapin:

import-module WebAdministration

After that you can get the list of available IIS cmdlets by typing

get-command –module WebAdministration

Refer to the article Managing IIS with IIS 7.0 PowerShell Snap-in for more information.

Enabling IIS Remote Management

In order to install IIS remote management service it is necessary to install .NET Framework first. Refer to the Installing .NET Framework section above for more details.

Install the IIS management service by executing this command:

start /w ocsetup IIS-ManagementService

Then enable remote management by running this command:

reg add HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WebManagement\Server ^
/v EnableRemoteManagement /t REG_DWORD /d 1

And finally start the management service:

net start wmsvc

Now you can connect to the IIS on the Server Core from a remote machine by using IIS Remote Manager.

Acknowledgement: thanks to Saad Ladki and Thomas Deml from IIS team for explaining to me all these steps.

ASP.NET postbacks and URL rewriting

ruslany — Thu, 23 Oct 2008 05:04:11 +0000

ASP.NET Web Forms extensively use postback mechanism in order to maintain the state of the server-side controls on the web page. This makes it somewhat tricky to perform URL rewriting for ASP.NET pages. When a server side form control is added to the web page, ASP.NET will render the response with HTML

tag that contains an action attribute pointing back to the page where the form control is. This means that if URL rewriting was used for that page, the action attribute will point back to the rewritten URL, not to the URL that was requested from the browser. This will cause the browser to show rewritten URL any time a postback occurs.

Let me demonstrate this on an example. Assume you have a very simple web form in a file called default.aspx. When you request http://localhost/default.aspx in a browser and then view the HTML source for the response, you will see that the response contains the element, which looks similar to this:

The action attribute contains the URL where the form data will be posted to when you click on the button in the web page.

Now, let’s create a very simple rewrite rule that rewrites URL from http://localhost/homepage to http://localhost/default.aspx.

When you request http://localhost/homepage in a browser, the URL will be rewritten in accordance to the above rule and the page will be shown correctly in the browser:

However, when you click on the Submit button, the browser’s address bar will display http://localhost/default.aspx, thus exposing the internal URL, that you wanted to hide by using URL rewriting:

A few workarounds to fix this behavior existed in previous versions of ASP.NET. For example, you could sub-class the form control, or use Control Adapter as explained here. However, these workarounds were not very easy to implement. Luckily, the ASP.NET in .NET Framework 3.5 SP1 provides a very simple way to fix that. Now you can use the property of the HtmlForm class called Action to set the postback URL to the one that was requested by browser before any rewriting happened. In ASP.NET you can obtain that URL by using HttpRequest.RawUrl property. So, to fix the postback URL for your web form when you use URL Rewrite Module, you would need to add the following code to the page:

protected void Page_Load(object sender, EventArgs e)
{
    form1.Action = Request.RawUrl;
}

After this change, if you reload the web page at http://localhost/homepage and then click on submit button you will see that the browser’s address bar still displays the correct URL:

If you view the HTML for the response you will see that the tag now contains the correct action URL:

When you use ASP.NET master pages you could add the Page_Load method to the master page and that would take care of postback action URL for all the pages in your web application. Note thought that in order to be able to use the HtmlForm.Action property you have to upgrade to .NET Framework 3.5 SP1.

Wildcard script mapping and IIS 7 integrated pipeline

ruslany — Wed, 01 Oct 2008 03:30:18 +0000

The big benefit of IIS 7 integrated request processing pipeline is the fact that all the nice and useful ASP.NET features can be used for any type of content on your web site; not just for ASP.NET-specific content. For example, ASP.NET SQL-based membership can be used to protect static files and folders. Also, ASP.NET extensibility API’s, such as IHttpHandler and IHttpModule can be used to add custom modules and handlers that would be executed even for non-ASP.NET content.

IIS 6 did not have this level of integration. ASP.NET was plugged into IIS 6 as an ISAPI extension and by default was configured to handle ONLY requests mapped to that extension – for example any request that ended with “.aspx” would be be processed by ASP.NET extension. This obviously was a big limitation for customers who wanted to be able to use ASP.NET features for all other contend on web site. The most common way to workaround that was to use “Wildcard script mapping”. This post explains how an application that used wildcard script mapping in IIS 6 can be migrated over to IIS 7.

Assume you had configured ASP.NET in IIS 6 to handle all requests by using wildcard script mapping. For example you had an ASP.NET module for URL rewriting and you wanted this module to handle extension-less URLs.

This wildcard script map configuration is typically done within IIS 6 manager by opening the properties dialog for either web server or web site and selecting Home Directory tab, then clicking on Configuration button and then clicking on the Insert button for “Wildcard application maps”:

Now, as you move your application to IIS 7, you want to configure it to achieve the same behavior of ASP.NET. There are two options on how this can be done: using Classic pipeline mode or using Integrated pipeline mode.

Wildcard script mapping in IIS 7 classic pipeline mode

With classic pipeline mode the ASP.NET is plugged into the IIS request processing pipeline as an ISAPI extension – exactly the same way as it was in IIS 6. In fact, if you open %WINDIR%\system32\inetsrv\config\applicationHost.config file and locate the section inside of it you can see how IIS is configured to map ASP.NET specific requests to the aspnet_isapi.dll:


  ...
  
  ...

Notice the preCondition attribute for the handler mapping. Among other things this attribute is set to classicMode, which ensures that this handler mapping only takes effect when the application pool is configured to run in classic mode.

Now if you want to configure wildcard mapping for the ASP.NET running in classic mode, you can add one more handler mapping to the section just before the handler mapping for static files:

...

The relative order of the wildcard handler mapping is important: if you define this handler mapping after the “StaticFile” handler mapping then “StaticFile” will handle all the requests and no requests will ever come to ASP.NET wildcard handler.

As you may already know the ASP.NET wildcard handler mapping in IIS 7 is subject to the same performance limitations as existed in IIS 6. The problem with this kind of handler mapping is that ALL requests are processed by it, including requests for static files. The ASP.NET static file handler is not as powerful as native IIS static file handler. Plus static files served by ASP.NET will not be cached by IIS in kernel mode. Because of these performance limitations it is recommended that you use IIS 7 integrated pipeline to achieve the same functionality as with wildcard mappings in IIS 6.

IIS 7 integrated pipeline instead of wildcard script mapping

With integrated pipeline, the ASP.NET functionality is fully integrated into the main request processing in IIS, which means that all ASP.NET features are now available for any type of requests. This effectively eliminates the need for the wildcard handler mapping. Now you can use your existing ASP.NET modules and have them applied to all requests.

For example let’s say you had a URL rewriting module written in ASP.NET. On IIS 6 this module was registered inside of the section in web.config file as below:


  
    ...
    
    ...

Since on IIS 6 this module was only executed for requests to managed content, it only worked for URLs that had .aspx extension, such as http://example.com/archive/2008/08/26/post-title.aspx. If you wanted it to handle extension-less URLs you had to configure wildcard script mapping for ASP.NET. With IIS 7 integrated pipeline you do not have to do that anymore. In order to make this module apply to extension-less URLs you need to register it within the section inside of web.config file as below:


  
    ...
    
    ...

Make sure that you leave the preCondition attribute empty here as it would enforce that the module will be executed for all requests, not just for requests for ASP.NET specific content.

Registering your managed modules this way does not have such dramatic performance impact as when using wildcard script mappings. Even though the module is invoked for all requests to web application, all the existing handler mappings are still in effect, which means that the static files are still served by the native IIS static file handler. Another benefit of registering your module this way is that now it can be applied to requests for PHP, ASP or any other dynamic pages. You would not be able to do that if you used wildcard script mappings.

One last thing to mention here is that you can also use the attribute on the section called runAllManagedModulesForAllRequests.


  
    ...
    
    ...

This attributes forces IIS to ignore the preCondition=”managedHandler” attribute, hence all managed modules will be invoked for all requests to web application.