10 URL Rewriting Tips and Tricks

This post describes some of the tips and tricks that one may find useful when solving URL-based problems for their web server or web site. Each tip/trick has a description of a problem and then an example of how it can be solved with IIS 7 URL Rewrite Module.

  1. Add or Remove Trailing Slash
  2. Enforce Lower Case URLs
  3. Canonical Hostnames
  4. Redirect to HTTPS
  5. Return HTTP 503 Status Code in Response
  6. Prevent Image Hotlinking
  7. Reverse Proxy to Another Site/Server
  8. Preserve Protocol Prefix in Reverse Proxy
  9. Rewrite/Redirect Based on Query String Parameter
  10. Avoid Rewriting of Requests for ASP.NET Web Resources

1. Add or Remove Trailing Slash

Many web applications use “virtual URLs” – that is the URLs that do not directly map to the file and directory layout on web server’s file system. An example of such application may be an ASP.NET MVC application with URL format similar to this: http://stackoverflow.com/questions/60857/modrewrite-equivalent-for-iis-7-0 or a PHP application with URL format that looks like this: http://ruslany.net/2008/11/url-rewrite-module-release-to-web/. If you try to request these URLs with or without trailing slash you will still get the same page. That is OK for human visitors, but may be a problem for search engine crawlers as well as for web analytics services. Different URLs for the same page may cause crawlers to treat the same page as different pages, thus affecting the page ranking. They will also cause Web Analytics statistics for this page to be split up.

This problem is very easy to fix with a rewrite rule. Having or not having a trailing slash in the URL is a matter of taste, but once you’ve made a choice you can enforce the canonical URL format by using one of these rewrite rules:

To always remove trailing slash from the URL:

<rule name="Remove trailing slash" stopProcessing="true">
<match url="(.*)/$" />
<conditions>
<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
</conditions>
<action type="Redirect" redirectType="Permanent" url="{R:1}" />
</rule>

To always add trailing slash to the URL:

<rule name="Add trailing slash" stopProcessing="true">
<match url="(.*[^/])$" />
<conditions>
<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
</conditions>
<action type="Redirect" redirectType="Permanent" url="{R:1}/" />
</rule>

2. Enforce Lower Case URLs

A problem similar to the trailing slash problem may happen when somebody links to your web page by using different casing, e.g. http://ruslany.net/2008/07/IISNET-Uses-Url-Rewrite-Module/ vs. http://ruslany.net/2008/07/iisnet-uses-url-rewrite-module/. In this case again the search crawlers will treat the same page as two different pages and two different statistics sets will show up in Web Analytics reports.

What you want to do is to ensure that if somebody comes to your web site by using a non-canonical link, then you redirect them to the canonical URL that uses only lowercase characters:

<rule name="Convert to lower case" stopProcessing="true">
<match url=".*[A-Z].*" ignoreCase="false" />
<action type="Redirect" url="{ToLower:{R:0}}" redirectType="Permanent" />
</rule>

3. Canonical Hostnames

Very often you may have one IIS web site that uses several different host names. The most common example is when a site can be accessed via http://www.yoursitename.com and via http://yoursitename.com. Or, perhaps, you have recently changed you domain name from oldsitename.com to newsitename.com and you want your visitors to use new domain name when bookmarking links to your site. A very simple redirect rule will take care of that:

<rule name="Canonical Host Name" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTP_HOST}" negate="true" pattern="^ruslany\.net$" />
</conditions>
<action type="Redirect" url="http://ruslany.net/{R:1}" redirectType="Permanent" />
</rule>

To see an example of how that works try browsing to http://www.ruslany.net/2008/10/aspnet-postbacks-and-url-rewriting/. You will see in the browser’s address bar that “www” is removed from the domain name.

4. Redirect to HTTPS

When a site that requires SSL is accessed via non-secure HTTP connection, IIS responds with HTTP 403 (Unauthorized) status code. This may be fine if you always expect that your site visitors will be typing “https://…” in the browser’s address bar. But if you want your site to be easily discoverable and more user friendly, you probably would not want to return 403 response to visitors who came over unsecure HTTP connection. Instead you would want to redirect them to the secure equivalent of the URL they have requested. A typical example is this URL: http://www.paypal.com. If you follow it you will see that browser gets redirected to https://www.paypal.com.

With URL Rewrite Module you can perform this kind of redirection by using the following rule:

<rule name="Redirect to HTTPS" stopProcessing="true">
<match url="(.*)" />
<conditions>
<add input="{HTTPS}" pattern="^OFF$" />
</conditions>
<action type="Redirect" url="https://{HTTP_HOST}/{R:1}" redirectType="Permanent" />
</rule>

Note that for this rule to work within the same web site you will need to disable “Require SSL” checkbox for the web site. If you do not want to do that, then you can create two web sites in IIS – one with http binding and another with https binding – and then add this rule to the web.config file of the site with http binding.

5. Return HTTP 503 Status Code in Response

HTTP status code 503 means that the server is currently unable to handle the request due to maintenance. This status code implies that the outage is temporary, so when search engine crawler gets HTTP 503 response from your site, it will know not to index this response, but instead to come back later.

When you stop the IIS application pool for your web site, IIS will return HTTP 503 for all requests to that site. But what if you are doing maintenance to a certain location of the web site and you do not want to shut down the entire site because of that? With URL Rewrite Module you can return 503 response only when HTTP requests are made to a specific URL path:

<rule name="Return 503" stopProcessing="true">
<match url="^products/sale/.*" />
<action type="CustomResponse" statusCode="503"
subStatusCode="0"
statusReason="Site is unavailable"
statusDescription="Site is down for maintenance" />
</rule>

6. Prevent Image Hotlinking

Image Hotlinking is the use of an image from one site into a web page belonging to a second site. Unauthorized image hotlinking from your site increases bandwidth use, even though the site is not being viewed as intended. There are other concerns with image hotlinking, for example copyrights or usage of images in an inappropriate context.

With URL Rewrite Module, it is very easy to prevent image hotlinking. For example the following rewrite rule prevents hotlinking to all images on a web site http://ruslany.net:

<rule name="Prevent image hotlinking">
<match url=".*\.(gif|jpg|png)$"/>
<conditions>
<add input="{HTTP_REFERER}" pattern="^$" negate="true" />
<add input="{HTTP_REFERER}" pattern="^http://ruslany\.net/.*$" negate="true" />
</conditions>
<action type="Rewrite" url="/images/say_no_to_hotlinking.jpg" />
</rule>

This rule will rewrite a request for any image file to /images/say_no_to_hotlinking.jpg only if the HTTP Referer header on the request is not empty and is not equal to the site’s domain.

7. Reverse Proxy To Another Site/Server

By using URL Rewrite Module together with Application Request Routing module you can have IIS 7 act as a reverse proxy. For example, you have an intranet web server and you want to expose its content over internet. To enable that you will need to perform the following configuration steps on the server that will act as a proxy:

Step1: Check the “Enable proxy” checkbox located in Application Request Routing feature view is IIS Manager.

Step2: Add the following rule to the web site that will be used to proxy HTTP requests:

<rule name="Proxy">
<match url="(.*)" />
<action type="Rewrite" url="http://internalserver/{R:1}" />
</rule>

Note the http:// prefix in the rewrite rule action. That is what indicates that this request must be proxy’ed, instead of being rewritten. When rule has “Rewrite” action with the URL that contains the protocol prefix, then URL Rewrite Module will not perform its standard URL rewriting logic. Instead it will pass the request to Application Request Routing module, which will proxy that request to the URL specified in the rule.

8. Preserve Protocol Prefix in Reverse Proxy

The rule in previous tip always uses non-secure connection to the internal content server. Even if the request came to the proxy server over HTTPS, the proxy server will pass that request to the content server over HTTP. In many cases this may be exactly what you want to do. But sometimes it may be necessary to preserve the secure connection all the way to the content server. In other words, if client connects to the server over HTTPS, then the proxy should use “https://” prefix when making requests to content server. Similarly, if client connected over HTTP, then proxy should use “http://” connection to content server.

This logic can be easily expressed by this rewrite rule:

<rule name="Proxy">
<match url="(.*)" />
<conditions>
<add input="{CACHE_URL}" pattern="^(https?)://" />
</conditions>
<action type="Rewrite" url="{C:1}://internalserver/{R:1}" />
</rule>

9. Rewrite/Redirect Based on Query String Parameters

When rewriting/redirection decisions are being made by using values extracted from the query string, very often one cannot rely on having the query string parameters always listed in exact same order. So the rewrite rule must be written in such a way so that it can extract the query string parameters independently of their relative order in the query string.

The following rule shows an example of how two different query string parameters are extracted from the query string and then used in the rewritten URL:

<rule name="Query String Rewrite">
<match url="page\.asp$" />
<conditions>
<add input="{QUERY_STRING}" pattern="p1=(\d+)" />
<add input="##{C:1}##_{QUERY_STRING}" pattern="##([^#]+)##_.*p2=(\d+)" />
</conditions>
<action type="rewrite" url="newpage.aspx?param1={C:1}&param2={C:2}" appendQueryString="false"/>
</rule>

With this rule, when request is made to page.asp?p2=321&p1=123, it will be rewritten to newpage.aspx?param1=123&param2=321. Parameters p1 and p2 can be in any order in the original query string.

10. Avoid Rewriting of Requests for ASP.NET Web Resources

ASP.NET-based web applications very often make requests to WebResources.axd file to retrieve assembly resources and serve them to the Web browser. There is no such file exists on the server because ASP.NET generates the content dynamically when WebResources.axd is requested. So if you have a URL rewrite rule that does rewriting or redirection only if requested URL does not correspond to a file or a folder on a web server’s file system, that rule may accidentally rewrite requests made to WebResources.axd and thus break your application.

This problem can be easily prevented if you add one extra condition to the rewrite rule:

<rule name="RewriteUserFriendlyURL1" stopProcessing="true">
<match url="^([^/]+)/?$" />
<conditions>
<add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
<add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
<!--  The following condition prevents rule from rewriting requests to .axd files -->
<add input="{URL}" negate="true" pattern="\.axd$" />
</conditions>
<action type="Rewrite" url="article.aspx?p={R:1}" />
</rule>

44 thoughts on “10 URL Rewriting Tips and Tricks”

  1. What a great collection of useful, day to day snippets.
    Some of the more grizzled coders may scoff; but for someone new to the hard core side of web dev, (albeit after a lifertime in IT on the tech/analyst side) this sort of mainstream/fundamentals collection is soooo valuable…

  2. Thanks, I am trying to implement the opposite of #3. In my case I want the www. I am trying to use the HTTPRewrite Module and compare your rule to the UI. I assume you are using regex and not wildcards. I cannot get it to work.

    M

  3. The rule to enforce www in the domain name:

    <rule name="WWW" enabled="true" stopProcessing="true">
    <match url=" (.*)" />
    <conditions >
    <add input="{HTTP_HOST}" pattern="^www\.company\.com$" negate=”true” />
    </conditions>
    <action type="Redirect" url="http://www.company.com/{R:1}" redirectType="Permanent" />
    </rule>

  5. Geo, could you clarify what you mean by rewrite the google bot? Do you really mean rewrite or redirect? These are two different things. Also why using https?

  6. i used #4 redirect to https and i was constantly getting a redirect loop, i suppose because its the same site tied to both http and https and i have absolutely no control over iis because it was on a cloud site.

    BUT! i got it working with a slight modification – i can only hope that maybe i’m going to save someone a little trouble here – and who knows maybe someone’s got advice for me hahaha 🙂

    the only change i made was in the match url line

    instead of “(.*)” i made it “http://(.*)”

    (so that way it only applies the rule if it’s not under the https already) way to go me! (and thank you very much for taking YOUR time to post shit like this for us programmers who aren’t so dedicated (and those who are – no offense haha))

  7. This is interesting. Normally, the match url is not supposed to contain the “http://” part as it will never match. May be this is because the cloud service provider uses some other rewrite rule at the server level? May be it is used together with IIS Application Request Router. That might also explain why the rule in #4 did not work. I used that rule on my IIS server (which I have full control of) and it worked fine.

  8. I’ve got an ignorant question that I’m hoping won’t be completely out of line here. We’re trying to do a canonical name redirect for SEO purposes for a site that’s on an IIS server. It’s a simple site, all the files are currently .html files, none of us really knows much about IIS servers, and we came across this page in trying to figure out our options. Can we adapt and copy the code you cite and use that somewhere (and where?), or is all of this relevant only if we’re using URL Rewrite Module?

    Thanks — and I hope the question isn’t so ignorant as to be offensive!

  9. Betsy, you can you this code with IIS 7 if you install URL Rewrite Module. IIS 7 is available on Windows Vista SP1, Windows Server 2008, Windows 7 and Windows Server 2008 R2. If you have any of those OS then you can use all the code examples I’ve listed in this post.

  10. Many thanks. Looks like that rules us out, though — the host uses IIS 6. If you have any tips for a canonical redirect on IIS 6, they’d be much appreciated — but it’s beginning to appear that the best option may be to transfer to the host’s linux platform.

  11. actually, as it turns out, that did work for a bit then it stopped

    my hosting service uses a different server variable name of
    HTTP_CLUSTER_HTTPS

    so that’s what it was

    thank you so much!

  12. Betsy all this can be accomplished on IIS 6 using Isapi Rewrite. Drop me an email if you need help: kevin at seoarchitect.com

  14. How about rewriting a domain to a specific folder like “domain.com.br/folderxxxx” -> “domainxxxx.com” and then have this folder suppressed ? (i.e. do not use domainxxxx.com/folderxxxx”)

  15. Renan, when user types “domain.com.br/folderxxxx” in the browser address bar and makes a request, what should the address bar contain after response is received: “domain.com.br/folderxxxx” or “domainxxxx.com”?

  16. This URL Rewrite module is excellent!

    For section 2. Enforce Lower Case URLs I think the regex pattern [A-Z] is not enough. We should match any character any number of times before and after capital English alphabets (add other capital alphabets if needed), i.e. ^.*[A-Z].*$.


    <rule name="Convert to lower case" stopProcessing="true">
    <match url=".*[A-Z].*" ignoreCase="false" />
    <action type="Redirect" url="{ToLower:{R:0}}"
    redirectType="Permanent" />
    </rule>

    If we want to exclude lower casing some file types to avoid multiple requests to the same file (with original upper / mixed case vs only lower case) we could use the rule below. Add extensions as you see fit:


    <rule name="Convert to lower case" stopProcessing="true">
    <match url=".*[A-Z].*" ignoreCase="false" />
    <conditions>
    <add input="{REQUEST_FILENAME}"
    pattern="^.*\.(axd|css|gif|png|js)$" negate="true" ignoreCase="true" />
    </conditions>
    <action type="Redirect" url="{ToLower:{R:0}}"
    redirectType="Permanent" />
    </rule>

  17. Samuli, thanks for your comment! You are correct – that was an error in the pattern. I have updated the post now.

  19. Re: Enforce Lower Case URLs

    Actually you were close. Match url=”[A-Z]” is fine. It will detect any upper case letters in the Url.

    You just need to change the Redirect Rule to url=”{ToLower:{URL}}”

  21. Thanks for wonderful tips. 4 of your rules are very important to us. Order of the rules is important. Ex:

    We experienced that “If you do not put “Remove trailing slash” at the end it did not work.”

    The order that we used is :
    1) Canonical Host Name
    2) Default Document
    3) Convert to lower case
    4) Remove trailing slash

    Can you give use the exact order of the ten rules?

  22. This is a great resource for me as I am jsut learning IIS Rewrite Module. I ran into an issue with jQuery ajax and the Enforce Lower Case URLs. Because it was forceing my method name into lower case, the .net web services was returning a 500 error. I added a condition and I was back in business. Thanks for this resource.


    <rule name="Convert to lower case" stopProcessing="true">
    <match url=".*[A-Z].*" ignoreCase="false" />
    <conditions>
    <add input="{URL}" negate="true" pattern="\.js$" />
    <add input="{URL}" negate="true" pattern="\.asmx$" />
    </conditions>
    <action type="Redirect" url="{ToLower:{R:0}}" redirectType="Permanent" />
    </rule>

  23. Yes, order of the rules is important. I do not think anyone would use all ten rules at the same time so the exact order may differ based on what subset is used. In general, the order should be as follows:

    1. Request blocking rules (e.g. rules 5 and 6)
    2. Redirect rules (e.g. rules 1, 2, 3, 4)
    3. Rewrite rules (e.g. rules 7, 8, 9, 10)

  24. Ruslany,

    Thanks for the answer. We used 4 of them in our all web sites .
    1) Canonical Host Name
    2) Default Document
    3) Convert to lower case
    4) Remove trailing slash

    Does order of these rules important?
    Can you give the order of Redirect rules?

  25. For redirect rules I would do:

    1. Canonical Host Name
    2. Lower case
    3. Add or remove trailing slash.

    In fact, this is how the rules are configured on my site. I do not have default document rule, but I think that placing it after “canonical host name” is a good idea.

  27. neo: URL Rewrite Module is not available on IIS6 so you will have to use some other rewriting module, for example ISAPI_Rewrite. The rule format will be different though, as it uses mod_rewrite-like rule syntax.

  28. Thanks for the reply.

    May I know how can I achieve this using ISAPI_Rewrite?

    Thanks once again.

  30. neo, I am not that familiar with ISAPI_Rewrite, but I think you can do it by using a rule similar to this:

    RewriteCond %{HTTP_HOST} ^([a-zA-Z0-9]+)\.abc\.com$
    RewriteRule (.*) %1/$1 [QSA]

    Note that the rule may have mistakes – I have not tested it. Refer to the ISAPI_Rewrite documentation for more details.

  31. Jamal: A rule similar to below may help:
    <rule name="redirect">
    <match url="^modules\.php" />
    <conditions>
    <add input="{QUERY_STRING}" pattern="id=([0-9]+)" />
    </conditions>
    <action type="Redirect" url="index.php?p={C:1}" appendQueryString="False"/>
    </rule>

    Refer to the URL Rewrite documenation for more details.

  35. Create a redirect rule that checks if the request came for anything but “default” or “app1” and then redirects to “default”:

    <rule name="redirect"stopProcessing="true">
    <match url="(.*)"/>
    <conditions>
    <add input="{URL}" pattern="^/(?:default|app1)" negate="true" />
    </conditions>
    <action type="Redirect" url="default/{R:1}" redirectType="Permanent" />
    </rule>

    Make sure that this rule is in the web.config file located at the root folder of your web site. Otherwise it won’t work.

  37. Ruslany,

    Thank you for sharing this. I followed “Step 7. Reverse Proxy To Another Site/Server”, and installed/configured ARR on my server. But I have several domains for which I don’t want ARR to affect their current functionality. Can I use ARR + URL Rewrite on only specific domains, and not have it affect the existing functionality of other domains?

    Thanks,
    Robert

  38. @robert: Yes, you can create modify the rule in #7 to affect only specific domains. For that you could add a condition to the rule that will force to rule to be executed only for a particular domain:

    <conditions>
    <add input="{HTTP_HOST}" pattern="mydomain\.com"/>
    </conditions>

  39. Ruslany, thanks for getting back to me, that makes sense. My only other concern with this configuration is that each domain that uses ARR will allow users to upload images/videos/etc. These domains/websites are built in Silverlight and use ADO.NET Entity Framework and Web Services to accomplish file upload. Ideally, I want to upload/save their files to the Proxy server using ARR, and then retrieve these files from the Proxy server for display within their domain. Is this possible with the ARR configuration described? And do you have any other advice or documentation that you could point me to? Thanks again for your help.

    Robert

Comments are closed.