WHAT’S IN THIS CHAPTER?
WROX.COM CODE DOWNLOADS FOR THIS CHAPTER
Please note that all the code examples in this chapter are available as a part of this chapter’s code download on the book’s website at www.wrox.com on the Download Code tab.
One of the most important considerations when building a web application is how well the application scales as it starts to get more traffic. Applications that consume lots of server resources do not tend to scale well, which leads to an unresponsive site and a bad user experience. In a lot of cases, precious server resources such as threads are waiting to perform some I/O-based call or are waiting for the result of a database call. This operation blocks the thread from processing any other requests that might come in.
You can solve most of these problems by following an asynchronous programming paradigm, which ensures that you are utilizing your resources effectively. The .NET Framework has had support for writing asynchronous code since .NET 1.0. The async features in the framework have evolved quite a lot during these years.
This chapter explores these asynchronous paradigms and describes how you can use them in your web application. It shows you how you can use the async/await pattern that was introduced in .NET 4.5 and talks about some pitfalls of writing bad asynchronous code. In the end, you look at some tweaks you can make to your web server for increasing concurrency of your web application.
This chapter begins with a quick look at why you need to write async code and the challenges it involves. It then looks at how and where you can write async code in ASP.NET. It also goes into some of the pitfalls of writing async code in web applications, which is different from writing async code in traditional desktop applications.
The web server has a finite number of threads to process requests. The need for writing async code arises from this limitation so that the application can use these resources effectively. Whenever a request comes to ASP.NET, a new thread from the CLR thread pool is allocated to process the request. If the application is written in a synchronous way, the thread will be busy processing this request. If part of the processing requires waiting for a result from a database call, this thread is waiting for the result and cannot be used to process any other requests. Now if all the threads from the thread pool are held up, any further request to this website will not be processed, and the user will be waiting. This would ultimately lead to an unresponsive site.
This is where the benefits of writing applications in an asynchronous pattern come into play. If the same application were written using async, when the thread is waiting on the results of the database call, the thread is given back to the thread pool by ASP.NET while the database call happens on a different thread. The thread that is given back to the thread pool is now available to process any other incoming requests to the application. When the database call finishes, the application is notified and the request is completed.
The considerations of when to write async code in web applications are quite different compared to traditional desktop applications.
A good rule of thumb of when to write async code for web applications is anytime your application needs to do an I/O-based network call. This includes anytime you need to access files on disk, or you need to make a call to a web service, or you need to make a call to a database server. In each of these cases, making I/O calls is highly optimized in Windows and lets ASP.NET reclaim your thread to process other requests while your application is waiting on the result of this call.
A more modern case of writing async code is when your application depends on a long-running request. In this scenario you have a long-running connection between a server and client, and the data is transferred between them when some event takes place. This is typically used in long polling and HTTP streaming. In the case when no data is being exchanged, you do not want a thread to be processing this request because there is no data, but as soon as some event happens, you want a thread to do to the work. You can look at these cases more in Chapter 26.
Since the inception of the .NET Framework programming for async, dealing with server resources has always been challenging. When it comes to writing async code, developers have had to grapple with an added complexity — namely, the conceptual overhead of writing async code. This makes writing and debugging async code difficult, because you have to deal with threads, callbacks, and synchronization state when the request is switched to different threads. The .NET Framework has introduced several patterns that have tried to solve these problems for the developer. In .NET Framework 4.5, a new pattern called async/await was released, which does the best at making it easier to write and debug async code. You take a brief look at all of these patterns in the following sections.
In .NET 1.0, the framework introduced a pattern called the Asynchronous Programming Model (APM), also called the IAsyncResult pattern. In this model, asynchronous operations require Begin and End methods (for example, BeginWrite and EndWrite for asynchronous write operations).
In .NET 2.0, the Event-based Asynchronous Pattern (EAP) was introduced. This pattern requires a method that has the Async suffix, and also requires one or more events, event handler delegate types, and EventArg-derived types. Listing 31-1 shows some synchronous code which makes an outbound call to download contents and calculates total bytes downloaded.
LISTING 31-1: Synchronous code
VB
Public Function RSSFeedLength(uris As IList(Of Uri)) As Integer
Dim total As Integer = 0
For Each uri In uris
Dim data = New WebClient().DownloadData(uri)
total += data.Length
Next
Return total
End FunctionC#
public int RSSFeedLength(IList<Uri> uris)
{
int total = 0;
foreach (var uri in uris)
{
var data = new WebClient().DownloadData(uri);
total += data.Length;
}
return total;
}Listing 31-2 shows how the same code looks in the EAP async pattern. As you can see from Listing 31-2, the code has become a lot more complicated. The foreach loop had to be broken up to manually get an enumerator. The code also looks recursive instead of iterative. This makes understanding and debugging hard as well.
LISTING 31-2: Asynchronous code using EAP
VB
Public Sub RSSFeedLengthAsync(uris As IList(Of Uri))
RSSFeedLengthAsyncHelper(uris.GetEnumerator(), 0)
End Sub
Private Sub RSSFeedLengthAsyncHelper(enumerator As IEnumerator(Of Uri), total As Integer)
If enumerator.MoveNext() Then
Dim client = New WebClient()
AddHandler client.DownloadDataCompleted,
Sub(sender, e)
RSSFeedLengthAsyncHelper(enumerator, total + e.Result.Length)
End Sub
client.DownloadDataAsync(enumerator.Current)
Else
enumerator.Dispose()
End If
End SubC#
public void RSSFeedLengthAsync(IList<Uri> uris)
{
RSSFeedLengthAsyncHelper(uris.GetEnumerator(), 0);
}
private void RSSFeedLengthAsyncHelper(IEnumerator<Uri> enumerator, int total)
{
if (enumerator.MoveNext())
{
var client = new WebClient();
client.DownloadDataCompleted += (sender, e) =>
{
RSSFeedLengthAsyncHelper(enumerator, total + e.Result.Length);
};
client.DownloadDataAsync(enumerator.Current);
}
else
{
enumerator.Dispose();
}
}In .NET Framework v4.0, Task-based Asynchronous Pattern (TAP) was introduced. This pattern uses a single method to represent the start and finish of the async operation. The TAP method returns either a Task or a Task<TResult>, based on whether the corresponding synchronous method returns void or a type TResult. A task is an action that can be executed independently of the rest of the program. In that sense, it is semantically equivalent to a thread, except that it is a more lightweight object and comes without the overhead of creating an OS thread. Along with TAP, .NET v4.0 also introduced Task as the basic unit of parallelism and provided rich support for managing tasks based on the Task Parallel library. You look at task in more detail later in the chapter.
The .NET Framework v4.5 introduced the async/await pattern. The async and await keywords are the heart of async programming. By using those two keywords, you can write async code as if you were writing synchronous code. This pattern is based on TAP and provides a nice programming paradigm to write asynchronous code.
Listing 31-3 shows how the synchronous code from Listing 31-1 would look using async/await.
LISTING 31-3: Async code using async/await
VB
Public Async Function RSSFeedLengthAsync(uris As IList(Of Uri)) As Task(Of Integer)
Dim total As Integer = 0
For Each uri In uris
Dim data = Await New WebClient().DownloadDataTaskAsync(uri)
total += data.Length
Next
Return total
End FunctionC#
public async Task<int> RSSFeedLengthAsync(IList<Uri> uris)
{
int total = 0;
foreach (var uri in uris)
{
var data = await new WebClient().DownloadDataTaskAsync(uri);
total += data.Length;
}
return total;
}The first thing you will notice about Listing 31-3 is how similar it looks to the synchronous code back in Listing 31-1. The control flow is not altered and there are no callbacks. Asynchrony is achieved here by returning Task<int>. Task represents a unit of ongoing work. The caller of this method can use the returned task to check if the task has completed and get the result from the task.
The method is also marked as async, which tells the compiler to compile the method in a special way, allowing parts of the code to be turned into callbacks and automatically creating a Task<int> in which to return the result. DownloadTaskAsync returns a Task<byte[]>, which will complete once the data is available. In this case you do not want to do anything until you have data, so you await the task.
It might appear that await does a blocking call, but in fact what happens is that await puts the rest of the method as a callback and returns immediately. When the awaited task returns, it invokes the callback and resumes the execution of the method from where it left off.
By the time it reaches the return statement, the code has suspended and resumed several times (due to the await call) and returned a Task<int> to the caller. The caller can wait on the task and get the result.
As you saw in the preceding section, the .NET Framework has evolved to support asynchronous programming. The async support in ASP.NET has also evolved at the same pace.
Asynchrony can be achieved in two places — namely, the page life cycle and the application life cycle. A developer can write async code when a page is being processed or when the request is being processed in the ASP.NET pipeline.
In .NET v4.5 the ASP.NET async pipeline has undergone a major change to support async/await and TAP. The new pipeline is task-based and has new base classes that you can derive from when writing async pages, handlers, or modules. Before you look into how to use the async/await pattern in ASP.NET, the following section looks at how threads are managed in ASP.NET.
Threads are precious limited resources on a web server, so you should use them wisely when architecting your applications. When a request comes to a web server, it goes through the following flow in Integrated Mode in IIS. HTTP.sys is the first point of entry. HTTP.sys is a kernel-level driver that posts requests on an I/O completion port on which IIS listens. IIS picks up this request on one of its own thread pool threads and calls ASP.NET. ASP.NET starts to process this thread from the CLR thread pool. Once the request has finished processing and a response needs to be sent back to the client, the response is switched from a CLR thread pool to one from an IIS thread pool.
The reason the flow is designed in such a way is to make the system more scalable and resilient. For example, the request switches from kernel mode (HTTP.sys) to user mode (IIS) so that HTTP.sys can process more requests and hand them over to other listeners, which could be non IIS. If the request were to be processed on one thread, a deadlock or error on that request could bring down HTTP on the entire server. Thus, even though you are paying a penalty for context switching, the benefits of having the kernel be more reliable far outweigh the cost of thread switching.
The other reason for switching the request from the IIS thread pool to the ASP.NET thread pool is improving performance for scenarios where IIS is serving a mixture of static and dynamic requests. This is the most common case for a web server. Static files are cached locally and are served by the IIS static file handler (the request does not enter ASP.NET at all). This also allows IIS to process other requests and hand off to other listeners (non ASP.NET), as in the case of HTTP.sys.
Once the request is processed and the response is ready to be sent, the response is switched from a thread in the CLR thread pool to one in the IIS thread pool. This is done because if the client is on a low bandwidth network, you do not want a CLR thread to be blocked while the response is being sent. You want the CLR thread to return back and process more requests.
These thread pools contain a finite number of threads as well. The IIS thread pool has a thread count of 256, and the CLR thread pool has 100 threads per processor. So you can imagine if your application is running synchronously and is holding up threads by waiting on results of database calls, you will soon run out of threads to process more requests from the users and your website will become sluggish, which will lead to a bad user experience. The following section looks at a few ways to write asynchronous code in ASP.NET. Figure 31-1 shows a flow on how a request is processed by the two thread pools.
As you saw earlier, in .NET Framework 4.5 async/await was introduced as a pattern to write async applications. To support this pattern in ASP.NET, the existing async pipeline has gone through a radical change. The whole of the async pipeline is now task-based. What has not changed, however, is where you can write async code. You can still use async pages, handlers, and modules to write async code. In Chapter 26, you learned about HTTP handlers and modules. This chapter looks at how you can write asynchronous handlers and modules using async/await.
You should write async pages when your page is accessing data from a database or a web service. In this case you do not want to stop the page life cycle and wait on the results of the database or web service call. You can tell ASP.NET to process this page as async by setting a page directive as shown in Listing 31-4. When you set this attribute, ASP.NET switches the processing of the page from a synchronous pipeline to an asynchronous pipeline.
LISTING 31-4: Marking pages as async
<%@ Page Async="true" %>
If you do not use this attribute and use async code in your page, ASP.NET throws an error indicating that you need to mark your page as async. It does this to prevent race conditions from happening because the code is async and if you do not mark it as async, the code will get executed in a synchronous manner and can cause deadlock or race conditions.
Once you have done this, you can write async methods in the page. In ASP.NET 4.5 you can now write async Page_Load events, which was not possible earlier. Listing 31-5 shows how you can call a web service asynchronously from Page_Load.
LISTING 31-5: Asynchronous Page_Load
VB
Protected Async Sub Page_Load(sender As Object, e As EventArgs) Handles Me.Load
Using client = New HttpClient()
Dim customersJson = Await client.GetStringAsync(url)
Dim customers = JsonConvert.DeserializeObject(Of IEnumerable(Of Customer))(customersJson)
results.DataSource = customers
results.DataBind()
End Using
End SubC#
protected async void Page_Load(object sender, EventArgs e)
{
using (var client = new HttpClient())
{
var customersJson = await client.GetStringAsync(url);
var customers = JsonConvert.DeserializeObject<IEnumerable<Customer>>(customersJson);
results.DataSource = customers;
results.DataBind();
}
}Although you can do this, this approach is not recommended for most cases. This is because when you execute Page_Load asynchronously, the page life cycle does not proceed further until the async Page_Load finishes execution. This means that the call to GetStringAsync() will happen asynchronously, but the page life cycle will wait until the await portion of the method finishes executing. In Listing 31-5 the code is simulating a call to an external web service by calling an ASP.NET generic handler. Notice also that the code is using HttpClient, which is a new type in .NET Framework 4.5. HttpClient is a modern HTTP client for .NET and supersedes WebClient.
A recommended approach is to use RegisterAsyncTask to register and execute async operations in a page. The benefit of using this approach is that ASP.NET will not halt the page life cycle execution to wait for async Page_Load to finish, but it will continue with the page life cycle until it reaches the PreRender event of the page. At this point ASP.NET will execute all tasks that were registered using RegisterAsyncTask. PreRender is an event in the life cycle where all the controls are created but the content has not been rendered, so it gives you a chance to do data binding if needed. Listing 31-6 shows how you can define Task with ASP.NET using PageAsyncTask, which is a new type introduced in ASP.NET 4.5.
LISTING 31-6: RegisterAsyncTask using PageAsyncTask
VB
Protected Sub Page_Load(sender As Object, e As EventArgs) Handles Me.Load
RegisterAsyncTask(New PageAsyncTask(Function() GetCustomersAsync()))
End Sub
Public Async Function GetCustomersAsync() As Task
Using client = New HttpClient()
Dim customersJson =
Await client.GetStringAsync("http://localhost:64927/Customers.ashx")
Dim customers =
JsonConvert.DeserializeObject(Of IEnumerable(Of Customer))(customersJson)
results.DataSource = customers
results.DataBind()
End Using
End FunctionC#
protected void Page_Load(object sender, EventArgs e)
{
RegisterAsyncTask(new PageAsyncTask(GetCustomersAsync));
}
private async Task GetCustomersAsync()
{
using (var client = new HttpClient())
{
var customersJson = await client.GetStringAsync("http://localhost:64927/Customers.ashx");
var customers = JsonConvert.DeserializeObject<IEnumerable<Customer>>(customersJson);
results.DataSource = customers;
results.DataBind();
}
}As you saw with writing async pages, there is also a new base type (HttpTaskAsyncHandler) for writing async http handlers using async/await. Listing 31-7 shows how you can write async handlers. This sample downloads the contents from a website and renders them on the page.
LISTING 31-7: Async handlers using HttpTaskAsyncHandler
VB
Public Class Listing31_7 : Inherits HttpTaskAsyncHandler
Public Overrides Async Function ProcessRequestAsync(context As HttpContext) As Task
Using client = New HttpClient()
Dim bingTask = Await client.GetStringAsync("http://bing.com")
context.Response.Write(bingTask)
End Using
End Function
End ClassC#
public class Listing31_7 : HttpTaskAsyncHandler
{
public async override Task ProcessRequestAsync(HttpContext context)
{
using (var client = new HttpClient())
{
var bingTask = await client.GetStringAsync("http://bing.com");
context.Response.Write(bingTask);
}
}
}One of the differences you will see when writing async modules in ASP.NET 4.5 is how the tasks are registered in ASP.NET modules. There is a new type, EventHandlerTaskAsyncHelper, in ASP.NET 4.5. This is a helper that takes in a task as an input and unwraps the beginning and end portions of the async task and passes them as begin and end handlers to the async events that you want to subscribe to in the module. Listing 31-8 shows how you can write async modules using this helper.
LISTING 31-8: Async modules using EventHandlerTaskAsyncHelper
VB
Public Sub Init(context As HttpApplication) Implements IHttpModule.Init
Dim helper As New EventHandlerTaskAsyncHelper(AddressOf DownloadWeb)
context.AddOnBeginRequestAsync(helper.BeginEventHandler, helper.EndEventHandler)
End Sub
Public Async Function DownloadWeb(caller As Object, e As EventArgs) As Task
Using client = New HttpClient()
Dim result = Await client.GetStringAsync("http://bing.com")
End Using
End FunctionC#
public void Init(HttpApplication context)
{
EventHandlerTaskAsyncHelper helper =
new EventHandlerTaskAsyncHelper(DownloadWeb);
context.AddOnBeginRequestAsync(
helper.BeginEventHandler, helper.EndEventHandler);
}
public async Task DownloadWeb(object caller, EventArgs e)
{
using (var client = new HttpClient())
{
var result = await client.GetStringAsync("http://bing.com");
}
}Hopefully, you have a good idea of how you can write asynchronous code in an ASP.NET application. One of the most common scenarios when dealing with external resources such as web service and database servers is the time it takes to get the response. Because both these calls depend on how slow or fast the network access is, the response time varies a lot, and in some cases you might not get the response if the server is down. In these cases you want your application to be resilient to these occurrences, and you should be able to handle such cases easily. A common operation that you will have to do is to check on the status of an existing task and, if it is canceled (due to server unavailability, for example), to take some custom action.
The object that invokes a cancelable operation, for example by creating a new task, passes the token to the caller doing some operation on the task. That operation can, in turn, pass copies of the token to other operations if needed. Listing 31-9 shows how you can specify a timeout for your async task running on the page. In this case if the async operation takes more than the AsyncTimeout value, you get a TimeOutException, which you can handle in your page. Listing 31-10 shows how you can handle this exception in your async code.
LISTING 31-9: Setting a timeout for async processing on a page
<%@ Page Async="true" AsyncTimeout="1" %>LISTING 31-10: Handling a TimeOutException due to AsyncTimeOut
VB
Protected Sub Page_Load(sender As Object, e As EventArgs) Handles Me.Load
RegisterAsyncTask(New PageAsyncTask
(Function() GetCustomersAsync(Context.Request.TimedOutToken)))
End Sub
Public Async Function GetCustomersAsync(ByVal cancelToken As CancellationToken) As Task
Using client = New HttpClient()
Dim response =
Await client.GetAsync("http://localhost:64927/Common/SlowCustomers.ashx", cancelToken)
Dim customersJson = Await response.Content.ReadAsStringAsync()
Dim customers =
sonConvert.DeserializeObject(Of IEnumerable(Of Customer))(customersJson)
results.DataSource = customers
results.DataBind()
End Using
End Function
Protected Sub Page_Error(sender As Object, e As EventArgs) Handles Me.Error
Dim exc As Exception = Server.GetLastError()
If TypeOf exc Is TimeoutException Then
Throw exc
End If
End SubC#
protected void Page_Load(object sender, EventArgs e)
{
RegisterAsyncTask(new PageAsyncTask(GetCustomersAsync));
}
private async Task GetCustomersAsync(CancellationToken cancelToken)
{
using (var client = new HttpClient())
{
var response = await
client.GetAsync("http://localhost:64927/Common/SlowCustomers.ashx", cancelToken);
var customersJson = await response.Content.ReadAsStringAsync();
var customers = JsonConvert.DeserializeObject<IEnumerable<Customer>>(customersJson);
results.DataSource = customers;
results.DataBind();
}
}
private void Page_Error(object sender, EventArgs e)
{
Exception exc = Server.GetLastError();
if (exc is TimeoutException)
{
throw exc;
}
}In the preceding listing, if the call to the external web service takes more than 1 second, then the call to GetAsync() will get timed out. There will be a TimeOutException that will be thrown by this code, which can be handled in the application to display a more meaningful error message or perform some customized error handling.
So far you have looked at the benefits of writing asynchronous applications and how you can write asynchronous code in ASP.NET. By writing asynchronous code, you can efficiently utilize threads to make your applications more scalable and responsive. There is another aspect to the asynchronous pattern that can increase the execution time of the application. This is called parallelism. With the advancements being made in the hardware, many computers and workstations have multiple cores that enable multiple threads to be executed simultaneously. Since the inception of .NET Framework, this was achieved by low-level manipulation of threads and locks, which made parallelizing and debugging your application very hard.
In .NET Framework 4.0, tasks were introduced to simplify parallel application development so you could write efficient and scalable parallel code without having to deal with low-level threads. The framework introduced various kinds of libraries to make it easier to write parallel code in your applications. These ranged from the Task Parallel Library, which makes it easier to parallelize basic for loops, to Parallel LINQ.
This section looks at some basic techniques for parallelizing your ASP.NET applications with the use of tasks. A task represents a unit of work that can be executed as an atomic unit independently. You can combine tasks, compose tasks, wait on tasks to finish, and many more such operations. This section looks at an example of how you can compose tasks to achieve parallelism.
Listing 31-7 showed an example of waiting on a task to download the contents of a website. Now suppose you have three such running tasks and you want to run them in parallel. Listing 31-11 shows how you can do so. Task.WhenAll will execute all the tasks in parallel and will wait until all the tasks have finished executing.
LISTING 31-11: Task.WhenAll to perform tasks in parallel
VB
Public Overrides Async Function ProcessRequestAsync(context As HttpContext) As Task
Using client = New HttpClient()
Dim bingTask = client.GetStringAsync(uri)
Dim microsoftTask = client.GetStringAsync(uri)
Dim twitterTask = client.GetStringAsync(uri)
Await Task.WhenAll(bingTask, microsoftTask, twitterTask)
End Using
End FunctionC#
public async override Task ProcessRequestAsync(HttpContext context)
{
using (var client = new HttpClient())
{
var bingTask = client.GetStringAsync(uri);
var microsoftTask = client.GetStringAsync(uri);
var twitterTask = client.GetStringAsync(uri);
await Task.WhenAll(bingTask,microsoftTask,twitterTask);
}
}One of the most important aspects of making your application more scalable is tuning your web server to get the most out of it. When you install ASP.NET on IIS, the default configuration of thread pools and concurrency is set to support most of the common cases, so it might or might not meet the needs of your application. It is very important to understand what these default limits are and what kinds of applications they target. It is equally important to understand what your application needs. For example, the default configuration of IIS and ASP.NET is not suited for building real-time applications, which need to maintain a long-running server connection, and the server needs to be optimized for high concurrency and high-latency calls. This section looks at some of the tweaks you can make on your web server to optimize performance for your application.
So far you have seen the benefits of using asynchronous processing in your application. It does make your application more scalable and responsive. However, there is a wise saying: “With great power comes great responsibility.” Although async processing does increase your application performance, it can create havoc if not used wisely. Poorly written async code can lead to race conditions, thread deadlock, and synchronization problems. All of these problems are difficult to debug and fix, and can waste hours of precious developer time in investigations. In this section, you learn some of the common pitfalls when writing asynchronous code and how you can avoid them.
In this chapter you looked at asynchronous processing and why you need it. As your applications become more popular and attract more traffic, you need to make sure that they scale well. You should write your applications in a way that enables them to effectively utilize the server resources and perform well.
The .NET Framework and ASP.NET have come a long way in supporting asynchronous processing. You saw the different patterns of writing async code. .NET Framework 4.5 introduced the async/await pattern, which makes writing async code similar to writing synchronous code. The code flows look similar in both approaches.
ASP.NET supports async/await when you want to write async pages, handlers, and modules. In ASP.NET 4.5, the pipeline has undergone a huge change to support async/await. You also looked at some tweaking that you can do on your web server to make your application scale well under load.
Finally, you looked at some of the pitfalls of writing bad asynchronous code and some of the ways by which you can avoid these pitfalls.