2006年10月11日
Cache.Insert("CacheKey", value, null, DateTime.Now.AddSeconds(30), TimeSpan.Zero, CacheItemPriority.Normal, null);
Cache.Insert("CacheKey", value, null, DateTime.Now.AddSeconds(30), System.Web.Caching.Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);
前者很少有命中.
后者才是按时间命中.
System.Web.Caching.Cache.NoSlidingExpiration实际也是设置为TimeSpan.Zero.
MSDN如是说:
使用后,该字段将 slidingExpiration 参数设置成 TimeSpan.Zero 字段,该字段具有常数值零。缓存项依照与 Insert 或 Add 方法调用相关的 absoluteExpiration 参数设置过期。
无法同时设置 absoluteExpiration 和 slidingExpiration 参数。如果要让缓存项在特定时间过期,可将 absoluteExpiration 参数设置为特定时间,并将 slidingExpiration 参数设置为 NoSlidingExpiration。
如果要让缓存项自最后一次访问该项后的某段时间之后过期,可将 slidingExpiration 参数设置为过期间隔,并将 absoluteExpiration 参数设置为 NoAbsoluteExpiration。
MSDN:示例
Cache.Insert("DSN", connectionString, null, DateTime.Now.AddMinutes(2), TimeSpan.Zero, CacheItemPriority.High, onRemove);
而这种方法,用户访问后,立即过期,缓存实际上根本无法命中.
MSDN上的示例都是如此.
?????
测试期...
目标,将建立成高负载的博客系统,分布式缓存策略等.
开篇纪念.
Enabling full text search in T-SQL is not as popular as doing it with the Enterprise Manager. Nonetheless, it can be useful in certain situations. Here are the steps to implement FTS in T-SQL.
- Enable full text on the database by executing the following SP:
EXEC sp_fulltext_database 'enable'
- Create the catalog (if it does not exist):
EXEC sp_fulltext_catalog 'MyCatalog','create'
- Add a full text index on a table:
EXEC sp_fulltext_table 'Products', 'create', 'MyCatalog', 'pk_products'
EXEC sp_fulltext_table 'Categories', 'create', 'MyCatalog', 'pk_categories'
- Add a column to the full text index:
EXEC sp_fulltext_column 'Products', 'ProductName', 'add'
EXEC sp_fulltext_column 'Categories', 'Description', 'add'
- Activate the index:
EXEC sp_fulltext_table 'Products','activate'
EXEC sp_fulltext_table 'Categories','activate'
- Start full population:
EXEC sp_fulltext_catalog 'MyCatalog', 'start_full'
Here are some examples on how to use this procedure:
-
USE Northwind
GO
SELECT ProductId, ProductName, UnitPrice
FROM Products
WHERE CONTAINS(
ProductName, ' "sasquatch " OR "stout" '
)
GO
-
USE Northwind
GO
SELECT CategoryName
FROM Categories
FREETEXT (
Description, 'sweetest candy bread and dry meat'
)
GO
This article highlights features new to SQL Server 2005 that make retrieving a particular paged subset of data incredibly easy and efficient. For those who've yet to switch to SQL Server 2005, check out
A More Efficient Method for Paging Through Large Result Sets. The stored procedure presented in that article can be plugged in for the stored procedure examined in this article if needed.
Introduction
A common pattern in web development is providing paged access to data. Rather than displaying the entire contents of a report or database table to an end user, developers often show only a subset of records per web page, with controls for moving from page to page. With ASP.NET 1.x, the DataGrid made paging incredibly simple - just set the AllowPaging property to True and add a few lines of code in the PageIndexChanged event handler and you were done! ASP.NET 2.0's GridView makes the process even simpler - just check the Enable Paging option from the GridView's smart tag - no code needed.
Of course nothing is free in life, and the tradeoff you make with the ease of checking a checkbox to enable paging (or, in the DataGrid's case, writing a couple lines of code) is performance. Out of the box, the DataGrid and GridView use default paging, which is a simple paging model that returns all of the records for each every page of data shown. When paging through small amounts of data (dozens to a hundred or so records), this inefficiency is likely outweighed by the ease of adding the feature. However, if you want to page through thousands, tens of thousands, or hundreds of thousands of records the default paging model is not viable.
The alternative to default paging is custom paging, in which you are tasked with writing code that intelligently grabs the correct subset of data. It requires a bit more work, but is essential when dealing with sufficiently-sized data. I discuss how to implement custom paging in ASP.NET 1.x in my book ASP.NET Data Web Controls Kick Start. In this article we'll look at how to implement custom paging in ASP.NET 2.0 using SQL Server 2005's new ROW_NUMBER() feature. (For more information on SQL Server's new ranking features, including ROW_NUMBER(), see Returning Ranked Results with Microsoft SQL Server 2005.)
Read on to learn more!
Default Paging vs. Custom Paging
The GridView in 2.0 (and the DataGrid in 1.x) offers two paging models: default paging and custom paging. The two models provide a tradeoff between performance and ease of setting up/configuring/using. The SqlDataSource control uses default paging (although you can wrestle it into using custom paging); the ObjectDataSource uses default paging by default, but has an easy mechanism to indicate that it should use custom paging. Keep in mind that the GridView merely displays data; it's the GridView's data source control that is actually retrieving data from the database.
With default paging, each time a new page of data in displayed in the GridView, all of the data is requeried from the and returned from the GridView's data source. Once all of the data has been returned, the GridView selectively displays part of the entire set of data, based on the page of data the user is viewing and how many records per page are displayed. The key thing to understand here is that every single time a page of data is loaded - be it on the first page visit when viewing the first page of data or when the user postsbacks after requesting to view a different page of data - the entire data result is retrieved.
For example, imagine that you work at an eCommerce company and you want to allow the user to page through a list of the 150 products your company sells. Specifically, you want to display 10 records per page. Now, when a user visits the web page, all 150 records will be returned by the data source control, but the GridView will display the first 10 products (products 1 to 10). Next, imagine that the user navigates to the next page of data. This will cause a postback, at which point the GridView will rerequest all 150 records from the data source control, but this time only display the second set of 10 (products 11 to 20).
| Caching and the SqlDataSource |
The SqlDataSource allows for the DataSet it returns to be cached by simply setting the EnableCaching property. With a cached DataSet, stepping to another page does not require the database be requiried since the data being paged through is cached in memory. However, on the initial page load the same problem arises - all of the data must be loaded into the cached DataSet. Furthermore, you must worry about stale data with this approach (although if you use SQL cache dependencies, then this point is moot).
Even with caching the DataSet, my unscientific tests found custom paging to be twice as fast... When we examine the performance metrics later, though, you'll see that this cached approach far outshines the non-cached approach. (But it still doesn't beat the custom paging approach!)
For more on caching the DataSet returned by the SqlDataSource see Caching Data With the SqlDataSource.
|
With custom paging, you, the developer, have to do a bit more work. Rather than just being able to blindly bind the GridView to a data source control and check the "Enable Paging" checkbox, you have to configure the data source control to selectively retrieve only those records that should be shown for the particular page. The benefit of this is that when displaying the first page of data, you can use a SQL statement that only retrieves products 1 through 10, rather than all 150 records. However, your SQL statement has to be "clever" enough to be able to know how to just snip out the right subset of records from the 150.
| The Performance Edge of Custom Paging |
| Realize that custom paging provides better performance than default paging because only those database records that need to be displayed are retrieved. In our products example, we assumed there were 150 products, showing 10 per page. With custom paging, if the user stepped through all 15 pages of data, precisely 150 records would have been queried from the database. With default paging, however, for each page of data, 150 records would have been accessed, leading to a total number of retrieved records of 15 times 150, or 2,250!
While custom paging exhibits better performance, default paging is much easier to use. Therefore, I would encourage you to use default paging if the data you are paging through is relatively small and/or the database server is not heavily trafficked. If you have several hundred, thousands, or tens of thousands of records you are paging through, by all means use custom paging. However, for paging through something like the ASPFAQs.com database, which only has, currently, ~200 FAQs, default paging is sufficient.
(Of course, if you use default paging on a small table with, say, 75 records, you are assuming that over time the table's row count will stay low. There will be some unhappy customers if you use default paging on that small table which later grows to be a table with 7,500 records!)
|
Efficiently Getting Back a Page of Data with SQL Server 2005
As discussed in an earlier 4Guys article, Returning Ranked Results with Microsoft SQL Server 2005, SQL Server 2005 introduces a number of new keywords for returning ranked results. In particular, the ROW_NUMBER() keyword enables us to associate a sequentially-increasing row number for the results returned. We can use ROW_NUMBER(), then, to get a particular page of data using a query like the following:
SELECT ... FROM (SELECT ... ROW_NUMBER() OVER(ORDER BY ColumnName) as RowNum FROM Employees e ) as DerivedTableName WHERE RowNum BETWEEN @startRowIndex AND (@startRowIndex + @maximumRows) - 1
|
Here @startRowIndex is the index of the row to start from and @maximumRows is the maximum number of records to show per page. This query returns the subset of records whose ROW_NUMBER() is between the starting index and the starting index plus the page size.
To help concretize this concept, let's look at the following example. Imagine that we have an Employees table with 5,000 records (business is good!). The following query:
SELECT RowNum, EmployeeID, LastName, FirstName FROM (SELECT EmployeeID, LastName, FirstName ROW_NUMBER() OVER(ORDER BY EmployeeID) as RowNum FROM Employees e ) as EmployeeInfo
|
Would return results like:
| RowNum |
EmployeeID |
LastName |
FirstName |
| 1 |
1000 |
Smith |
Frank |
| 2 |
1001 |
Jackson |
Lucy |
| 3 |
1011 |
Lee |
Sam |
| 4 |
1012 |
Mitchell |
Jisun |
| 5 |
1013 |
Yates |
Scott |
| 6 |
1016 |
Props |
Kathryn |
| ... |
| 5000 |
6141 |
Jordan |
DJ |
Notice that even though the EmployeeID fields may have gaps and may not start at 1, the ROW_NUMBER() value starts at 1 for the first record and steadily increases. Therefore, if we want to view 10 records per page, and we want to see the third page, we know that we want records 31-40, and can accomplish that in a simple WHERE clause.
Configuring the ObjectDataSource for Custom Paging
As aforementioned, the SqlDataSource isn't designed to provide custom sorting capabilities. The ObjectDataSource, on the other hand, was designed to support this scenario. The ObjectDataSource is a data source control that's designed to access data from an object. The object can retrieve its data however it likes, be it from a Web Service, a database, the file system, an XML file... whatever. The ObjectDataSource doesn't care, it simply acts as a proxy between the data Web control that wants to consume the data (such as a GridView control) and the underlying data that the object provides. (For more information on the ObjectDataSource see the ObjectDataSource Control Overview.)
When binding a data Web control to an ObjectDataSource the "Enable Paging" option is available. If you've not specifically set up the ObjectDataSource to support custom paging, the paging provided will be of the default paging flavor. To setup custom paging with the ObjectDataSource you need to be using an object that provides the following functionality:
- A method that takes in as its final two input parameters two integer values. The first integer value specifies the starting index from which to retrieve the data (it's zero-based), while the second integer value indicates the maximum number of records to retrieve per page. This method needs to return the precise subset of data being requested, namely the data starting at the specified index and not exceeding the total number of records indicated.
- A method that returns an integer value specifying the total number of records that are being paged through. (This information is used by the data Web control when rendering the paging controls, since it needs to know how many total pages of data there are when showing page numbers or when deciding whether to enable the Next link.)
If you are using an underlying object that provides these features, configuring the ObjectDataSource to support custom paging is a breeze. Just set the following ObjectDataSource properties:
- Set
EnablePaging to True
- Set
SelectMethod to the method that accepts the starting index and maximum number of rows input parameters
- Set the
StartRowIndexParameterName to the name of the integer input parameter in your SelectMethod that accepts the starting index; if you do not provide this value it defaults to startRowIndex
- Set the
MaximumRowsParameterName to the name of the integer input parameter in your SelectMethod that accepts the maximum number of rows to return; if you do not provide this value it defaults to maximumRows
- Set
SelectCountMethod to the method that returns the total number of records being paged through
That's it. Once you've done the above, the ObjectDataSource will be using the custom paging functionality. Of course, the hard part of this all is creating the underlying object that can intelligently grab the right subset of data. But once you have that object, configuring the ObjectDataSource to utilize custom paging is just a matter of setting a few properties.
Creating an Object That Supports Custom Paging
In order to bind an ObjectDataSource to a GridView we need to first have an underlying object that the ObjectDataSource will use, and this object must have methods for accessing a particular subset of the data and returning the number of rows to be paged through. As discussed in Joseph Chancellor's article, Using Strongly-Typed Data Access in Visual Studio 2005 and ASP.NET 2.0 and Brian Noyes's article Build a Data Access Layer with the Visual Studio 2005 DataSet Designer, creating objects that can be bound to the ObjectDataSource is a breeze in Visual Studio 2005. The first step is to define the stored procedures (or SQL queries) that will be used to populate the strongly-typed DataSets returned by these object's methods.
The download, available at the end of this article, has a sample database with 50,000 employee records (plus an easy way to add additional records in bulk). The database includes three stored procedures that are used by the two custom paging demos:
GetEmployeesSubset(@startRowIndex int, @maximumRows int) - returns at most @maximumRows records from the Employees table starting at @startRowIndex when ordered by EmployeeID.
GetEmployeesRowCount - returns the total number of records in the Employees table.
GetEmployeesSubsetSorted(@sortExpression nvarchar(50), @startRowIndex int, @maximumRows int) - this sproc returns a page of data sorted by a specified sort expression. This allows the a page of data ordered by, say, Salary, to be returned. (GetEmployeesSubset returns records always ordered by EmployeeID.) This flexibility is needed if you want to create a sortable GridView that employs custom paging.
We won't be discussing implementing the sortable, custom pageable GridView in this article, although examples are included in this article's download; see Sorting Custom Paged Results for a look at how to create a custom paging and bi-directional sortable UI...
Once these stored procedures are created, I created the underlying object by adding a Typed DataSet to my project (Employees.xsd). I then added three methods, one against each of the stored procedures listed above. I ended up with an EmployeesTableAdapter object with methods GetEmployeesSubset(startRowIndex, maximumRows) and GetEmployeesRowCount() that can then be plugged into the ObjectDataSource's properties. (For step-by-step instructions on creating the Typed DataSet, see Using Strongly-Typed Data Access in Visual Studio 2005 and ASP.NET 2.0 and Scott Guthrie's blog entry Building a DAL using Strongly Typed TableAdapters and DataTables in VS 2005 and ASP.NET 2.0.)
Comparing the Performance of Default Paging and Custom Paging
To compare the performance between default and custom paging against the database included in this article's download (which has a table with 50,000 records), I used both SQL Profile and ASP.NET tracing to ascertain relative performance differences. (These techniques were done very unscientifically on my computer, which had other processes running in the background and such. While the results can hardly be called conclusive, I think the performance differences between the two methods clearly highlights custom paging's advantages.)
| SQL Profiler Results |
| Default Paging |
(Selecting All Records from Employees) |
| Duration (sec) |
Reads |
| 1.455 |
383 |
| 1.405 |
383 |
| 1.434 |
383 |
| 1.394 |
383 |
| 1.365 |
383 |
| Avg: 1.411 |
Avg: 383 |
|
| Custom Paging |
(Selecting a Page of Records from Employees) |
| Duration (sec) |
Reads |
| 0.003 |
29 |
| 0.000 |
29 |
| 0.000 |
29 |
| 0.003 |
29 |
| 0.003 |
29 |
| Avg: 0.002 |
Avg: 29 |
|
| ASP.NET Trace Results |
| Default Paging |
(Selecting All Records from Employees) |
| Page Load Duration (sec) |
| 2.34136852588807 |
| 2.35772228034569 |
| 2.43368277253115 |
| 2.43237562315881 |
| 2.33167064529151 |
| Avg: 2.379363969 |
|
| Custom Paging |
(Selecting a Page of Records from Employees) |
| Page Load Duration (sec) |
| 0.0259611207569677 |
| 0.0280046765720224 |
| 0.0359054013848129 |
| 0.0295534767686955 |
| 0.0300096800012292 |
| Avg: 0.029886871 |
|
| Cached SqlDataSource |
| (Selecting All Records, But Caching Them) |
| Page Load Duration (sec) |
| 2.39666633608461 |
| 0.0431529705591074 |
| 0.0443528437273452 |
| 0.0442313199023898 |
| 0.0491523364002967 |
| Avg: 0.515511161 |
|
As you can see, the custom paging is roughly two order of magnitudes faster than the default paging. At the database level, the GetEmployeesSubset(@startRowIndex int, @maximumRows int) stored procedure is about 470 times faster than the simple SELECT statement that returns all records from the Employees table. Custom paging is about 120 times faster than default paging at the ASP.NET level. The reduction is performance gain is probably due to expensive workloads common to both approaches, namely setting up the database connection and issuing the command. Regardless, two orders of magnitude is a very big difference in the world of performance. And this disparity would be more pronounced with larger data sets or a server that was experiencing any kind of load.
The cached SqlDataSource has a high cost when the cache is empty, as it must go to the database and get all of the records. The frequency that the cache needs to be reloaded depends upon free resources on the web server (if you have low resources available, the cached DataSet may get evicted) and your cache expiration policy. After the data has been cached, though, it greatly improves in performance and is comparable to the custom paging approach. The 0.516 second average time would be amortized to closer to 0.05 seconds as more requests were served with the cached data.
Conclusion
As with the DataGrid in ASP.NET 1.x, the GridView in 2.0 offers two flavors of paging: default and custom. Default paging is easier to setup, but involves requerying the database when viewing each and every page of data. Custom paging, however, more intelligently just grabs those records needing to be displayed and therefore affords much higher degree of performance. SQL Server 2005 simplifies obtaining the precise subset of records for an arbitrary page due to its ability to rank results, which includes the ROW_NUMBER() feature.
If you are building web applications that need to scale or either now or in the future will allow users to page through potentially large data sets, it behooves you to implement custom paging.
Happy Programming!
By
Scott Mitchell
David Hayden blogged about a cool new ROW_NUMBER() function that SQL 2005 provides that got me excited and inspired to write a little code last night.
You can download the sample app I built here.
Features of the Sample
This sample is a self-contained ASP.NET application that demonstrates a few things:
- How to implement a data-result search page built with ASP.NET 2.0 and Atlas that can present hundreds of thousands of row results using the ASP.NET 2.0 GridView control. The results are formatted using a “paging” based UI model – where 15 results per page are displayed, and the user can skip from page to page to see their data. For kicks I also added support for editing and deleting each individual row.
- How to optimize the number of rows returned by the SQL Server to the middle-tier web-server. This sample demonstrates how a large query result (think 1000s, 10s of thousands or 100s of thousands of rows) can be paged efficiently so that only the 15 rows currently on display in a page are ever retrieved by the web-server from the SQL database (this is done using the ROW_NUMBER() function that David describes above, as well as the support for optimized paging provided by the GridView and ObjectDataSource controls). This avoids your web-server and SQL server grinding to a halt when you execute large queries, and makes for much more scalable performance.
- How to easily implement Ajax UI support for paging/editing/deleting on top of hundreds of thousands of rows of data (so no full page refreshes – instead it only updates the portion of the page that changes). This took me only 60 seconds to-do, and uses the same <atlas:updatepanel> control support I talked about in this earlier blog post.
- How to easily implement Ajax UI support for adding “auto-suggest” behavior to controls like text-boxes. The December release of the Atlas Project provides a super-easy server control called the <atlas:autocompleteextender> control that you can point at a TextBox, and that will then call a web-service to provide a list of suggestions when a browser user starts typing in the text-box. This sample demonstrates how to use this to auto-suggest items based on the contents in the database.
- How to implement a business class façade/wrapper around a data access layer. This is in turn used by the new ASP.NET 2.0 ObjectDataSource control for databinding scenarios. For this simple sample, I could have just used the data access layer built by using File->Add New Item->DataSet from the ObjectDataSource control directly (like I did with my earlier To-do List sample), and basically eliminate the need to write any code in the app. But I wanted to use this sample to help demonstrate how to build a richer business library layer abstraction that was separate from my data access layer. The business layer implementation in this sample is pretty trivial (and doesn’t really add much value), but it demonstrates a skeleton of how/where you could easily add business logic rules that were cleanly separated from your data layer.
How To Run and Use the Application
To setup the sample, follow the below steps:
1) Have VS 2005 or Visual Web Developer and SQL Express installed. Visual Web Developer and SQL Express are both free downloads and provide everything you need to build, debug and run the application. You can download them from here.
2) Download this .zip file containing the sample. Expand it into any directory you want.
3) Using Visual Web Developer or VS 2005, choose File->Open Web-site and point to the root directory of the sample sub-directory (the one with the files in it).
4) Select the “PagedData.aspx” file and hit run.
This will bring up a page that looks like this:
This allows you to query for lists by categories in your database. The default database in the sample doesn’t have any list items, so first we’ll want to create some. To-do this click the “Bulk Add” link.
Create a category called “test” and add 100 items to it. This will create a bunch of sample data in the database that will help simulate paging. Then repeat this with a new category called “test2” and add 10,000 items to it. Then repeat this with a new category called “test3” and add 100,000 items to it. Note that this last category will probably take a few minutes to create (since the logic in my app is simple and just adds them one row at a time).

Then click on the “Return to Paged Data Page” link and start typing a search string in the textbox. Pause slightly after you type “tes” – and then notice how the auto-suggest box will list the three new categories you added:

These are being populated using the <atlas:autocompleteextender> server control (which is hitting a web-service on the server to lookup the available categories – and so will return the three new ones we just created).
Search using “test3” (the one with 100,000 items), and you’ll see this paged view of 100,000 items in the GridView:

Each page of the screen contains 15 items from the database. You can switch around to see the different pages. Click the “…” link to go to pages 11-20. Click “Last” to jump to the final page of data:

Note that as you choose the different pages, the only thing that gets updated is the grid of data – the overall page does not refresh. This is because the GridView control is wrapped using the <atlas:updatepanel> control which allows incremental Ajax refreshes of the page (no code required to accomplish this <g>).
Page back a few screens, and then click “edit” on one of the rows. Note that you can now edit and update its items. You can also click “delete” on a row to remove one. Both operations occur in an Ajax way:

How is this Application Built
Here is what the solution directory looks like in Visual Web Developer:

It contains two pages – PagedData.aspx and AddData.aspx – that are each based on the Site.master master-page. It also contains one web-service – categories.asmx – which is used by the <atlas:autocompleteextender> control.
The database is implemented in PagingSample.mdf and contains 1 simple table called “Lists” (note: you can create new SQL Express databases by select File->Add New Item->Database File):

ListId is the primary key, and the Category column has been marked to be indexed (note: this is important since we’ll be creating 100,000+ rows). To set indexes, right click on a column in the table designer and choose “Indexs/Keys” and make sure that “Categories” is being indexed.
The database has one stored procedure called “GetPagedLists” that was created using the ROW_NUMBER() approach described in David’s blog:

Note: you can double-click on the sproc name in the solution explorer to open and edit it.
The data access layer was implemented using a DataSet component (choose File->Add New Item->DataSet to create one). It is defined declaratively within the “MyDataLayer.xsd” file and contains two table-adapters:

GetPagedListData goes against the SPOC above, but all other methods are normal SQL queries (note: the GetListCountByCategory method is defined as: SELECT COUNT(*) FROM Lists where category=@category and returns a scalar integer value).
I could have just used the DAL layer directly from the UI tier, but instead chose to wrap the DAL with a business layer façade that looks like this:

The two significant methods for our paging sample are “GetListCountByCategory” and “GetListsByCategory”. Here are the simple implementations of them in our business façade layer (note: right now they just thinly wrap the data access layer and expose the row data as custom “ListItem” types – so they don’t provide much value over calling the DAL directly, but do demonstrate how you could add your own custom logic around it):

Databinding a paged GridView control to this middle-tier object is easy (and automated if you are using the WYSIWYG designer in Visual Web Developer). Just set the “AllowPaging=true” property on the GridView and set the PageSize value you want (for this sample I am using 15 rows per page). Then point the GridView at an ObjectDataSource control, which is configured to use the ListManager business façade class. It has the “AllowPaging” attribute set to true, and has the “SelectMethod” and “SelectCount” properties pointing to the “GetLisByCategory” and “GetListCountByCategory” methods above.

Now, when you run the application, the GridView binds against the ObjectDataSource which will invoke the “GetListCountByCategory” method on our “ListManager” business façade class to calculate the total number of rows to page, and then invoke the “GetListByCategory” method passing the appropriate “category” parameter along with the “startRowIndex” and “maximumRows” parameter values that correspond to the page index that the GridView is currently on.
The beauty is that because we only retrieve and return the 15 rows of data we actually need from the database as part of this operation, the result is fast and scalable. As a browser user pages back and forth on the data within the GridView across the 100,000 rows of data, all UI paging logic and update/edit semantics are handled for us.
The last little implementation note to call out is the use of the <atlas:AutoCompleteExtender> control to auto-suggest items within the Category TextBox. This is a new control provided by the Atlas December drop, and is simple to implement. Basically, you just add the control to your page and point at the TextBox you want to complete, along with a web-service URL and method to call when the browser user types three letters in the TextBox -- these are then passed to the web-service, which can provide a list of suggestions back:
The implementation of the web-service method then looks like this within this sample:
Hope this helps and proves to be a useful sample. Thanks again to David for the blog post on the new ROW_NUMBER() function in SQL 2005 that makes this much easier to implement.
Hope this helps – and happy new year!
Scott
P.S. Note that with SQL 2000 you can use the same approach as above, although the SPROC ends up being slightly more complicated.
The configuration API in .NET 2.0 gives us the ability to read and update configuration files, including web.config and machine.config files. You can read and write configuration files for your application, for another application on the same machine, or even an application on a different server. In this article, we will take a look at some of the highlights of the configuration API from the perspective of an ASP.NET developer, including how to use encryption and alternate configuration files.
AppSettings and Connection Strings
Two common tasks in ASP.NET development are reading application setting strings and connection strings from the configuration file. In .NET 2.0 these settings reside in the <appSettings> and <connectionStrings> respectively. A sample web.config file for an ASP.NET site might look like the following.
<?xml version="1.0"?>
<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">
<appSettings>
<add key="message" value="Hello World!" />
</appSettings>
<connectionStrings>
<add name="AdventureWorks" connectionString="..."/>
<add name="pubs" connectionString="..."/>
</connectionStrings>
<system.web>
<compilation debug="true" />
<authentication mode="Windows"/>
<identity impersonate="true"/>
</system.web>
</configuration>
The configuration API for ASP.NET developers begins with the WebConfigurationManager class in the System.Web.Configuration namespace. The WebConfigurationManager includes static (shared) properties to fetch application settings and connection string. For example, to read the “message” appSetting from the web.config we could use the following code:
string message;
message = WebConfigurationManager.AppSettings["message"];
Similarly, if we want to grab the second connection string, the connection with the name of “pubs”, we could use the following code:
string connectionString =
WebConfigurationManager.ConnectionStrings["pubs"].ConnectionString;
The configuration API makes easy work of reading any setting in a configuration file using the GetSection static method. GetSection takes an XPath expression to indicate the section you want to get, and you can coerce the resulting object reference into a strongly typed reference for built-in section types. For instance, there is an AuthorizationSection class to manipulate the settings inside the <authorization> section, and a PagesSection class to manipulate the settings in the <pages> section.
If we want to write out the value of the impersonate attribute in the <identity> section of web.config, we could use the following:
protected void readImpersonationButton_Click(object sender, EventArgs e)
{
// note: currently broken in BETA2, works in post BETA2 builds.
// in BETA2 GetSection returns a wrapper
// that will not cast to IdentitySection
IdentitySection section;
section = WebConfigurationManager.GetSection("system.web/identity")
as IdentitySection;
if (section != null)
{
WriteMessage("Impersonate = " + section.Impersonate);
}
}
private void WriteMessage(string message)
{
// this method assumes a PlaceHolder control
// on the web form with the ID of messagePlaceHolder
HtmlGenericControl generic = new HtmlGenericControl();
generic.InnerHtml = message;
messagePlaceHolder.Controls.Add(generic);
}
Modify Configuration Files
The WebConfigurationManager class also allows us to open a web configuration for update using the static method OpenWebConfiguration. We can open a configuration file inside of our application by passing just a relative path. We can also read configuration files in other applications by passing the IIS sitename and a virtual directory. It’s even possible to open application configuration files on another machine.
If we want to toggle the debug attribute in the <compilation>section of the web.config for the current application from true to false and back again, we could use the following code in the event handler for a button click event:
protected void toggleDebugButton_Click(object sender, EventArgs e)
{
Configuration config;
config = WebConfigurationManager.OpenWebConfiguration("~");
CompilationSection compilation;
compilation = config.GetSection("system.web/compilation")
as CompilationSection;
if (compilation != null)
{
compilation.Debug = !compilation.Debug;
config.Save();
WriteMessage(
"Debug setting is now: " + compilation.Debug
);
}
}
Using a strongly typed CompilationSection object allows to use to read and write the attributes inside a <compilation> section. We can make changes to this section (and any others) and save all the changes at once using the Save method of the System.Configuration.Configuration object returned from OpenWebConfiguration.
There are a few caveats to updating configuration files. First, your application will need write permissions on the file. Typically, the NETWORK SERVICE and ASPNET accounts used by the runtime do not have write permissions on files and directories in an application’s home folder. One safe way to approach the problem is the technique used here - a combination of Windows authentication and impersonation. These settings allow the request to execute with the identity of the client. If the client has write permissions to the configuration file, the above snippet will be successful.
Another caveat is that the ASP.NET runtime watches web.config and will restart the web application when a change occurs. Specifically, the runtime will create a new instance of your application inside of a new AppDomain anytime you write to web.config. A restart can have a performance impact, so writing to web.config should not occur often.
If you need more control over permissions and application restarts when it comes to updating web.config, you might want to look at using external configuration sources, as described in the next section.
Using an External Configuration Source
You can take any configuration section and place the section into it’s own, dedicated file.. As an example, let’s take a look at a new version of our web.config file:
<?xml version="1.0"?>
<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">
<appSettings configSource="appSettings.config"/>
<connectionStrings configSource="connections.config"/>
<system.web>
<compilation debug="true" />
<authentication mode="Windows"/>
<identity impersonate="true"/>
</system.web>
</configuration>
In this example, we’ve moved our <appSettings> and <connectionStrings> sections into external files. These external files will be XML fragments containing a single section element, for instance, the appSettings.config file will look like the following.
<appSettings>
<add key="message" value="Hello World!"/>
</appSettings>
Using an external configuration source can be useful in a number of scenarios. For instance, you could place a section into an external configSource if you needed an easy method to swap settings for the section depending on the environment (development, test, or production).
You could also use an external configSource if you needed granular control over permissions. For instance, you could lock down your web.config file so that only Administrators could modify the file, but keep the <appSettings> section in an external file that additional roles could modify
There is an additional benefit to using an external file, and that is the ability to have some amount of control over application restarts. If the web.config files changes, the application will restart – there is no alternative. However, if you move a section into an external file, you can tell the runtime if it should, or should not restart the application when the external configuration source changes.
If you look inside of the machine.config file for your computer, in the <configSections> area, you’ll see where a section handler type is defined for each configuration section. Each <section> entry can include an attribute: restartOnExternalChanges. Notice the <section> configuration for the appSettings section uses restartOnExternalChanges="false". This means if your appSettings section lives in an external file, and changes are made to the file, the application will not restart, but you will see the new values in calls to WebConfigurationManager.AppSettings.
Use restartOnExternalChanges with some care, as some parameters can truly only take effect if the application restarts. If you do set restartOnExternalChanges to false for a section, make sure not to cache the parameters for the section in our application, and always read values through the WebConfigurationManager.
Using Encryption
Encrypting an entire section of a configuration file is straightforward with the 2.0 configuration API. There are several configuration areas where sensitive information may appear, for instance, the <connectionStrings> section may contain database usernames and passwords, the <identity> section will contain a username and password when you need the runtime to impersonate a fixed identity. You may even keep a password for a third party web service in appSettings or a custom section. Whenever secrets like these appear, consider encrypting the section instead of leaving the secrets and passwords in plain text.
Note: there are sections that may contain passwords that you cannot encrypt, namely the <processModel> section. You can still use the Aspnet_setreg.exe tool to store a password for this section securely.
The following section of code shows how easy it is to protect (encrypt) and unprotect (decrypt) an entire configuration section. (Note: you do not need to unprotect a section in order to read configuration settings from the section. The runtime will read the encrypted data and perform the decryption necessary for your application to read the plain text values. The Unprotect method call is here to demonstrate how to return a section to unencrypted form).
protected void toggleEncryptionButton_Click(object sender, EventArgs e)
{
Configuration config;
config = WebConfigurationManager.OpenWebConfiguration("~");
ConnectionStringsSection section;
section = config.GetSection("connectionStrings")
as ConnectionStringsSection;
if (section.SectionInformation.IsProtected)
{
section.SectionInformation.UnprotectSection();
}
else
{
section.SectionInformation.ProtectSection(
"DataProtectionConfigurationProvider"
);
}
config.Save();
WriteMessage("connections protected = " +
section.SectionInformation.IsProtected);
}
If we were to examine our web.config file after toggling encryption to on, we’d notice the configuration API has added some additional information:
<?xml version="1.0"?>
<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">
<protectedData>
<protectedDataSections>
<add name="connectionStrings"
provider="DataProtectionConfigurationProvider"
inheritedByChildren="false" />
</protectedDataSections>
</protectedData>
<appSettings configSource="appSettings.config"/>
<connectionStrings configSource="connections.config"/>
<system.web>
<compilation debug="true" />
<authentication mode="Windows"/>
<identity impersonate="true"/>
</system.web>
</configuration>
In addition, we’d find our connectionStrings.config file would contain a cipherValue instead of plaintext connection strings. (Note: we do not need to use an external configuration source to take advantage of encryption, the configuration API would have happily encrypted the connection strings section if it lived inside of web.config).