.Net 技术

Google 博客搜索 Ping 服务应用

venjiang — Thu, 14 Jun 2007 17:02:49 GMT

Google Ping 介绍

通过 Google“博客搜索”Ping API, 用户可以程序化的方式将博客内容的更新通知给 Google“博客搜索”引擎。这对于经常更新博客内容的用户尤其有用。博客服务提供商的管理人员也可以利用此API将其平台上的博客内容变化向 Google 通告，以便 Google“博客搜索”及时抓取来自这一服务提供商的最新内容。为设置对 Google“博客搜索”的自动 Ping 机制，请按照如下所述设置XML-RPC客户端或REST客户端以发送请求。您可以任选一种方法进行通知；两者都将按照相同的方式进行处理。

设置XML-RPC客户

XML-RPC客户的请求应包含如下元素：

RPC端点： http://blogsearch.google.com/ping/RPC2
调用方法名： weblogUpdates.extendedPing
参数： (应按照如下所列的相同顺序传送)

站点名

站点URL

需要检查更新的页面URL

相应的RSS、RDF或Atom种子的URL

可选页面内容的分类名称(或标签)。您可以指定多个值，之间用'|'字符进行分隔。

XML-RPC响应中将返回含有两个元素的<struct> ：

flerror (Boolean)：当有错误发生时设为true/1

message (string) ："Thanks for the ping." (如果成功) 或者一个错误信息(如果不成功)。

XML-RPC请求的例子

POST /RPC2 HTTP/1.0
User-Agent: request
Host: blogsearch.google.com
Content-Type: text/xml
Content-length: 447

<?xml version="1.0"?>
<methodCall>
  <methodName>weblogUpdates.extendedPing</methodName>
  <params>
    <param>
      <value>Official Google Blog</value>
    </param>
    <param>
      <value>http://googleblog.blogspot.com/</value>
    </param>
    <param>
      <value>http://googleblog.blogspot.com/</value>
    </param>
    <param>
      <value>http://googleblog.blogspot.com/atom.xml</value>
    </param>
  </params>
</methodCall>

XML-RPC响应的例子

HTTP/1.1 200 OK
Connection: close
Content-Length: 451
Content-Type: text/xml
Date: Sun, 30 Sep 2001 20:02:30 GMT
Server: Apache

<?xml version="1.0"?>
<methodResponse>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>flerror</name>
            <value>
              <boolean>0</boolean>
            </value>
          </member>
          <member>
            <name>message</name>
              <value>Thanks for the ping.</value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodResponse>

设置REST客户

REST客户的请求中包含如下元素：

URL: http://blogsearch.google.com/ping

参数:

name = 博客的名字

url = 博客的URL

changesURL = RSS、RDF或Atom种子的URL (可选)

响应中将返回文本内容，在成功的时候是"Thanks for the ping."，在失败时是一条错误消息。

例子：

http://blogsearch.google.com/ping?name=Official+Google+Blog&url=http%3A%2F%2Fgoogleblog.blogspot.com%2F&changesURL=http%3A%2F%2Fgoogleblog.blogspot.com%2Fatom.xml

changes.xml的格式

Google 将所接受的“博客搜索” Ping 记录以XML格式发布到http://blogsearch.google.com/changes.xml。changes.xml文件格式包含一个根元素<weblogUpdates>和一个或多个的<weblog>元素。

<weblogUpdates version="2" updated="Wed, 30 May 2006 14:10:00 GMT" count="1384779">
  <weblog name="Some Blog"
          url="http://googleblog.blogspot.com" 
          rssUrl="http://googleblog.blogspot.com/atom.xml"
          when="1"/>
  ...
</weblogUpdates>

<weblogUpdates>元素

version：用来指示一个主要的格式变化，目前的版本是2。

updated：以 HTTP 1.0格式表示的日期时间。它指示了该文件上一次更新的时间。

count：一个递增的数字，表示了changes.xml文件的版本。新版本changes.xml文件通常具有更大的count值。

<weblog>元素

name：Blog的名称。
url： Blog的url。
rssUrl：相应的RSS、Atom或其它种子的URL。
when： <weblogUpdates>元素的updated属性对应的时间减去相应的 Ping 发生的时间的差值，单位为秒。

====================================================================

下面实现如何使用XML RPC编程实现Google Ping服务.

首先,下载CookComputing.XmlRpc

我使用的是2.0版本.最新版为2.1.0

1.建立xml rpc客户端

public class GoogleXmlRpc
    {
        /// <summary>
        /// 返回值,用于处理Google Ping返回值的结构
        /// </summary>
        public struct ReturnValue
        {
            public bool flerror;
            public string message;
        }

        [XmlRpcUrl("http://blogsearch.google.com/ping/RPC2")]
        public interface IGooglePing : IXmlRpcProxy
        {
            [XmlRpcMethod("weblogUpdates.extendedPing")]
            ReturnValue Ping(string websiteName, string websiteUrl, string changedUrl, string feedUrl);
        }

    }

2.向Google发出一个ping通告

在你的程序添加文章或更新文章时,发出ping通告.

        string websiteName="venjiang 的博客";                                                 // 博客名称
        string websiteUrl="http://blog.hidotnet.com/venjiang/";      // 博客网站地址
        string changeUrl = "http://blog.hidotnet.com/venjiang/";                  // 要更新的地址
        string feedUrl = "http://blog.hidotnet.com/venjiang/Rss.aspx";      // 聚合地址rss或atom

GoogleXmlRpc.IGooglePing proxy = (GoogleXmlRpc.IGooglePing)XmlRpcProxyGen.Create(typeof(GoogleXmlRpc.IGooglePing)); // 创建代理类

// 最新版本,支持泛型,可以直接使用:

// GoogleXmlRpc.IGooglePing proxy =XmlRpcProxyGen.Create<GoogleXmlRpc.IGooglePing>();

GoogleXmlRpc.ReturnValue returnValue = proxy.Ping(websiteName, websiteUrl, changeUrl, feedUrl); // ping

        if (returnValue.flerror)
            Response.Write( "Google Ping 出错!");
        else
            Response.Write "Google Ping 成功!";

3.查看google 更新列表

下载:http://blogsearch.google.com/changes.xml 你可以看到你的博客相关信息已加入更新列表,目前google更新列表,有一定延迟.

安装Visual Studio 2005 Service Pack 1

venjiang — Mon, 18 Dec 2006 03:02:50 GMT

前几日jacky告诉我vs2k5 sp1出了,随即下载.今日抽空赶紧安装.

准备环境:
Microsoft? Visual Studio? 2005 Service Pack 1

下面为中文版
下载地址:
Microsoft? Visual Studio? 2005 Service Pack 1
下载地址:
http://www.microsoft.com/downloads/details.aspx?FamilyID=bb4a75ab-e2d4-4c96-b39d-37baf6b5b1dc&DisplayLang=zh-cn

安装要求：

目标计算机上必须安装了受支持的 Visual Studio® 2005 产品副本。
目标计算机上安装了 Microsoft® Windows® Installer 3.1 或更高版本。
至少 192MB 的 RAM。建议使用 256MB 或更大内存。
硬盘空间至少为 6.2GB
- 如果目标计算机上安装了多个 Visual Studio® 2005 产品，硬盘空间要求也会相应大大增加。

安装可能出现问题解决办法:
http://support.microsoft.com/kb/925336

几个朋友提示安装过程费了些周折,为避免安装出现问题,请按上面的方法,先修改本地安全策略.
M$这个要求有6G空间,这个实在有些愁人,一些朋友也因空间问题安装出错,抓紧整理我的硬盘空间,但资料实在太多,一时半日无法整理出6G空间.一个补丁400M,干嘛要求这么多空间,M$不是在折腾我们嘛. 决定尝试安装.

安装前环境:
剩余空间:
C: 3.88G
D: 3.57G

Visual Studio 2005 安装在D盘,临时文件夹在C盘,如果你有修改系统临时文件夹路径,确保所在盘有足够空间.
Visual Studio 2005我是重新安装的,没有安装VB,VC++,其余都有安装.上面所列硬盘空间容量为安装VS2005后剩余.

安装过程中:
剩余空间:
C: 2.54G
临时文件写入,然后开始收集信息,过程比较漫长.这个时间你可以去看电视了.或者干脆小睡一会儿,不知道M$花这么长时间收集什么,小心把电脑里的MM搜集走. 在这个过程中,临时文件夹写入450多M,C盘其它地方写入不到1G, 任务管理器查看,整个SP1补丁被全部载入内存,安装程序占用CPU 50%左右.为避免出现什么异外,建议不要开太多程序.

搜集完信息,开始安装....

这个时间不准确,可以去抽颗烟,喝杯咖啡.

安装完成后:
剩余空间:
C: 2.98G
D: 3.56G

VS2005版本

总结:
整个过程安装比较顺利,可以看到在搜集信息时,C盘写入1.34G,安装完成后,删除了临时文件,减少450M,D盘,也就是VS2005所在文件夹写入信息并不多.这样看来,如果只安装C#版本的VS,打补丁,C盘有1.5G应该足够了,VS2005安装所在目录并不需要太多空间.当然,如果你的VS2005安装的组件比较多的话,相应空间要求会提高不少.但至少6G这个要求并不是必须的.

测试:
终于可以在中文版上运行Web Application Project了.
新建:

项目解决方案:

编译:

运行:

另.打开先前M$提供的Web Application Project 项目,可以成功加载.

至此,Visual Studio? 2005 Service Pack 1安装完毕.

测试

venjiang — Wed, 13 Dec 2006 02:40:06 GMT

BSP防范Trackback垃圾的一种方法

发布Asp.Net Forums V2.2.1929 官方中文Beta版

测试trackback

BlogML 2.0 is Released

venjiang — Wed, 25 Oct 2006 11:15:04 GMT

BlogML是什么?
BlogML是blog内容的XML格式存储,你能够使用它归档blog内容,或者作为博客转换/迁移的标准格式.当前已有数个blog引擎提供了这个支持.

Community Server
Das Blog
Subtext
SingleUserBlog

你能够在这里下载 BlogML 2.0.

SqlServer数据访问帮助类 for .Net 2.0

venjiang — Fri, 13 Oct 2006 23:06:54 GMT

=================================================================
// SqlServer数据访问帮助类
=================================================================
// 名字空间: hiDotNet.Data
// 描    述: SqlServer数据访问帮助类.
// 日    期: 2006年3月14日
// 修订日期: 2006年3月17日
// 联    系: venjiang@msn.com
// 说    明: 实现SqlServer的快速访问
=================================================================
// 版权所有,hiDotNet.com 更多内容请访问: www.hiDotNet.com
=================================================================

[示例运行环境]
.Net Framework 2.0
Sql Server 2000

[步骤]
1.执行SetupDataBase.bat,安装示例数据库所需内容.
2.使用Vs2005打开SqlHelperTest.csproj,编译运行.

欢迎测试反馈.
示例截图:

帮助

点击下载

关于Cache的疑惑

venjiang — Wed, 11 Oct 2006 16:35:33 GMT

Cache.Insert("CacheKey", value, null, DateTime.Now.AddSeconds(30), TimeSpan.Zero, CacheItemPriority.Normal, null);

Cache.Insert("CacheKey", value, null, DateTime.Now.AddSeconds(30), System.Web.Caching.Cache.NoSlidingExpiration, CacheItemPriority.Normal, null);

前者很少有命中.
后者才是按时间命中.

System.Web.Caching.Cache.NoSlidingExpiration实际也是设置为TimeSpan.Zero.
MSDN如是说:
使用后，该字段将 slidingExpiration 参数设置成 TimeSpan.Zero 字段，该字段具有常数值零。缓存项依照与 Insert 或 Add 方法调用相关的 absoluteExpiration 参数设置过期。

无法同时设置 absoluteExpiration 和 slidingExpiration 参数。如果要让缓存项在特定时间过期，可将 absoluteExpiration 参数设置为特定时间，并将 slidingExpiration 参数设置为 NoSlidingExpiration。

如果要让缓存项自最后一次访问该项后的某段时间之后过期，可将 slidingExpiration 参数设置为过期间隔，并将 absoluteExpiration 参数设置为 NoAbsoluteExpiration。

MSDN:示例
Cache.Insert("DSN", connectionString, null, DateTime.Now.AddMinutes(2), TimeSpan.Zero, CacheItemPriority.High, onRemove);

而这种方法,用户访问后,立即过期,缓存实际上根本无法命中.
MSDN上的示例都是如此.

?????

hiDotNet博客开通

venjiang — Wed, 11 Oct 2006 16:26:22 GMT

测试期...

目标,将建立成高负载的博客系统,分布式缓存策略等.

开篇纪念.

Stored procedures: Implement full text search

venjiang — Wed, 11 Oct 2006 15:36:15 GMT

Enabling full text search in T-SQL is not as popular as doing it with the Enterprise Manager. Nonetheless, it can be useful in certain situations. Here are the steps to implement FTS in T-SQL.

Enable full text on the database by executing the following SP:
```
EXEC sp_fulltext_database  'enable' 
```

Create the catalog (if it does not exist):

EXEC sp_fulltext_catalog   'MyCatalog','create'

Add a full text index on a table:

EXEC sp_fulltext_table     'Products', 'create', 'MyCatalog', 'pk_products'
EXEC sp_fulltext_table     'Categories', 'create', 'MyCatalog', 'pk_categories'

Add a column to the full text index:

EXEC sp_fulltext_column    'Products', 'ProductName', 'add' 
EXEC sp_fulltext_column    'Categories', 'Description', 'add'

Activate the index:

EXEC sp_fulltext_table     'Products','activate'
EXEC sp_fulltext_table     'Categories','activate'

Start full population:

EXEC sp_fulltext_catalog   'MyCatalog', 'start_full'

Here are some examples on how to use this procedure:

USE Northwind
GO
SELECT ProductId, ProductName, UnitPrice
FROM Products
WHERE CONTAINS(  
                    ProductName, ' "sasquatch " OR "stout" '
                                )
GO

USE Northwind
GO
SELECT CategoryName
FROM Categories
FREETEXT (
                    Description, 'sweetest candy bread and dry meat'
                   )
GO

Custom Paging in ASP.NET 2.0 with SQL Server 2005

venjiang — Wed, 11 Oct 2006 15:35:08 GMT

This article highlights features new to SQL Server 2005 that make retrieving a particular paged subset of data incredibly easy and efficient. For those who've yet to switch to SQL Server 2005, check out A More Efficient Method for Paging Through Large Result Sets. The stored procedure presented in that article can be plugged in for the stored procedure examined in this article if needed.

Introduction
A common pattern in web development is providing paged access to data. Rather than displaying the entire contents of a report or database table to an end user, developers often show only a subset of records per web page, with controls for moving from page to page. With ASP.NET 1.x, the DataGrid made paging incredibly simple - just set the AllowPaging property to True and add a few lines of code in the PageIndexChanged event handler and you were done! ASP.NET 2.0's GridView makes the process even simpler - just check the Enable Paging option from the GridView's smart tag - no code needed.

Of course nothing is free in life, and the tradeoff you make with the ease of checking a checkbox to enable paging (or, in the DataGrid's case, writing a couple lines of code) is performance. Out of the box, the DataGrid and GridView use default paging, which is a simple paging model that returns all of the records for each every page of data shown. When paging through small amounts of data (dozens to a hundred or so records), this inefficiency is likely outweighed by the ease of adding the feature. However, if you want to page through thousands, tens of thousands, or hundreds of thousands of records the default paging model is not viable.

The alternative to default paging is custom paging, in which you are tasked with writing code that intelligently grabs the correct subset of data. It requires a bit more work, but is essential when dealing with sufficiently-sized data. I discuss how to implement custom paging in ASP.NET 1.x in my book ASP.NET Data Web Controls Kick Start. In this article we'll look at how to implement custom paging in ASP.NET 2.0 using SQL Server 2005's new ROW_NUMBER() feature. (For more information on SQL Server's new ranking features, including ROW_NUMBER(), see Returning Ranked Results with Microsoft SQL Server 2005.)

Read on to learn more!

Default Paging vs. Custom Paging
The GridView in 2.0 (and the DataGrid in 1.x) offers two paging models: default paging and custom paging. The two models provide a tradeoff between performance and ease of setting up/configuring/using. The SqlDataSource control uses default paging (although you can wrestle it into using custom paging); the ObjectDataSource uses default paging by default, but has an easy mechanism to indicate that it should use custom paging. Keep in mind that the GridView merely displays data; it's the GridView's data source control that is actually retrieving data from the database.

With default paging, each time a new page of data in displayed in the GridView, all of the data is requeried from the and returned from the GridView's data source. Once all of the data has been returned, the GridView selectively displays part of the entire set of data, based on the page of data the user is viewing and how many records per page are displayed. The key thing to understand here is that every single time a page of data is loaded - be it on the first page visit when viewing the first page of data or when the user postsbacks after requesting to view a different page of data - the entire data result is retrieved.

For example, imagine that you work at an eCommerce company and you want to allow the user to page through a list of the 150 products your company sells. Specifically, you want to display 10 records per page. Now, when a user visits the web page, all 150 records will be returned by the data source control, but the GridView will display the first 10 products (products 1 to 10). Next, imagine that the user navigates to the next page of data. This will cause a postback, at which point the GridView will rerequest all 150 records from the data source control, but this time only display the second set of 10 (products 11 to 20).

Caching and the SqlDataSource

Caching and the SqlDataSource
The SqlDataSource allows for the DataSet it returns to be cached by simply setting the `EnableCaching` property. With a cached DataSet, stepping to another page does not require the database be requiried since the data being paged through is cached in memory. However, on the initial page load the same problem arises - all of the data must be loaded into the cached DataSet. Furthermore, you must worry about stale data with this approach (although if you use SQL cache dependencies, then this point is moot). Even with caching the DataSet, my unscientific tests found custom paging to be twice as fast... When we examine the performance metrics later, though, you'll see that this cached approach far outshines the non-cached approach. (But it still doesn't beat the custom paging approach!) For more on caching the DataSet returned by the SqlDataSource see Caching Data With the SqlDataSource.

The SqlDataSource allows for the DataSet it returns to be cached by simply setting the EnableCaching property. With a cached DataSet, stepping to another page does not require the database be requiried since the data being paged through is cached in memory. However, on the initial page load the same problem arises - all of the data must be loaded into the cached DataSet. Furthermore, you must worry about stale data with this approach (although if you use SQL cache dependencies, then this point is moot).

Even with caching the DataSet, my unscientific tests found custom paging to be twice as fast... When we examine the performance metrics later, though, you'll see that this cached approach far outshines the non-cached approach. (But it still doesn't beat the custom paging approach!)

For more on caching the DataSet returned by the SqlDataSource see Caching Data With the SqlDataSource.

With custom paging, you, the developer, have to do a bit more work. Rather than just being able to blindly bind the GridView to a data source control and check the "Enable Paging" checkbox, you have to configure the data source control to selectively retrieve only those records that should be shown for the particular page. The benefit of this is that when displaying the first page of data, you can use a SQL statement that only retrieves products 1 through 10, rather than all 150 records. However, your SQL statement has to be "clever" enough to be able to know how to just snip out the right subset of records from the 150.

The Performance Edge of Custom Paging
Realize that custom paging provides better performance than default paging because only those database records that need to be displayed are retrieved. In our products example, we assumed there were 150 products, showing 10 per page. With custom paging, if the user stepped through all 15 pages of data, precisely 150 records would have been queried from the database. With default paging, however, for each page of data, 150 records would have been accessed, leading to a total number of retrieved records of 15 times 150, or 2,250! While custom paging exhibits better performance, default paging is much easier to use. Therefore, I would encourage you to use default paging if the data you are paging through is relatively small and/or the database server is not heavily trafficked. If you have several hundred, thousands, or tens of thousands of records you are paging through, by all means use custom paging. However, for paging through something like the ASPFAQs.com database, which only has, currently, ~200 FAQs, default paging is sufficient. (Of course, if you use default paging on a small table with, say, 75 records, you are assuming that over time the table's row count will stay low. There will be some unhappy customers if you use default paging on that small table which later grows to be a table with 7,500 records!)

The Performance Edge of Custom Paging

Realize that custom paging provides better performance than default paging because only those database records that need to be displayed are retrieved. In our products example, we assumed there were 150 products, showing 10 per page. With custom paging, if the user stepped through all 15 pages of data, precisely 150 records would have been queried from the database. With default paging, however, for each page of data, 150 records would have been accessed, leading to a total number of retrieved records of 15 times 150, or 2,250!

While custom paging exhibits better performance, default paging is much easier to use. Therefore, I would encourage you to use default paging if the data you are paging through is relatively small and/or the database server is not heavily trafficked. If you have several hundred, thousands, or tens of thousands of records you are paging through, by all means use custom paging. However, for paging through something like the ASPFAQs.com database, which only has, currently, ~200 FAQs, default paging is sufficient.

(Of course, if you use default paging on a small table with, say, 75 records, you are assuming that over time the table's row count will stay low. There will be some unhappy customers if you use default paging on that small table which later grows to be a table with 7,500 records!)

Efficiently Getting Back a Page of Data with SQL Server 2005
As discussed in an earlier 4Guys article, Returning Ranked Results with Microsoft SQL Server 2005, SQL Server 2005 introduces a number of new keywords for returning ranked results. In particular, the ROW_NUMBER() keyword enables us to associate a sequentially-increasing row number for the results returned. We can use ROW_NUMBER(), then, to get a particular page of data using a query like the following:

SELECT ...
FROM
   (SELECT ... 
         ROW_NUMBER() OVER(ORDER BY ColumnName) as RowNum
    FROM Employees e
   ) as DerivedTableName
WHERE RowNum BETWEEN @startRowIndex AND (@startRowIndex + @maximumRows) - 1

Here @startRowIndex is the index of the row to start from and @maximumRows is the maximum number of records to show per page. This query returns the subset of records whose ROW_NUMBER() is between the starting index and the starting index plus the page size.

To help concretize this concept, let's look at the following example. Imagine that we have an Employees table with 5,000 records (business is good!). The following query:

SELECT RowNum, EmployeeID, LastName, FirstName
FROM
   (SELECT EmployeeID, LastName, FirstName 
       ROW_NUMBER() OVER(ORDER BY EmployeeID) as RowNum
    FROM Employees e
   ) as EmployeeInfo

Would return results like:

RowNum	EmployeeID	LastName	FirstName
1	1000	Smith	Frank
2	1001	Jackson	Lucy
3	1011	Lee	Sam
4	1012	Mitchell	Jisun
5	1013	Yates	Scott
6	1016	Props	Kathryn
...
5000	6141	Jordan	DJ

Notice that even though the EmployeeID fields may have gaps and may not start at 1, the ROW_NUMBER() value starts at 1 for the first record and steadily increases. Therefore, if we want to view 10 records per page, and we want to see the third page, we know that we want records 31-40, and can accomplish that in a simple WHERE clause.

Configuring the ObjectDataSource for Custom Paging
As aforementioned, the SqlDataSource isn't designed to provide custom sorting capabilities. The ObjectDataSource, on the other hand, was designed to support this scenario. The ObjectDataSource is a data source control that's designed to access data from an object. The object can retrieve its data however it likes, be it from a Web Service, a database, the file system, an XML file... whatever. The ObjectDataSource doesn't care, it simply acts as a proxy between the data Web control that wants to consume the data (such as a GridView control) and the underlying data that the object provides. (For more information on the ObjectDataSource see the ObjectDataSource Control Overview.)

When binding a data Web control to an ObjectDataSource the "Enable Paging" option is available. If you've not specifically set up the ObjectDataSource to support custom paging, the paging provided will be of the default paging flavor. To setup custom paging with the ObjectDataSource you need to be using an object that provides the following functionality:

A method that takes in as its final two input parameters two integer values. The first integer value specifies the starting index from which to retrieve the data (it's zero-based), while the second integer value indicates the maximum number of records to retrieve per page. This method needs to return the precise subset of data being requested, namely the data starting at the specified index and not exceeding the total number of records indicated.
A method that returns an integer value specifying the total number of records that are being paged through. (This information is used by the data Web control when rendering the paging controls, since it needs to know how many total pages of data there are when showing page numbers or when deciding whether to enable the Next link.)

If you are using an underlying object that provides these features, configuring the ObjectDataSource to support custom paging is a breeze. Just set the following ObjectDataSource properties:

Set EnablePaging to True
Set SelectMethod to the method that accepts the starting index and maximum number of rows input parameters
Set the StartRowIndexParameterName to the name of the integer input parameter in your SelectMethod that accepts the starting index; if you do not provide this value it defaults to startRowIndex
Set the MaximumRowsParameterName to the name of the integer input parameter in your SelectMethod that accepts the maximum number of rows to return; if you do not provide this value it defaults to maximumRows
Set SelectCountMethod to the method that returns the total number of records being paged through

That's it. Once you've done the above, the ObjectDataSource will be using the custom paging functionality. Of course, the hard part of this all is creating the underlying object that can intelligently grab the right subset of data. But once you have that object, configuring the ObjectDataSource to utilize custom paging is just a matter of setting a few properties.

Creating an Object That Supports Custom Paging
In order to bind an ObjectDataSource to a GridView we need to first have an underlying object that the ObjectDataSource will use, and this object must have methods for accessing a particular subset of the data and returning the number of rows to be paged through. As discussed in Joseph Chancellor's article, Using Strongly-Typed Data Access in Visual Studio 2005 and ASP.NET 2.0 and Brian Noyes's article Build a Data Access Layer with the Visual Studio 2005 DataSet Designer, creating objects that can be bound to the ObjectDataSource is a breeze in Visual Studio 2005. The first step is to define the stored procedures (or SQL queries) that will be used to populate the strongly-typed DataSets returned by these object's methods.

The download, available at the end of this article, has a sample database with 50,000 employee records (plus an easy way to add additional records in bulk). The database includes three stored procedures that are used by the two custom paging demos:

GetEmployeesSubset(@startRowIndex int, @maximumRows int) - returns at most @maximumRows records from the Employees table starting at @startRowIndex when ordered by EmployeeID.
GetEmployeesRowCount - returns the total number of records in the Employees table.
GetEmployeesSubsetSorted(@sortExpression nvarchar(50), @startRowIndex int, @maximumRows int) - this sproc returns a page of data sorted by a specified sort expression. This allows the a page of data ordered by, say, Salary, to be returned. (GetEmployeesSubset returns records always ordered by EmployeeID.) This flexibility is needed if you want to create a sortable GridView that employs custom paging.

We won't be discussing implementing the sortable, custom pageable GridView in this article, although examples are included in this article's download; see Sorting Custom Paged Results for a look at how to create a custom paging and bi-directional sortable UI...

Once these stored procedures are created, I created the underlying object by adding a Typed DataSet to my project (Employees.xsd). I then added three methods, one against each of the stored procedures listed above. I ended up with an EmployeesTableAdapter object with methods GetEmployeesSubset(startRowIndex, maximumRows) and GetEmployeesRowCount() that can then be plugged into the ObjectDataSource's properties. (For step-by-step instructions on creating the Typed DataSet, see Using Strongly-Typed Data Access in Visual Studio 2005 and ASP.NET 2.0 and Scott Guthrie's blog entry Building a DAL using Strongly Typed TableAdapters and DataTables in VS 2005 and ASP.NET 2.0.)

Comparing the Performance of Default Paging and Custom Paging
To compare the performance between default and custom paging against the database included in this article's download (which has a table with 50,000 records), I used both SQL Profile and ASP.NET tracing to ascertain relative performance differences. (These techniques were done very unscientifically on my computer, which had other processes running in the background and such. While the results can hardly be called conclusive, I think the performance differences between the two methods clearly highlights custom paging's advantages.)

SQL Profiler Results

Default Paging
(Selecting All Records from `Employees`)
Duration (sec)	Reads
1.455	383
1.405	383
1.434	383
1.394	383
1.365	383
Avg: 1.411	Avg: 383

Custom Paging
(Selecting a Page of Records from `Employees`)
Duration (sec)	Reads
0.003	29
0.000	29
0.000	29
0.003	29
0.003	29
Avg: 0.002	Avg: 29

ASP.NET Trace Results

Default Paging
(Selecting All Records from `Employees`)
Page Load Duration (sec)
2.34136852588807
2.35772228034569
2.43368277253115
2.43237562315881
2.33167064529151
Avg: 2.379363969

Custom Paging
(Selecting a Page of Records from `Employees`)
Page Load Duration (sec)
0.0259611207569677
0.0280046765720224
0.0359054013848129
0.0295534767686955
0.0300096800012292
Avg: 0.029886871

Cached SqlDataSource
(Selecting All Records, But Caching Them)
Page Load Duration (sec)
2.39666633608461
0.0431529705591074
0.0443528437273452
0.0442313199023898
0.0491523364002967
Avg: 0.515511161

As you can see, the custom paging is roughly two order of magnitudes faster than the default paging. At the database level, the GetEmployeesSubset(@startRowIndex int, @maximumRows int) stored procedure is about 470 times faster than the simple SELECT statement that returns all records from the Employees table. Custom paging is about 120 times faster than default paging at the ASP.NET level. The reduction is performance gain is probably due to expensive workloads common to both approaches, namely setting up the database connection and issuing the command. Regardless, two orders of magnitude is a very big difference in the world of performance. And this disparity would be more pronounced with larger data sets or a server that was experiencing any kind of load.

The cached SqlDataSource has a high cost when the cache is empty, as it must go to the database and get all of the records. The frequency that the cache needs to be reloaded depends upon free resources on the web server (if you have low resources available, the cached DataSet may get evicted) and your cache expiration policy. After the data has been cached, though, it greatly improves in performance and is comparable to the custom paging approach. The 0.516 second average time would be amortized to closer to 0.05 seconds as more requests were served with the cached data.

Conclusion
As with the DataGrid in ASP.NET 1.x, the GridView in 2.0 offers two flavors of paging: default and custom. Default paging is easier to setup, but involves requerying the database when viewing each and every page of data. Custom paging, however, more intelligently just grabs those records needing to be displayed and therefore affords much higher degree of performance. SQL Server 2005 simplifies obtaining the precise subset of records for an arbitrary page due to its ability to rank results, which includes the ROW_NUMBER() feature.

If you are building web applications that need to scale or either now or in the future will allow users to page through potentially large data sets, it behooves you to implement custom paging.

Happy Programming!

By Scott Mitchell

Paging through lots of data efficiently (and in an Ajax way) with ASP.NET 2.0

venjiang — Wed, 11 Oct 2006 15:34:05 GMT

David Hayden blogged about a cool new ROW_NUMBER() function that SQL 2005 provides that got me excited and inspired to write a little code last night.

You can download the sample app I built here.

Features of the Sample

This sample is a self-contained ASP.NET application that demonstrates a few things:

- How to implement a data-result search page built with ASP.NET 2.0 and Atlas that can present hundreds of thousands of row results using the ASP.NET 2.0 GridView control. The results are formatted using a “paging” based UI model – where 15 results per page are displayed, and the user can skip from page to page to see their data. For kicks I also added support for editing and deleting each individual row.

- How to optimize the number of rows returned by the SQL Server to the middle-tier web-server. This sample demonstrates how a large query result (think 1000s, 10s of thousands or 100s of thousands of rows) can be paged efficiently so that only the 15 rows currently on display in a page are ever retrieved by the web-server from the SQL database (this is done using the ROW_NUMBER() function that David describes above, as well as the support for optimized paging provided by the GridView and ObjectDataSource controls). This avoids your web-server and SQL server grinding to a halt when you execute large queries, and makes for much more scalable performance.

- How to easily implement Ajax UI support for paging/editing/deleting on top of hundreds of thousands of rows of data (so no full page refreshes – instead it only updates the portion of the page that changes). This took me only 60 seconds to-do, and uses the same <atlas:updatepanel> control support I talked about in this earlier blog post.

- How to easily implement Ajax UI support for adding “auto-suggest” behavior to controls like text-boxes. The December release of the Atlas Project provides a super-easy server control called the <atlas:autocompleteextender> control that you can point at a TextBox, and that will then call a web-service to provide a list of suggestions when a browser user starts typing in the text-box. This sample demonstrates how to use this to auto-suggest items based on the contents in the database.

- How to implement a business class façade/wrapper around a data access layer. This is in turn used by the new ASP.NET 2.0 ObjectDataSource control for databinding scenarios. For this simple sample, I could have just used the data access layer built by using File->Add New Item->DataSet from the ObjectDataSource control directly (like I did with my earlier To-do List sample), and basically eliminate the need to write any code in the app. But I wanted to use this sample to help demonstrate how to build a richer business library layer abstraction that was separate from my data access layer. The business layer implementation in this sample is pretty trivial (and doesn’t really add much value), but it demonstrates a skeleton of how/where you could easily add business logic rules that were cleanly separated from your data layer.

How To Run and Use the Application

To setup the sample, follow the below steps:

1) Have VS 2005 or Visual Web Developer and SQL Express installed. Visual Web Developer and SQL Express are both free downloads and provide everything you need to build, debug and run the application. You can download them from here.

2) Download this .zip file containing the sample. Expand it into any directory you want.

3) Using Visual Web Developer or VS 2005, choose File->Open Web-site and point to the root directory of the sample sub-directory (the one with the files in it).

4) Select the “PagedData.aspx” file and hit run.

This will bring up a page that looks like this:

This allows you to query for lists by categories in your database. The default database in the sample doesn’t have any list items, so first we’ll want to create some. To-do this click the “Bulk Add” link.

Create a category called “test” and add 100 items to it. This will create a bunch of sample data in the database that will help simulate paging. Then repeat this with a new category called “test2” and add 10,000 items to it. Then repeat this with a new category called “test3” and add 100,000 items to it. Note that this last category will probably take a few minutes to create (since the logic in my app is simple and just adds them one row at a time).

Then click on the “Return to Paged Data Page” link and start typing a search string in the textbox. Pause slightly after you type “tes” – and then notice how the auto-suggest box will list the three new categories you added:

These are being populated using the <atlas:autocompleteextender> server control (which is hitting a web-service on the server to lookup the available categories – and so will return the three new ones we just created).

Search using “test3” (the one with 100,000 items), and you’ll see this paged view of 100,000 items in the GridView:

Each page of the screen contains 15 items from the database. You can switch around to see the different pages. Click the “…” link to go to pages 11-20. Click “Last” to jump to the final page of data:

Note that as you choose the different pages, the only thing that gets updated is the grid of data – the overall page does not refresh. This is because the GridView control is wrapped using the <atlas:updatepanel> control which allows incremental Ajax refreshes of the page (no code required to accomplish this <g>).

Page back a few screens, and then click “edit” on one of the rows. Note that you can now edit and update its items. You can also click “delete” on a row to remove one. Both operations occur in an Ajax way:

How is this Application Built

Here is what the solution directory looks like in Visual Web Developer:

It contains two pages – PagedData.aspx and AddData.aspx – that are each based on the Site.master master-page. It also contains one web-service – categories.asmx – which is used by the <atlas:autocompleteextender> control.

The database is implemented in PagingSample.mdf and contains 1 simple table called “Lists” (note: you can create new SQL Express databases by select File->Add New Item->Database File):

ListId is the primary key, and the Category column has been marked to be indexed (note: this is important since we’ll be creating 100,000+ rows). To set indexes, right click on a column in the table designer and choose “Indexs/Keys” and make sure that “Categories” is being indexed.

The database has one stored procedure called “GetPagedLists” that was created using the ROW_NUMBER() approach described in David’s blog:

Note: you can double-click on the sproc name in the solution explorer to open and edit it.

The data access layer was implemented using a DataSet component (choose File->Add New Item->DataSet to create one). It is defined declaratively within the “MyDataLayer.xsd” file and contains two table-adapters:

GetPagedListData goes against the SPOC above, but all other methods are normal SQL queries (note: the GetListCountByCategory method is defined as: SELECT COUNT(*) FROM Lists where category=@category and returns a scalar integer value).

I could have just used the DAL layer directly from the UI tier, but instead chose to wrap the DAL with a business layer façade that looks like this:

The two significant methods for our paging sample are “GetListCountByCategory” and “GetListsByCategory”. Here are the simple implementations of them in our business façade layer (note: right now they just thinly wrap the data access layer and expose the row data as custom “ListItem” types – so they don’t provide much value over calling the DAL directly, but do demonstrate how you could add your own custom logic around it):

Databinding a paged GridView control to this middle-tier object is easy (and automated if you are using the WYSIWYG designer in Visual Web Developer). Just set the “AllowPaging=true” property on the GridView and set the PageSize value you want (for this sample I am using 15 rows per page). Then point the GridView at an ObjectDataSource control, which is configured to use the ListManager business façade class. It has the “AllowPaging” attribute set to true, and has the “SelectMethod” and “SelectCount” properties pointing to the “GetLisByCategory” and “GetListCountByCategory” methods above.

Now, when you run the application, the GridView binds against the ObjectDataSource which will invoke the “GetListCountByCategory” method on our “ListManager” business façade class to calculate the total number of rows to page, and then invoke the “GetListByCategory” method passing the appropriate “category” parameter along with the “startRowIndex” and “maximumRows” parameter values that correspond to the page index that the GridView is currently on.

The beauty is that because we only retrieve and return the 15 rows of data we actually need from the database as part of this operation, the result is fast and scalable. As a browser user pages back and forth on the data within the GridView across the 100,000 rows of data, all UI paging logic and update/edit semantics are handled for us.

The last little implementation note to call out is the use of the <atlas:AutoCompleteExtender> control to auto-suggest items within the Category TextBox. This is a new control provided by the Atlas December drop, and is simple to implement. Basically, you just add the control to your page and point at the TextBox you want to complete, along with a web-service URL and method to call when the browser user types three letters in the TextBox -- these are then passed to the web-service, which can provide a list of suggestions back:

The implementation of the web-service method then looks like this within this sample:

Hope this helps and proves to be a useful sample. Thanks again to David for the blog post on the new ROW_NUMBER() function in SQL 2005 that makes this much easier to implement.

Hope this helps – and happy new year!

Scott

P.S. Note that with SQL 2000 you can use the same approach as above, although the SPROC ends up being slightly more complicated.