Doug’s World

Business Data Catalog Overview

Last time I blogged about Office 12 in much detail, I covered the new XML file formats. Next I want to cover the aspect of Office 12’s suite of technologies that I personally find most exciting: The Business Data Catalog (BDC).

Why do I find the BDC so exciting? Well, some background will make that clear …

Back in the 1990s, I developed highly customized bar-code tracking systems for large corporate filerooms. These systems were based on my Gen-Bar framework, which provided a foundation for building applications that tracked entities (files, audits, transactions, whatever) that moved between various locations (shelves, people, storage boxes, other offices). My clients were organizations like Ernst & Young, KPMG Peat Marwick, and the Chicago Board of Trade — companies with millions of paper files to track. Gen-Bar helped them manage this data, from printing the color-coded labels to providing search functions to people throughout the organization.

I added lots of fancy search functions to Gen-Bar, based on the feedback we got from users. These search functions worked great on the data in Gen-Bar, but the whole Gen-Bar database was disconnected from other types of information that users might want to search. For example, we had a Gen-Bar installation in the HR department of a large company. When those people wanted to search for everything related to John Doe, they had to search Gen-Bar, then search their internal HR system, then search their intranet, and perhaps search other systems as well.

I tried a few things to get around this clumsiness, but none of the solutions were ideal. The problem is, when you have data in a relational database (or other types of databases, for that matter), you can’t do meaningful searches without understanding some of the details of how the data is structured and stored.

So it’s hard to make a single search function work well across databases from different systems. Some organizations have done it, but it’s a lot of work, with lots of custom programming, and somebody has to understand and maintain all that custom code. As more and more systems are used in a large organization, a single integrated search function becomes more and more difficult to accomplish.

The Business Data Catalog provides an elegant solution to this messy problem. The basic concepts are deceptively simple:

• a BDC “connector” — an XML file — contains information about how to retrieve data from a data source
• the data source can be any ADO.NET data source (a SQL database, for instance), or a web service
• Sharepoint’s enterprise-wide search function uses the BDC connectors to search various custom data sources in addition to searching all of the web pages and document libraries within Sharepoint

The end result is impressively simple and straightforward. For example, we have an internal implementation of the BDC here at Microsoft (where we like to dogfood our products and verb our nouns), and you can search for a customer name and find hits on web pages, in documents, and in line-of-business applications such as CRM and other types of systems. You just type in what you’re looking for, and through the BDC connectors Sharepoint can search all of our internal system and give you a consolidated set of search results.

The BDC represents an approach that’s here to stay: declarative interfaces between line-of-business applications and portal search functions. Declarative meaning XML, of course: no custom programming involved.

Also note that the interface defined by the BDC isn’t just used in search functions. Web parts and document lists in Sharepoint can use the BDC to filter content and do other creative things; think of the BDC as a true “catalog” of business data, and application developers can use that catalog to retrieve data for whatever their application needs to do with the data.

The details of building BDC connectors are beyond the scope of this post, but here are a few of the key concepts involved:

Finder method. This is a method that you define which will take some filter conditions and return a set of instances of business data entities that meet the filter. Example: a finder method that retrieves a list of the customer meeting some type of sales-history criteria.

Specific Finder method. This method returns the details of a specific instance of an entity, given an instance ID. Example: a specific finder that returns customer details, given a unique customer ID number.

IDEnumerator method. This method abstracts the concept of retrieving a list of ID values and timestamps. Example: an IDEnumerator method that returns customer IDs and when each one was last updated. Note that the IDEnumerator method is used for searching data sources that are exposed via web services, so a web-service data source needs to have services that can return ID values and services that get the details for those ID values (i.e., something to hook up to the Specific Finder method).

Profile page. This defines how an entity is displayed to the user. The default profile page is simply a list of field names and values, but it’s designed to be customized.

There is a lot more detail, of course. If you’re on the Office 12 beta for ISVs, you can get more information from the latest Sharepoint SDK. For everyone else, details of the BDC architecture will be forthcoming after the release of the publicly available Beta 2 this spring. And don’t forget to check out the Sharepoint blogs and web sites listed in the sidebar on the home page of my blog — there are lots of people listed there who will be covering the BDC in great detail as the Office 12 beta cycle moves forward.

This entry was posted on Friday, January 27th, 2006 at 12:18 pm. You can subscribe to comments on this post through its RSS feed.