I recently attended an event at Microsoft about understanding and developing with Microsoft SharePoint services. I should point out that I’m not a developer, so some of the development topics meant very little to me, but I have reproduced the bits I understand here anyway!

The need for collaborative applications

According to a 2002 report by Gartner: employees get 50%-75% of their relevant information directly from other people; more than 80% of enterprise’s digitised information reside in individual hard drives and in personal files; and individuals hold the key to the knowledge economy with most of that knowledge being lost when they leave the enterprise. Not surprisingly, Microsoft has interpreted this as a need to provide easy to use and effective collaboration tools and technologies, citing that CEOs recognise that improving knowledge worker productivity will play a major role in their business’ ability to compete.

SharePoint products and technologies

Microsoft claims that its SharePoint products and technologies strategy facilitates a consistent rich experience for users, developers, and IT staff; smart connections for people, teams, divisions or the enterprise; and will support flexible deployment – bottoms-up, centralised, or hybrid on large scale farms.

Despite Microsoft’s claims, this SharePoint products and technologies naming can be confusing as Microsoft actually has two separate products which use the SharePoint name. The basic Windows SharePoint Services (WSS) product is shipped with Windows Server 2003 whereas SharePoint Portal Server (SPS) is a chargeable product which provides extra functionality, bringing SharePoint sites together as a single portal, to make them navigable, searchable, and adding enterprise level features such as:

• Hierarchical navigation and category services with areas and topics.
• Publishing and alerts.
• Search/cross-enterprise indexing (using an okapi probabilistic ranking algorithm).
• Line of business integration and single sign on (SSO) and BizTalk integration.
• Document management (alerts, version control and check-in/out).
• Profiling system.
• Personalisation (My Sites).

WSS is actually v2.0 of the earlier SharePoint Team Services (STS) product but whereas STS v1.0 only shared the SharePoint name, WSS and SPS are now very tightly integrated. Nearly all SharePoint content is stored in a SQL Server (or MSDE) database.

Microsoft is positioning SharePoint as a platform for an integrated work environment with:

• Application integration – providing a single place where knowledge workers go to to their work (in context), improving efficiency. Because Office 2003 applications integrate with WSS, Word’s Research Pane can be used to search across the Internet, or a SharePoint-based intranet; by adding document properties to a view (using the same names as in Word), Word document properties can be displayed on a SharePoint site; another example of Office product integration is using InfoPath 2003 for electronic forms, stored and accessed via a WSS forms library, allowing the data to be queried, merged, etc.; Outlook 2003 is SharePoint aware and gives users the option of creating a document workspace, rather than sending an attachment many times over. Users don’t have to be on the latest versions of Office to benefit from SharePoint integration but each new version provides tighter integration and greater functionality: Office 2000 users can save and retrieve documents to a SharePoint repository; Office XP allows users to manage SharePoint lists; but it is Office 2003 that provides the greatest level of integration including colleague presence, instant messaging and site creation directly from Office System applications.
• Collaboration – out of the box, WSS can be used to manage document lifecycles using document workspaces; facilitate organisation of meetings with meeting workspaces and their integration with Outlook 2003; facilitate communication and build consensus with forums for discussion of team activities and surveys to gather information on team status. Through integration with Live Communications Server (LCS), SharePoint is able to show team members of sites or workspaces as being online or offline to allow instant messaging if desired. Finally, with SPS, notifications and alerts can be sent when chosen documents (or entire lists) change or are added, improving workflow.
• Personalisation – provided through SPS’s MySite functionality, a personal site where users can save their own private data and make some of that data public but one which also allows for built-in functions to allow viewing of content directed at audience groups that the user is a member of.
• Scalability – because SharePoint is based on the .NET Framework and SQL Server 2000 it will scale up for improved performance. It will also scale out to a three tier application architecture with separable components for web rendering, database, searching and indexing, allowing web servers to be added as the load increases and SQL storage to be added as the data increases. SharePoint also supports the use of storage technologies such as SANs for disaster recovery and multiple server farms working together.
• Customisation/extensibility functionality – SharePoint sites, lists and views can be customised in Internet Explorer or with FrontPage. Whilst FrontPage is not required to customise WSS, it does allow a lot more flexibility (especially when working with web parts). No coding is required for immediate use of SharePoint, but it is extensible using its ASP.NET object model which allows the creation of web parts and direct mode .ASPX pages.
• Manageability – SharePoint provides improved administration with flexible roles to control site usage, specifying quotas and removing dead sites, as well as more granular management for backup and restoration.

SharePoint architecture

SharePoint works by installing an ISAPI filter in IIS, which separates content into static content – including folder views, via distributed authoring and versioning (DAV) – and dynamic content.

The dynamic content is further separated into application pages (direct mode) and user pages (safe mode), which are generally more fluid.

A web part is an ASP.NET server control which is deployed as an assembly. Web parts cannot be run unless they are safe (in order to prevent users from adding their own web parts that may have undesirable effects).

Direct mode pages reside in the file system of the web server and are processed with the standard ASP.NET runtime processing model. Direct mode pages are available for all sites on a server computer and are placed in the _layouts directory whilst _vti_bin contains a set of web services which may be called remotely.

Safe mode pages exist within the content database and can be customised and personalised; however they can only use web parts marked as safe and can not directly use in-line scripts.

When a site is created, user is specified as the site administrator. A template is then applied to the site and either shared or individual versions of pages may be modified. Each page is divided into zones, which are used to control the areas of the page which can be customised (e.g. the left zone may be fixed).

The okapi ranking used by SPS for search results is interesting – rather than using Windows’ Indexing Service, SharePoint has it’s own crawler/indexer and then uses the okapi algorithm to rank the results, so the whole process is similar to:

1. First expand the query to think about which columns make the most sense – weigh particular terms and fields as more important at query time (avoiding re-indexing to change weightings). Also coerce a field called the best bets field and make it really important in the overall ranking.
2. Break the query terms down and get alternate term forms (stemming – ran, running etc.) doing this in the query maintains the fidelity of terms in the index, and expand the thesaurus for related terms.
3. Now do the recall phase – find all the documents that match one of the terms in the query – this will result in a large list which is then trimmed down to the documents that a user has access to, and only the documents within the scope of the query.
4. Finally, allow the probability algorithm to do the precision phase – ranking the best fit documents at the top of the list.

Web parts are installed using the SharePoint Administration Tool (stsadm.exe). This tool also allows items to be saved as a template and brought in to another site, e.g. between development, test and production sites. Additionally, a whole site can be saved into a template library.

Using the stsadm.exe tool to deploy web parts requires a .CAB file (built with Visual Studio) containing:

• WebPart assembly.
• Description of contents (manifest.xml).
• Description of web part (as a .DWP file).

Example stsadm.exe command lines are:

• stsadm –o addwpppack –filename mycab.cab
• stsadm –o deletewppack –name mycab.cab
• stsadm –o enumwppacks

For security, SharePoint can use either Active Directory (AD) or the local security database. Because of this, anonymous access is supported, as is use of built-in AD groups.

Administrators and developers should be made aware that when WSS is installed (onto a server with IIS), it installs into the default web site and makes drastic changes to the web.config file , for example, directing *.ASPX to SharePoint and setting the trust level to WSS_Minimal. Because of this, any existing applications running on the default web site will probably not work and although the changes can be undone, it is probably easiest to install WSS onto its own server (or to move other web applications to their own web site prior to installation).

Developing with SharePoint

SharePoint uses its own object model to create the user interface, so there are no hidden
APIs; however to develop on a Windows XP machine (where WSS will not be available), a few additional DLLs from WSS will be required:

• Microsoft.SharePoint.DLL
• Microsoft.SharePoint.Portal.DLL
• Microsoft.SharePoint.Portal.SingleSignOn.DLL
• Microsoft.SharePoint.Security.DLL

There is also a SharePoint products and technologies software development kit (SDK), which contains a single file (spptsdk.chm).

SharePoint supports a number of application types:

• Web parts.
• .ASPX pages.
• Console/Windows tools.
• Document library events.
• Remote client via web services.

Web parts can be used in either static or dynamic scenarios and are rendered through RenderWebPart(HtmlTextWriter). Each web part cab have custom properties, the values for which can be stored per-zone (Storage.Shared) or per-user, per-zone (Storage.Personal) and the attributes for which are used to control storage, default values and the property user interface. Web parts can make use of child controls in order to achieve their requirements and support connection interfaces to allow the parts to communicate with each other at run time (e.g. as consumer and provider for a cell, list or row). Connections can be either client side or server side but cannot be mixed.

When deploying web parts, ASP.NET needs to be able to find the web part assembly, which can either be placed it in the global assembly cache (GAC) or placed in a private bin folder for the web site. Additionally, types have to be marked as safe in web.config. In order to load the web part into SharePoint, it must be packaged as a .DWP file.

Code cannot exist inline on a page within the site and creating pages in the _layouts directory is often the best option for custom .ASPX applications on top of SharePoint as it allows the page to be accessible from any site. For example, if mypage.aspx exists in in _layouts, it is accessible from the following URLs:

• http://myweb/_layouts/myapp/mypage.aspx
• http://myweb/subweb/_layouts/myapp/mypage.aspx

.ASPX pages run using the context of the web under which they is running.

A console tool is the best option for writing code that performs operations on multiple sites (e.g. list the URL and size of each site on the farm; or process all document libraries and archive file versions more than six months old).

SharePoint supports events on document libraries with operations such as add, update, delete, check-in, check-out, etc. Events are asynchronous and call the IListEventSink managed interface, running in the context of the IIS worker process.

SharePoint also has a web services APIs for accessing content remotely (e.g. from a Microsoft Office 2003 application). The web services layer is built on top of the server object model and allows manipulation of lists, webs, views, list items, etc. Functionality is similar to the server object model, but with fewer interfaces optimised to minimise transactions. Web services are added as web references in Visual Studio.NET and available web service include:

• lists.asmx (list information).
• webs.asmx (web information).
• views.asmx (view information).
• alerts.asmx (alerts).
• admin.asmx (administering sites).
• permissions.asmx, usergroups.asmx (site permissions).
• versions.asmx (file version information).
• infoforms.asmx (form information).

It will often be necessary to send the logged on users’ credentials from the client in order to make use of web services. This may be achieved by adding the following code in the web reference object’s constructor:

public Lists() {
this.Url = http://server/_vti_bin/lists.asmx;
this.Credentials=System.Net.CredentialCache.DefaultCredentials;
}

Besides using SharePoint’s own web services, it is possible to build custom web services and place them in the _vti_bin directory.

The SharePoint Object Model
The SharePoint object model has four top-level objects:

• SPWeb (represents an individual site).
• SPSite (represents a site collection, which is a set of web sites).
• SPVirtualServer (represents a virtual server).
• SPGlobalAdmin (used for global administration settings).

In order to perform actions on data within a web, it is necessary to first get an SPWeb object (e.g. SPWeb MyWeb = SPControl.GetContextWeb(Context);)

The complete object model is grouped into lists, files (documents), security and administration:

• Lists – use these objects under the Microsoft.SharePoint namespace to view and edit data in SharePoint lists:
  • SPList (basic list object for getting to list data).
  • SPListCollection (collection of list objects).
  • SPListItem (item/row in a list).
  • SPListItemCollection (collection of list items).
  • SPView (view of a SharePoint list).
  • SPField (field/column in a list).
  • SPListTemplate (template for a list).
• Files – use these objects under the Microsoft.SharePoint namespace to access document files in SharePoint sites:
  • SPFile (file object).
  • SPFileCollection (collection of files).
  • SPFileVersion (version of a file).
  • SPFolder (folder object).
  • SPDocumentLibrary (document library object).
  • SPDocDiscussion (discussions on a file).
  • SPDocTemplate (used when creating a new file).
• Security – use these objects under the Microsoft.SharePoint namespace to edit access rights and security information:
  • SPUser (user object).
  • SPRole (site group object).
  • SPGroup (cross-site group object).
  • SPPermission (assigned permissions).
  • SPRightsEnumeration (available permissions).
• Administration – use these objects under the Microsoft.SharePoint.Administration namespace to edit server-wide administrative settings.
  • SPGlobalAdmin (top level administration object).
  • SPVirtualServer (virtual Server object).
  • SPQuota (storage/user quota limit object).
  • SPGlobalConfig (configuration options).
  • SPSiteCollection (collection of sites on a virtual server).

In terms of mapping the user interface onto the object model terminology:

• Site Collection = site.
• Site = web.
• Top-level site = rootweb.
• Subsite = subweb.

Tips and Tricks

Paul Appleby, from Microsoft UK’s Developer and Platform Group, gave the following advice for developing applications with SharePoint:

• Keep objects around. If objects are created and destroyed objects frequently, extra SQL queries will be required.
• In spite of the point above, free objects when they are no longer required, calling close or dispose on web and site objects.
• Use SPWeb Web = SPControl.GetContextWeb(Context); to get the current SPWeb object from a web part or .ASPX page.
• SPGlobalAdmin and SPSite are the only SharePoint objects created with new. All others are opened from another object.
• The URL taken by the SPSite constructor must be absolute, and must refer to the actual computer name, not the load-balanced name.
• Send the user’s credentials to the server when using web services to access data in SharePoint sites and include <sharepoint runat="server" /></sharepoint> in any .ASPX page that needs to make updates.
• To optimise performance, use foreach() to step through collections. Iterating through collections by index can result in unnecessary database calls to collections such as lists. Items are expensive. Preserve the collection rather than requesting it again.
• For best performance, use SQL Profiler to minimise the number of queries that an application makes to the database.

Links

Microsoft SharePoint products and technologies.
Microsoft SharePoint products and technologies software development kit (SDK).