I am moving away from SharePoint and consulting into serious custom software development and this is my last blog post here. I wish all the best to all of you that remain in the SharePoint world.

The SPTraceView utility is now moved to CodePlex to address:

http://sptraceview.codeplex.com/

If the project is not published yet, please check again later. It must be published before 6 May 2009.

One place this could be handy is when you provision a web application on multiple front ends using SPWebApplication.ProvisionGlobaly(). This method creates a timer job and returns immediately. However if you need the web application to be created before you continue you need to wait for the timer job(s) to finish. One way to do this programmatically is using the code above.

All list item, list and web event receivers in SharePoint with the exception of the ListItemFileConverted receiver are run from the SPRequest unmanaged class. There are two types of event receivers – synchronous (such as ItemAdding and ItemUpdating) and asynchronous (such as ItemAdded and ItemUpdated). Because the event receivers are implemented in managed code, the unmanaged SPRequest class needs a way to invoke them. This is done via the ISPEventManager COM interface which is implemented by the Microsoft.SharePoint.SPEventManager internal class.

As we can see from the interface all event receivers are passed to be executed in one call and there are two types of methods in the ISPEventManager class: EnqueueEventReceivers – used for the asynchronous ones and ExecuteEventReceivers – used for the synchronous.

So because the unmanaged code will make a call back to the managed SPEventManager class to run the event receivers, the first important question is which process will those event receivers run in. Well the answer is both synchronous and asynchronous event receivers will execute in the same process that has made the change that resulted in the event receivers being called. So for example if you are adding a new file into a document library from a web part, then the event receivers will execute in the w3wp.exe process of the application pool of the web application. If you are adding the file from a rich client (MyWinFormsApp.exe) then the event receivers, including the asynchronous ones will execute in the MyWinFormsApp.exe process.

The asynchronous event receivers may need some time to complete (could be a lot of time) and if you close your MyWinFormsApp.exe immediately after you have added the file, and while the event receivers are still running, what will happen is that they will be terminated and some of them may not run at all.

The situation with IIS is not very different unfortunately. If you do an iisreset without the /noforce switch then all currently running asynchronous event receivers will be terminated immediately. My tests showed that executing iisreset /noforce will indeed wait for an asynchronous event receiver to finish however if you are trying to save any changes to the item for example, those changes will not survive.

So this means that by doing iisreset you may be interrupting a running asynchronous (or synchronous) event receiver and corrupting your application integrity/state. When executing iisreset /noforce the event receivers will complete but they may not be able to update any of the SharePoint objects they could be trying to manipulate. This finding it actually very alarming! I wouldn’t recommend to anyone writing any event receiver that will take more than 1 second to complete as an iisreset may corrupt your application!

The asynchronous event receivers will be run in their own thread from the System.Threading.ThreadPool. And a new thread will be queued up with every call from the unmanaged SPRequest class. This means that if you add say 100 files to a document library this will result in 100 calls to EnqueueItemEventReceivers for each of the items. And this will create and en-queue 100 threads each of which will run one by one all asynchronous event receivers for the corresponding file. Now if some of those event receivers are taking some time to complete, which is a few times more than the time it takes you to add a file, what will happen is you will end up with enormous amount of running threads.

Switching between threads is a very resource intensive operation and if you have 100 threads running in your application or in IIS, this will pretty much slow down the process to a crawl. I’ve seen this happening with a custom built migration tool that moves documents from one document library to another. There were some heavy ItemAdded event receivers and after few hundreds files were moved for a minute or so, it took another 45 minutes for the event receivers to complete. The migration would have been faster if after adding each file the migration tool was waiting for the event receivers to complete because after adding all the files at once 90+% of the time the process was spending in switching between the few hundred threads and only up to 10% were the actual threads running the event receivers.

The next interesting question is what is the lifespan for the SPItemEventProperties that are passed to all item event receivers (synchronous and asynchronous). Well looking at the SPEventManager class code it turns out that the SPItemEventProperties are created once per batch of events receivers and are shared among all of them. So this means that all synchronous event receivers will share the same properties and all asynchronous event receivers will share different instance of the properties. However the unmanaged SPRequest class will keep the changes to the properties done by the synchronous event receivers and will pass them to the asynchronous event receivers.  This means that it is generally possible to store information in the SPEventProperties from some event receivers that could be passed to other event receivers executed later on.

This could only be done from an ItemAdding or ItemUpdating event receiver. You could add anything you want to the AfterProperties collection and the information you have added will be availabe as AfterProperties in the ItemAdded and ItemUpdated event receivers.

A question that have been bothering me for a while is should the web retrieved by SPItemEventProperties.OpenWeb() be disposed by the event receiver that have opened it. The answer as always lies in the implementation details of the class (in out case SPItemEventProperties) that can be retrieved using Reflector.

So the SPItemEventProperties class actually implements IDisposable and disposes the allocated site or web. The SPEventManager class makes sure to call Dispose() after the batch of event receivers have executed.

Waiting for ItemAdded to Complete Before Showing EditForm.aspx

There is an issue with document libraries and the Upload.aspx page as descibed in my previous blog entry. The problem will be encountered if you have an ItemAdded event receiver attached to the document library that updates fields of the list item/document after it has been created. Because the ItemAdded event receiver runs asynchronously it may update the list item after the EditForm.aspx page has been displayed. This would result either in the EditForm.aspx form to not show the updated fields or in an exception to be generated when you click OK.

In order to solve the problem we need some code to wait for the event receiver to complete before loading the EditForm.aspx page. One way to do this is to customize the EditForm.aspx page either on a Content Type level or a List level or just using SharePoint designer and to insert a web control that will wait for the ItemAdded event receiver to complete. We also need a special ItemAdded event receiver which the web control will be able to communicate with.

And the good news for you is that I did the hard work or creating both the base class for the event receiver and the web control. They can be donwloaded here:

SharePointInternals.SynchronousItemAdded.dll

SharePointInternals.SynchronousItemAdded – Source Code

To use them you need to create your event receiver class to inherit from the SPSynchronousReceiver class and override the ItemAddedSynchronously method:

Then you need to insert the WaitForItemAdded web control in the EditPage.aspx. The best place to do this is as a first control in the PlaceHolderMain.

Unfortunately this is the only way to integrate the WaitForItemAdded control. If we add the web control to the ListFormTemplate used by the ListFormWebPart from EditForm.aspx, then it will wait for the event receivers to complete, but at this stage the ListItem would have been already loaded in the SPContext and the EditForm.aspx will show the field values of the list item before the ItemAdded event receivers have completed. If any of them have updated the list item then pressing OK on the EditForm.aspx will actually cause an exception:

The file … has been modified by … on …

In order to get around the problem that the ListItem from the SPContext have been initialized we should put the WaitForItemAdded web control somewhere in the EditForm.aspx before the ListFormWebPart and before any other control that may be using SPContext.Current.ListItem. Accessing this property will load the list item which may happen before the asynchronous event receivers have completed.

You can also wait for the ItemAdded event receivers to complete when adding a file using code. This could be handy when doing migration of documents. In order to wait for the event receiver to complete just call SPSynchronousReceiver.WaitForItemAddedReceivers().

Of course this will only wait for those event receivers that inherit from the SPSynchronousReceiver class. If you need to do the same for another asycnhronous event receiver, this could be easily done by changing the code of the base class.

And for completeness I also want to mention that the ListItemFileConverted event receivers execution is initiated from managed code when SPFile.Convert() is called. This is the only exception and the execution of all other event receivers is initialized from unmanaged code. The ListItemFileConverted event receivers run asynchronously.

Finally to recap our findings. The execution of all event receivers with the exception of the ListItemFileConverted will be initiated from unmanaged code. This happens when the SharePoint object model calls the corresponding method from the SPRequest unmanaged object to perform the operation that will trigger the event receivers. Then the unmanaged WSS world will use callbacks on the ISPEventManager COM interface implemented by the Microsoft.SharePoint.SPEventManager class to start the event receivers execution on batches of synchronous and asynchronous event receivers. You don’t have to dispose the SPWeb returned by properties.OpenWeb() in your event receiver code. Sometimes it may be a better idea to create a fresh SPSite and SPWeb instead of using properties.OpenWeb(). It is hard to say which one is good for you as it all depends on the parallel processes that may be running and the other event receivers being executed. All asynchronous event receivers for the given object (list item, list of web) will be executed in a separate thread created on the ThreadPool. SharePoint will queue a new thread for every object. This could potentially cause the server or the application to slow down dramatically as it struggles to switch between the different threads.  The asynchonous event receivers run in the same process in which the object was changed that lead to the event receivers being executed. If you stop the application or reset IIS all running event receivers will be terminated and all scheduled event receivers will not be executed at all. If you do iisreset /noforce this will wait for the event receivers to complete, but they may not be able to update the SharePoint objects they are working with. When you upload a file to a document library there is a nasty problem that you will be redirected to the EditForm.aspx page before the ItemAdded event receivers have completed. If you are doing any changes to the item in the event receivers these changes will not show up when EditForm.aspx page loads. Even more if you press OK in the EditForm.aspx page after the event receivers have modified the item you will get an exception. There is workaround if you use the SharePointInternals.SynchronousItemAdded.dll as described in this blogpost.

The first place where those forms are defined is in the List schema xml. In SharePoint 2003 (WSS 2.0) the forms used to be defined using a complex CAML but from SharePoint 2007 (WSS 3.0) they are implemented in ASPX pages using web controls. The Form element in the List schema xml is used to define the aspx page to be used for the given List Form and by default most lists use NewForm.aspx, EditForm.aspx and DistForm.aspx pages.

If necessary the aspx pages for the List forms can be redefined per item content type. This can be achieved by including a FormUrls XmlDocument with your custom content type.

So sfter SharePoint has determined from the List schema xml and the ContentType FormUrls element which aspx page should be used it opens up that page. Each of those pages contain a ListFormWebPart which does all the job of rendering the list item. SharePoint 2007 still supports legacy forms and you can use the UseLegacyForm attribute of the Form element in the List schema xml to tell SharePoint you want to use the CALM forms. In such a case the rendering of the List Form will be done from the unmanaged SPRequest class, which will return the HTML to be rendered.

In most of the cases no legacy forms are used and then the ListFormWebPart needs to determine which Form Template has to be used with the List Form. The Form Templates provide a way to further customize the list forms without changing the aspx pages. This is what SharePoint does as it uses exactly the same NewForm.aspx page for all new forms but uses different Form Templates for a standard list and for a document library. The Form Templates also allow to customize the list forms for items in the same list but for different content types.

The Form Templates are defined as an XmlDocument per content type and all content types that inherit from the Item content type also inherit the Form Templates as defined in the Item Content Type. All forms use a Form Template called ListForm.

However not all forms in SharePoint use these Form Templates. For example they are redefined by the Document content type (and all content types that inherit from Document) and they use a Form Template called DocumentLibraryForm instead of ListForm. 

The next question is where those Form Templates come from and the answer is they usually come from the 12\TEMPLATE\CONTROLTEMPLATES\DefaultTemplates.ascx page. The form templates are simply RenderingTemplate web controls and define the sets of controls to be added to the List Form. The Form Templates are matched by their ID attribute and it is actually possible to overwrite an existing template or define your own Form Templates and deploy them in an ascx page in  12\TEMPLATE\CONTROLTEMPLATES.

The code below shows the standard ListForm templates used by all generic lists:

You can probably recognize all the controls you have seen many times in those NewForm.aspx pages. There are many “interesting” controls in that form, such as the ListFieldIterator, InitContentType, SaveButton, AttachmentUpload. A detailed information on how to customize your ListForms, include custom fields and customize the Form Templates could be found in the following MSDN article.

Issues with Document Libraries and _layouts/Upload.aspx

There is one specific issue when using a NewForm with a ListFormWebPart in a Document Library. You will be unconditionally redirected to the _layouts/Upload.aspx page so you can upload the file first. After the file is uploaded you will be redirected to the EditForm. So what does this mean:

- Well first of all the “List Form” and “Form Template” of  the NewForm are actually not used because you will be unconditionally redirected to the Upload.aspx page. Of course you have the alternative of creating a custom New Form that _does not_ contain a ListFormWebPart. However all the SharePoint Field controls require to be inside a ListFormWebPart so you won’t be able to use those. You can actually use the SharePoint Field controls without a ListFormWebPart but you will need to create a new SPContext object in the Page.OnInit() event using reflection and set the ItemContext property of the SharePoint Field web control. No need to say that this will probably not be supported by Microsoft.

- Secondly the ItemAdded and ItemAdding event receivers will be initiated from the Upload.aspx page after the file is uploaded and a blank list item has been created. However this happens before you are redirected to the Edit Form and before the user has filled in the metadata of the document. Even worse the ItemAdded event receiver is executed asynchronously and there is no guarantee when it will be run. Depending on how loaded is the server and how quick the user clicks “Check In” on the Edit Form, the ItemAdded event receiver can be actually executed after the Edit Form has been submitted! Also if you rely the ItemAdded event receiver to set some default values for the item after it has been created, which values  you expect to be displayed in the Edit Form, it is quite possible that the ItemAdded will be executed after the Edit Form has rendered the old values of the item (before they have been changed by the event receiver).

- Lastly if you press “Cancel” on the Edit Form after the file has been already uploaded the file and the list item may remain in a strange state where it is only visible to the person who has uploaded the file and is even not visible to the system account. This happens when the document library is in a “Force Check Out” mode. See more in this post.

It may come as a surprise to you (which only shows how little it has been written about this) but most of the ‘important’ things in SharePoint are actually done through unmanaged code and the SharePoint .NET object model provided by the Microsoft.SharePoint.dll is a sort of a wrapper to this unmanaged world. Understanding how your code does what you want it to do can be very helpful when troubleshooting issues, fixing bugs, doing performance tuning and writing solid code in general so I believe that every SharePoint developer should know about SPRequest.

As you are aware the gateway to doing anything in SharePoint is the SPSite object. In order to get to any other objects that reside in a site collection such as SPWeb, SPList, SPListItem, SPFile and others we first need to obtain an SPSite and there is a very good reason for that. Every SPSite instance holds a reference to the unmanaged WSS world in the means of a field member of type SPRequest which is called m_Request. The internal SPRequest class has an unmanaged reference to a COM object called SP.SPRequest and having a ClassID of BDEADEE2-C265-11D0-BCED-00A0C90AB50F which is implemented in and exposed by the OWSSVR.DLL class library.

The SP.SPRequest COM object exposes almost 400 basic operations and almost everything you do with the Microsoft.NET managed SharePoint object model that reads from or writes to the ContentDatabase (including data, fields, content types, list schemas, etc) will actually go via this unmanaged COM object. Even more the OWSSVR.DLL is actually an ISAPI extension registered in IIS and its methods can be called directly via an HTTP request to /_vti_bin/owssvr.dll. Many of the Office applications (Word, Excel, InfoPath, SharePoint Designer etc) are using HTTP calls to OWSSRV directly in order to integrate with a remote SharePoint server. It won’t be too much of an exaggeration to say that OWSSRV.DLL is the soul and body of WSS 3.0. It comes historically from SharePoint ver 1.0 from the days before Microsoft.NET when web applications were developed using technologies like ISAPI and DCOM. For more information on OWSSRV and the WSS architecture have a look at the Overview of the SharePoint Team Services Architecture in MSDN.

Let’s now have a look at the public ISPRequest interface which is implemented by the SP.SPRequest COM object and is available for .NET via the SPRequest class. The list below shows only a fraction of the ~400 actions exposed by this COM object and demonstrates what sort of operations are done through the SPRequest class. As you can see this is pretty much everything in WSS:

To see the full list of methods exposed by SPRequest you can open up this class in Reflector. In order to do this you first need to get the Microsoft.SharePoint.Library assembly from the GAC. To get a physical copy of this file you can go Start -> Run -> type ‘C:\WINDOWS\ASSEMBLY\GAC_MSIL’  -> press ‘Enter’. This is a little trick that will open up the physical store of the GAC assemblies. And remember you must do this from Start -> Run – it will not work otherwise. So once you have it opened, navigate to Microsoft.SharePoint.Library\12.0.0.0__71e9bce111e9429c and copy the file to a place you can open it with Reflector.

[Updated 23 Jan]

ISPRequest is actually a COM interface and the public SPRequestInternalClass from the Microsoft.SharePoint.Library.dll is a wrapper to it. The ISAPI extension  /_vti_bin/owssvr.dll can be used to do various things as documented in the Windows SharePoint Services URL Protocol.

Even that the SPRequestInternalClass is public it is actually undocumented according to the Microsoft SharePoint Escalation Engineer Stefan Gossner. (See his comment below and the link to his blog Documented / Undocumented API – Why Should I care?).  Stefan also points out that any SharePoint database which is modified using direct access to this undocumented API will become an unsupported state as if you would have modified the database using direct SQL queries. So be warned!

[/Updated 23 Jan]

So now that we know that when we call the SharePoint .NET object model almost everything is done via unmanaged code the next question is what is the implication of this. Well firstly this means there are unmanaged objects being created all the time, and those objects cannot be cleaned up automatically by the .NET garbage collector. This is exactly why it is so important to call Dispose() or Close() on an SPSite when we have finished using it as we need to free those unmanaged resources. If we don’t do this they will cause unmanagedmemory leaks and will severely slow down the server. We need to call Close()/Dispose() with every object that implements the IDisposable interface and from the ‘common’ SharePoint objects this are both the SPSite and SPWeb class instances.

If you try to allocate too many SPRequest objects (create too many SPSites) without releasing them you will start getting medium severity trace messages in the ULS Log similar to the one below:

“Potentially excessive number of SPRequest objects (10) currently unreleased on thread 23. Ensure that this object or its parent (such as an SPWeb or SPSite) is being properly disposed. This object will not be automatically disposed.”

There are two registry keys you can use to control the ULS trace log related to allocating and disposing SPRequest objects. Firstly the warning threshold can be controlled via the LocalSPRequestWarnCount value in the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions. You should however always try to have a minimal number of SPSite objects created at the same time and should always dispose them as soon as you’ve finished working with them. Secondly to get the stack trace of the not correctly disposed SPRequests (which is not enabled by default) you can set a value of 1 for SPRequestStackTrace  in the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings registry key.  You can monitor the ULS trace messages in real time using the very light-weight SPTraceView utility. For detailed information and best practices on disposing SharePoint objects see the following MSDN article: Best Practices: Using Disposable Windows SharePoint Services Objects

Because creating too many unmanaged objects is a serious concern for a server application, Microsoft have implemented a couple of ways for those SPRequest objects to be cached and reused internally whenever this is possible. However this caching might cause additional problems such as: Reusing a cached version when you don’t really want this or attempting to reuse a cached version of the SPRequest on a thread different from the thread it has been created on. Before looking into those potential issues lets see how the caching of SPRequests works.

For managing the SPRequests SharePoint gets the help of two internal classes – SPRequestManager and SPRequestContext.  All SPRequest objects created in the managed world of Microsoft.NET are tracked and cached per process using an SPRequestManager object which singleton instance is saved in the HttpContext for web applications or in the ThreadContext of the SPFarm for non web applications (such as OWS Timer jobs, command line application such as custom STSADM commands or rich clients). This also means that this caching is per thread. The code below shows how the SPRequestManager is created and associated with the running thread:

 Additionally to this the SharePoint Object Model may be also caching the opened SPSites when running in a web context. This caching is done with the help of the SPRequestContext class which single instance is stored in the HttpContext per server thread that processes the request.

The site instances will be cached only if they have not been created with a farm admin account (when SPSite.m_bAdministratorOperationMode is false) . Please note that runing code using SPUtility.RunWithElevatedPrivilegies() method doesn’t mean your code will have farm administrator privileges. Whether a user is a farm administrator or not is determined by the SPFarm.CurrentUserIsAdministrator() method. An account will be a  farm admin if:

- It is the identity of the application pool used by the SharePoint Central Administration web application

- The account has been added to the SharePoint Administrators site group of the Central Administration web application

- The farm is being provisioned and the account is a local administrator or the Local System account

When you are using  SPUtility.RunWithElevatedPrivilegies() this will simply revert the current user identity to the identity of the current application pool, and if this identity is not a farm administrator then your SPSite _will_ be cached in the SPRequestContext. This is done when the SPSite object is created:

The following diagram shows at a high level the relationship between the various objects working with SPRequests:

As you see one of  the classes I added on the above diagram is the SPFeatreManager class. This class can be actually called from the unmanaged world to activate features when you are provisioning a WSS site and this is when a cached SPSite may get reused. Here is what will happen every time you create a site or a web from a Web-based SharePoint application (web part, aspx page or SharePoint itself) if the WSS site definition you are using has some features to be activated:

- Before you can initiate a site creation first there must be an SPSite object created. This will be either done by the SharePoint UI code behind or by you directly creating the SPSite. If the user identity used to create the site is not a farm administrator then the SPSite will be cached by SiteId and by UserToken in the current SPRequestContext.

- You now initiate a new site creation for example by calling SPWebCollection.Add()

- The SharePoint Object Model calls the corresponding unmanaged function of the SP.SPRequest COM object to create the site. This is ISRequest.CreateWeb()

- Based on the WSS site definition to be used for the site, there may be features to be activated. The SP.SPRequest COM object makes calls back to .NET using the ISPFeatureManager COM visible interface of the Microsoft.SharePoint.SPFeatureManager managed class in order to activate those features. It possibly calls EnsureTemplateAssociatedWebFeaturesActivated(). This results in any existing SPSitescached in the same HttpRequest for the same user to be reused. The SPFeatureManager.EnsureSite() method will reuse the cached SPSite or create a new one if there is no cached (when not running in HttpContext).

Now if you are using custom site definitions and depending on how complex are they i.e. how many features do you have and what they are doing, in some cases because the cached SPSite will be reused, you may get some very weird errors particularly when trying to provision two sites at the same time. Some of those errors which I have seen at least a couple of times each include:

———————————————————————————————————————————-

System.InvalidOperationException : You cannot invalidate the SPRequest object while it’s in use.
Source: Microsoft.SharePoint; Help Link: 
Stack Trace:    at Microsoft.SharePoint.Library.SPRequest.ReleaseResources()
    at Microsoft.SharePoint.SPRequestManager.Release(SPRequest request)
    at Microsoft.SharePoint.SPWeb.Invalidate()
    at Microsoft.SharePoint.SPWeb.Close()
    at Microsoft.SharePoint.SPSite.Close()
    at Microsoft.SharePoint.SPSite.Dispose()

———————————————————————————————————————————-

System.Runtime.InteropServices.COMException : Attempted to make calls on more than one thread in single threaded mode. (Exception from HRESULT: 0×80010102 (RPC_E_ATTEMPTED_MULTITHREAD))
Stack Trace:    at Microsoft.SharePoint.Library.SPRequestInternalClass.GetListsWithCallback(String bstrUrl, Guid foreignWebId, String bstrListInternalName, Int32 dwBaseType, Int32 dwBaseTypeAlt, Int32 dwServerTemplate, UInt32 dwGetListFlags, UInt32 dwListFilterFlags, Boolean bPrefetchMetaData, Boolean bSecurityTrimmed, Boolean bGetSecurityData, ISP2DSafeArrayWriter p2DWriter, Int32& plRecycleBinCount)
                       at Microsoft.SharePoint.Library.SPRequest.GetListsWithCallback(String bstrUrl, Guid foreignWebId, String bstrListInternalName, Int32 dwBaseType, Int32 dwBaseTypeAlt, Int32 dwServerTemplate, UInt32 dwGetListFlags, UInt32 dwListFilterFlags, Boolean bPrefetchMetaData, Boolean bSecurityTrimmed, Boolean bGetSecurityData, ISP2DSafeArrayWriter p2DWriter, Int32& plRecycleBinCount)

———————————————————————————————————————————-

ExeName: w3wp.exe, Machine: localhost, Severity: Unexpected, Product: ‘Windows SharePoint Services’, Category: ‘General’
ERROR: request not found in the TrackedRequests. We might be creating and closing webs on different threads. ThreadId = 24, Free call stack =    at Microsoft.SharePoint.SPRequestManager.Release(SPRequest request)
    at Microsoft.SharePoint.SPWeb.Invalidate()
    at Microsoft.SharePoint.SPWeb.Close()
    at Microsoft.SharePoint.SPContentType.CloseWebAsNecessary()
    at Microsoft.SharePoint.SPWeb.Close()
    at Microsoft.SharePoint.SPSite.Close()
    at Microsoft.SharePoint.SPSite.Dispose()

———————————————————————————————————————————-

All of the 3 errors suggest that a cached SPSite may have been reused from a different thread, when it actually shouldn’t have been. It is very difficult to say why those errors do occur but I believe it has to do with the SPRequestContext caching some SPSites which are being reused when activating features. One of the facts that supports this theory is that you don’t get those errors when you are creating the sites from a non HttpContext application such as a custom STSADM command for example. In that case there is no HttpRequest and no SPRequestContext class i.e. no SPSites are being cached and reused. One of the times when I was getting those errors and was trying to resolve the problem I found a couple of post in Internet from people having the same problem but there were no resolutions. Almost everyone experiencing those errors was saying that they don’t get the errors when their code was called by a console application (custom STSADM command).  I wasn’t able to resolve the issue and it left as a known issue in the product. If you have experienced the same problem and have managed to nail it down I would love to hear about your findings/resolution. To mitigate the risk of getting those errors I would suggest that you should:

- Avoid using custom WSS site definitions if they include custom feature receivers that contain complex code

- Consider provisioning your sites from a non HttpContext application whenever possible. OWSTimer jobs count as non HttpContext way of provisioning sites

Also remember that you may be getting those errors for a different reason that is completely unrelated to the SPRequestContext cache. For example if you really are trying to reuse SPWeb or SPSite on a different thread or if you are not disposing properly the SPSite/SPWeb.

And finally lets recap the story about the SPRequest. Almost all read/write content/metadata operations that you do using the .NET SharePoint object model use unmanaged class to do the job. This unmanaged class is called SP.SPRequest and is used from .NET via the SPRequest internal class. The class is implemented and exposed by the OWSSRV.DLL library and has almost 400 methods. Because the SPSite and SPWeb objects have a reference to unmanaged objects you must keep the number of SPSite/SPWeb objects that are active at the same time to a minimum and should dispose them as soon as you have finished using them. You can monitor the creation and disposal of SPRequest objects via the ULS log. In order to minimize the number of SPRequests used in a HttpContext application, SharePoint uses internal cache. It is possible your SPSite objects to be reused when you are creating a site or a web from a web application which site/web is based on a custom WSS site definition that contains custom feature receivers. In rare cases you may be getting errors because of this so if you are using custom WSS site definitions then keep the complexity of your custom feature receivers to a minimum. Consider creating your sites from a non HttpContext application such as an STSADM command, Windows Forms application or a OWSTimer job.

I hope you’ve enjoyed this not very short reading. My next post in the “Understanding SharePoint” series, which will not necessarily be my next post, will be about how asynchronous Event Receivers work in SharePoint and what traps people can get into.

constructor, concat, join, pop, push, reverse, shift, slice, sort, splice or unshift

even if these site columns do not exist in the SharePoint site. If you try to create one of these fields you will get an error message saying that:

The column name that you entered is already in use or reserved. Choose another name.

Hmmm … Well after investigating this one it turned out to be a genuine bug caused by a JavaScript artifact. The way the fldnew.aspx page checks at the client side if a column already exists is by having all existing columns stored in a JavaScript Array called g_FieldName and then checking if the column that user tries to create is already in the array. This is done by the following code:

The problem with this code is that for some unknown to me reason which is probably related to the way the JavaScript VM is implemented, if you create a new Array called arr and then try to read arr["shift"] for example, this will return the following string:

And this happens on both Internet Explorer and Firefox. And this will happen for all strings that are names of methods of the JavaScript Array object and this is case sensitive. Because SharePoint does the check in lower case, then only the methods that are entirely in lower case will be a problem and the list of SharePoint site columns you won’t be able to create from the UI given at the top is the list of all methods of the Array object that are fully lower case.

So how to get around this. Well first of all this is a bug that should be resolved by Microsoft. They could either store everything in upper case (no JavaScript methods are fully upper case) or can change the function to

The problem with the first function is that g_FieldName[strName.toLowerCase()] will be evaluated to true for anything which is not 0, false, null or unknown. In the case of arr["shift"] it will return the string shown on the screen-shot above, which will be evaluated to “true”. So explicitly testing for “1″ will resolve the bug.

In the meantime if you need to create a site column with one of those names you could do it using code or features. And if you wonder how I discovered this issue – well I wanted to create a site column called “Shift” as in work shift which I consider as a totally legitimate use case.

According to the MSDN documentation for NameCtrl:

A developer can create a custom control for displaying presence information if it has the same ProgID (Name.NameCtrl.1), uses the same method or property names, and provides the same functionality that is described in this reference. Functions can be created that handle the event in which the online status of a specified person changes.

What this means is that if you don’t have Office 2007 installed (or another version of the name.dll) then you can implement your own ActiveX control (in .NET or VB6 or C++) and name the COM class “Name.NameCtrl“. Then automatically this control will be called by SharePoint and will display whatever you want.

This could be handy for example when your customers don’t use Outlook as an email client or Office Communicator as an instant office messenger.

The idea is that you add a DelegateControl in your master page or page layout and configure it’s ControlId to a unique id that you are going to define in a separate feature by using a Control element. This way you can replace the control/web part without the need of changing the master page but only by updating your feature. Even more you can activate different controls in different sites by activating different features in those sites, which define different controls with the same Id that matches the ControlId configured in the DelegateControl.

When SharePoint is searching for the control specified in the DelegateControl it can also search only features of a specific scope i.e. Web, Site, etc. This can be configured from the Scope attribute of the DelegateControl. Also you can configure only one control to be rendered or all controls with this id to be rendered with the AllowMultipleControls attribute. If no matching Control element could be found, then no control is rendered i.e. if you want to remove the web part, just deactivate the feature. Additionally the DelegateControl allows a PrefixHtml and SuffixHtml to be defined.

You will probably want to use this approach when you need for different sites to show different web parts on the same location of the page. You do this by using a single page layout (or master page) with a single DelegateControl and then activating different features on those sites which will activate the different web parts to be rendered. Very nice indeed!

Delegate controls are used for defining the search controls in SharePoint. The web controls that are going to be hosted must be registered as safe. You can also host ASCX web user controls the same way.

As you know some tokens can be used when specifying the Url of the custom action: 

Those token will be replaced by SharePoint at runtime with values derived from the current context. To get the complete list we just need to find the function that does the replacement. Luckily this function is in the .NET libraries of SharePoint and can be disassembled with Reflector. So here is the complete list of the tokens:

For the records the method that does the replacement is Microsoft.SharePoint.SPCustomActionElement.ReplaceUrlTokens() and looks like this:

The web, list and item arguments are taken from the context before the user is redirected to the custom action page.

[COMException (0x81020036): The file is currently checked out or locked for editing by another user.]
   Microsoft.SharePoint.Library.SPRequestInternalClass.AddOrUpdateItem(String bstrUrl, String bstrListName, Boolean bAdd, Boolean bSystemUpdate, Boolean bPreserveItemVersion, Boolean bUpdateNoVersion, Int32& plID, String& pbstrGuid, Guid pbstrNewDocId, Boolean bHasNewDocId, String bstrVersion, Object& pvarAttachmentNames, Object& pvarAttachmentContents, Object& pvarProperties, Boolean bCheckOut, Boolean bCheckin, Boolean bMigration, Boolean bPublish) +0
   Microsoft.SharePoint.Library.SPRequest.AddOrUpdateItem(String bstrUrl, String bstrListName, Boolean bAdd, Boolean bSystemUpdate, Boolean bPreserveItemVersion, Boolean bUpdateNoVersion, Int32& plID, String& pbstrGuid, Guid pbstrNewDocId, Boolean bHasNewDocId, String bstrVersion, Object& pvarAttachmentNames, Object& pvarAttachmentContents, Object& pvarProperties, Boolean bCheckOut, Boolean bCheckin, Boolean bMigration, Boolean bPublish) +411

[SPException: The file is currently checked out or locked for editing by another user.]
   Microsoft.SharePoint.Library.SPRequest.AddOrUpdateItem(String bstrUrl, String bstrListName, Boolean bAdd, Boolean bSystemUpdate, Boolean bPreserveItemVersion, Boolean bUpdateNoVersion, Int32& plID, String& pbstrGuid, Guid pbstrNewDocId, Boolean bHasNewDocId, String bstrVersion, Object& pvarAttachmentNames, Object& pvarAttachmentContents, Object& pvarProperties, Boolean bCheckOut, Boolean bCheckin, Boolean bMigration, Boolean bPublish) +556
   Microsoft.SharePoint.SPListItem.AddOrUpdateItem(Boolean bAdd, Boolean bSystem, Boolean bPreserveItemVersion, Boolean bNoVersion, Boolean bMigration, Boolean bPublish, Boolean bCheckOut, Boolean bCheckin, Guid newGuidOnAdd, Int32& ulID, Object& objAttachmentNames, Object& objAttachmentContents, Boolean suppressAfterEvents) +3032
   Microsoft.SharePoint.SPListItem.UpdateInternal(Boolean bSystem, Boolean bPreserveItemVersion, Guid newGuidOnAdd, Boolean bMigration, Boolean bPublish, Boolean bNoVersion, Boolean bCheckOut, Boolean bCheckin, Boolean suppressAfterEvents) +636
   Microsoft.SharePoint.SPListItem.Update() +194
   Microsoft.SharePoint.WebControls.SaveButton.SaveItem(SPContext itemContext, Boolean uploadMode, String checkInComment) +1806
   Microsoft.SharePoint.WebControls.SaveButton.SaveItem() +111
   Microsoft.SharePoint.WebControls.SaveButton.OnBubbleEvent(Object source, EventArgs e) +476
   System.Web.UI.Control.RaiseBubbleEvent(Object source, EventArgs args) +50
   System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) +32
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +3215

 It turns out that SharePoint will not allow you to rename a folder if there is a file under it which is currently checked out. The trouble is SharePoint doesn’t tell you which is the file.

To find out which is the file you can use SharePoint Designer to rename the same folder. Then the error message returned by SharePoint Designer will include the file name being checked out and the user it is checked out to.