Running execadmsvcjobs through code

Well there are many ways you could achieve this and this is just one of  them. It simply creates a new instance of the SPExecuteAdministrationServiceJobs class and calls its Run method. This will start and wait for all currently registered and/or running timer jobs to finish. The code is below:

string stsadmPath = SPUtility.GetGenericSetupPath(“BIN”);

Assembly stsadmAsm = Assembly.LoadFile(Path.GetFullPath(stsadmPath + “\\STSADM.EXE”));

Type admSvcJobType = stsadmAsm.GetType(“Microsoft.SharePoint.StsAdmin.SPExecuteAdministrationServiceJobs”);

ConstructorInfo ci = admSvcJobType.GetConstructor(new Type[] { typeof(SPGlobalAdmin) });

using (SPGlobalAdmin admin = new SPGlobalAdmin())

{

    object svcjobs = ci.Invoke(new object[] { admin });

    MethodInfo runMi = admSvcJobType.GetMethod(“Run”, BindingFlags.Instance | BindingFlags.Public);

 

    runMi.Invoke(svcjobs, new object[] { new StringDictionary() });

}

One place this could be handy is when you provision a web application on multiple front ends using SPWebApplication.ProvisionGlobaly(). This method creates a timer job and returns immediately. However if you need the web application to be created before you continue you need to wait for the timer job(s) to finish. One way to do this programmatically is using the code above.

Understanding SharePoint: Event Receivers

This blog post shows some details about how the event receivers in SharePoint work and also provides a solution how to wait for an ItemAdded event receiver to complete before the EditForm.aspx page is displayed when uploading a document. The solution could be used to wait for any asynchronous event receiver when you are changing the list using code.

All list item, list and web event receivers in SharePoint with the exception of the ListItemFileConverted receiver are run from the SPRequest unmanaged class. There are two types of event receivers – synchronous (such as ItemAdding and ItemUpdating) and asynchronous (such as ItemAdded and ItemUpdated). Because the event receivers are implemented in managed code, the unmanaged SPRequest class needs a way to invoke them. This is done via the ISPEventManager COM interface which is implemented by the Microsoft.SharePoint.SPEventManager internal class.

[ComImport, SuppressUnmanagedCodeSecurity, InterfaceType((short)1), Guid(“BDEADF0F-C265-11D0-BCED-00A0C90AB50F”)]

public interface ISPEventManager

{

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]

    void ExecuteItemEventReceivers(

        ref byte[] userToken, ref objecteventReceivers, ref ItemEventReceiverParams itemEventParams,

        out objectchangedFields, outEventReceiverResult eventResult, out stringerrorMessage);

 

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]

    void EnqueueItemEventReceivers(ref byte[] userToken, ref objecteventReceivers, ref ItemEventReceiverParams itemEventParams);

 

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]

    void ExecuteListEventReceivers(

        ref byte[] userToken, ref objecteventReceivers, ref ListEventReceiverParams ListEventParams,

        outEventReceiverResult eventResult, out stringerrorMessage);

 

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]

    void EnqueueListEventReceivers(ref byte[] userToken, ref objecteventReceivers, ref ListEventReceiverParams ListEventParams);

 

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]

    void ExecuteWebEventReceivers(

        ref byte[] userToken, ref objecteventReceivers, ref WebEventReceiverParams webEventParams,

        outEventReceiverResult eventResult, out stringerrorMessage);

 

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]

    void EnqueueWebEventReceivers(ref byte[] userToken, ref objecteventReceivers, ref WebEventReceiverParams webEventParams);

}

As we can see from the interface all event receivers are passed to be executed in one call and there are two types of methods in the ISPEventManager class: EnqueueEventReceivers – used for the asynchronous ones and ExecuteEventReceivers – used for the synchronous.

So because the unmanaged code will make a call back to the managed SPEventManager class to run the event receivers, the first important question is which process will those event receivers run in. Well the answer is both synchronous and asynchronous event receivers will execute in the same process that has made the change that resulted in the event receivers being called. So for example if you are adding a new file into a document library from a web part, then the event receivers will execute in the w3wp.exe process of the application pool of the web application. If you are adding the file from a rich client (MyWinFormsApp.exe) then the event receivers, including the asynchronous ones will execute in the MyWinFormsApp.exe process.

The asynchronous event receivers may need some time to complete (could be a lot of time) and if you close your MyWinFormsApp.exe immediately after you have added the file, and while the event receivers are still running, what will happen is that they will be terminated and some of them may not run at all.

The situation with IIS is not very different unfortunately. If you do an iisreset without the /noforce switch then all currently running asynchronous event receivers will be terminated immediately. My tests showed that executing iisreset /noforce will indeed wait for an asynchronous event receiver to finish however if you are trying to save any changes to the item for example, those changes will not survive.

So this means that by doing iisreset you may be interrupting a running asynchronous (or synchronous) event receiver and corrupting your application integrity/state. When executing iisreset /noforce the event receivers will complete but they may not be able to update any of the SharePoint objects they could be trying to manipulate. This finding it actually very alarming! I wouldn’t recommend to anyone writing any event receiver that will take more than 1 second to complete as an iisreset may corrupt your application!

The asynchronous event receivers will be run in their own thread from the System.Threading.ThreadPool. And a new thread will be queued up with every call from the unmanaged SPRequest class. This means that if you add say 100 files to a document library this will result in 100 calls to EnqueueItemEventReceivers for each of the items. And this will create and en-queue 100 threads each of which will run one by one all asynchronous event receivers for the corresponding file. Now if some of those event receivers are taking some time to complete, which is a few times more than the time it takes you to add a file, what will happen is you will end up with enormous amount of running threads.

Switching between threads is a very resource intensive operation and if you have 100 threads running in your application or in IIS, this will pretty much slow down the process to a crawl. I’ve seen this happening with a custom built migration tool that moves documents from one document library to another. There were some heavy ItemAdded event receivers and after few hundreds files were moved for a minute or so, it took another 45 minutes for the event receivers to complete. The migration would have been faster if after adding each file the migration tool was waiting for the event receivers to complete because after adding all the files at once 90+% of the time the process was spending in switching between the few hundred threads and only up to 10% were the actual threads running the event receivers.

The next interesting question is what is the lifespan for the SPItemEventProperties that are passed to all item event receivers (synchronous and asynchronous). Well looking at the SPEventManager class code it turns out that the SPItemEventProperties are created once per batch of events receivers and are shared among all of them. So this means that all synchronous event receivers will share the same properties and all asynchronous event receivers will share different instance of the properties. However the unmanaged SPRequest class will keep the changes to the properties done by the synchronous event receivers and will pass them to the asynchronous event receivers.  This means that it is generally possible to store information in the SPEventProperties from some event receivers that could be passed to other event receivers executed later on.

This could only be done from an ItemAdding or ItemUpdating event receiver. You could add anything you want to the AfterProperties collection and the information you have added will be availabe as AfterProperties in the ItemAdded and ItemUpdated event receivers.

A question that have been bothering me for a while is should the web retrieved by SPItemEventProperties.OpenWeb() be disposed by the event receiver that have opened it. The answer as always lies in the implementation details of the class (in out case SPItemEventProperties) that can be retrieved using Reflector.

public sealed class SPItemEventProperties : SPEventPropertiesBase, IDisposable

{

    …

 

    private SPSite OpenSite()

    {

        if (((this.m_site == null) && (this.WebUrl != null)) && (this.m_site == null))

        {

            if (this.m_userToken == null)

            {

                this.m_site = new SPSite(this.WebUrl);

            }

            else

            {

                this.m_site = new SPSite(this.WebUrl, this.m_userToken);

            }

            this.m_siteCreatedByThis = true;

        }

        return this.m_site;

    }

 

    …

 

    public SPWeb OpenWeb()

    {

        this.OpenSite();

        if (this.m_site == null)

        {

            return null;

        }

        return this.m_site.OpenWeb(this.RelativeWebUrl);

    }

 

    …

 

    public void Dispose()

    {

        if (this.m_site != null)

        {

            while (this.m_siteCreatedByThis)

            {

                this.m_site.Dispose();

                this.m_site = null;

                this.m_siteCreatedByThis = false;

                break;

            }

        }

    }

}

So the SPItemEventProperties class actually implements IDisposable and disposes the allocated site or web. The SPEventManager class makes sure to call Dispose() after the batch of event receivers have executed.

Waiting for ItemAdded to Complete Before Showing EditForm.aspx

There is an issue with document libraries and the Upload.aspx page as descibed in my previous blog entry. The problem will be encountered if you have an ItemAdded event receiver attached to the document library that updates fields of the list item/document after it has been created. Because the ItemAdded event receiver runs asynchronously it may update the list item after the EditForm.aspx page has been displayed. This would result either in the EditForm.aspx form to not show the updated fields or in an exception to be generated when you click OK.

In order to solve the problem we need some code to wait for the event receiver to complete before loading the EditForm.aspx page. One way to do this is to customize the EditForm.aspx page either on a Content Type level or a List level or just using SharePoint designer and to insert a web control that will wait for the ItemAdded event receiver to complete. We also need a special ItemAdded event receiver which the web control will be able to communicate with.

And the good news for you is that I did the hard work or creating both the base class for the event receiver and the web control. They can be donwloaded here:

SharePointInternals.SynchronousItemAdded.dll

SharePointInternals.SynchronousItemAdded – Source Code

To use them you need to create your event receiver class to inherit from the SPSynchronousReceiver class and override the ItemAddedSynchronously method:

public class SPTestReceiver : SPSynchronousReceiver

{

          

    protected override void ItemAddedSynchronously(SPItemEventProperties properties)

    {

        // Your code goes here

    }

 

 

    // Your other item receiver overrides go here   

 

}

Then you need to insert the WaitForItemAdded web control in the EditPage.aspx. The best place to do this is as a first control in the PlaceHolderMain.

<%@Register TagPrefix=”SharePointInternals”

            Assembly=”SharePointInternals.SynchronousItemAdded, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d7dbdc19a16aed51″

            namespace=”SharePointInternals.WebControls”%>

 

 

…

 

<asp:Content ContentPlaceHolderId=”PlaceHolderMain” runat=”server”>

    <SharePointInternals:WaitForItemAdded ID=”waitForItemAdded1″ runat=”server”/>

 

    …

   

</asp:Content>

Unfortunately this is the only way to integrate the WaitForItemAdded control. If we add the web control to the ListFormTemplate used by the ListFormWebPart from EditForm.aspx, then it will wait for the event receivers to complete, but at this stage the ListItem would have been already loaded in the SPContext and the EditForm.aspx will show the field values of the list item before the ItemAdded event receivers have completed. If any of them have updated the list item then pressing OK on the EditForm.aspx will actually cause an exception:

The file … has been modified by … on …

In order to get around the problem that the ListItem from the SPContext have been initialized we should put the WaitForItemAdded web control somewhere in the EditForm.aspx before the ListFormWebPart and before any other control that may be using SPContext.Current.ListItem. Accessing this property will load the list item which may happen before the asynchronous event receivers have completed.

You can also wait for the ItemAdded event receivers to complete when adding a file using code. This could be handy when doing migration of documents. In order to wait for the event receiver to complete just call SPSynchronousReceiver.WaitForItemAddedReceivers().

using (SPSite site = new SPSite(“http://server/sites/test&#8221;))

using (SPWeb web = site.OpenWeb())

{

    SPList list = web.Lists[“Shared Documents”];

 

    SPFile file = list.RootFolder.Files.Add(fileName, fileBytes);

 

    SPSynchronousReceiver.WaitForItemAddedReceivers(list, file.Item.ID);

}

Of course this will only wait for those event receivers that inherit from the SPSynchronousReceiver class. If you need to do the same for another asycnhronous event receiver, this could be easily done by changing the code of the base class.

And for completeness I also want to mention that the ListItemFileConverted event receivers execution is initiated from managed code when SPFile.Convert() is called. This is the only exception and the execution of all other event receivers is initialized from unmanaged code. The ListItemFileConverted event receivers run asynchronously.

Finally to recap our findings. The execution of all event receivers with the exception of the ListItemFileConverted will be initiated from unmanaged code. This happens when the SharePoint object model calls the corresponding method from the SPRequest unmanaged object to perform the operation that will trigger the event receivers. Then the unmanaged WSS world will use callbacks on the ISPEventManager COM interface implemented by the Microsoft.SharePoint.SPEventManager class to start the event receivers execution on batches of synchronous and asynchronous event receivers. You don’t have to dispose the SPWeb returned by properties.OpenWeb() in your event receiver code. Sometimes it may be a better idea to create a fresh SPSite and SPWeb instead of using properties.OpenWeb(). It is hard to say which one is good for you as it all depends on the parallel processes that may be running and the other event receivers being executed. All asynchronous event receivers for the given object (list item, list of web) will be executed in a separate thread created on the ThreadPool. SharePoint will queue a new thread for every object. This could potentially cause the server or the application to slow down dramatically as it struggles to switch between the different threads.  The asynchonous event receivers run in the same process in which the object was changed that lead to the event receivers being executed. If you stop the application or reset IIS all running event receivers will be terminated and all scheduled event receivers will not be executed at all. If you do iisreset /noforce this will wait for the event receivers to complete, but they may not be able to update the SharePoint objects they are working with. When you upload a file to a document library there is a nasty problem that you will be redirected to the EditForm.aspx page before the ItemAdded event receivers have completed. If you are doing any changes to the item in the event receivers these changes will not show up when EditForm.aspx page loads. Even more if you press OK in the EditForm.aspx page after the event receivers have modified the item you will get an exception. There is workaround if you use the SharePointInternals.SynchronousItemAdded.dll as described in this blogpost.

Understanding SharePoint: List Forms

For my Event Receiver’s post I had to do some more investigation on how the List Forms work and I thought it will be a good idea to put together my findings. As you know each list has 3 forms – one for creating a new item, one for editing an existing item and one for displaying an exiting item.

The first place where those forms are defined is in the List schema xml. In SharePoint 2003 (WSS 2.0) the forms used to be defined using a complex CAML but from SharePoint 2007 (WSS 3.0) they are implemented in ASPX pages using web controls. The Form element in the List schema xml is used to define the aspx page to be used for the given List Form and by default most lists use NewForm.aspx, EditForm.aspx and DistForm.aspx pages.

If necessary the aspx pages for the List forms can be redefined per item content type. This can be achieved by including a FormUrls XmlDocument with your custom content type.

So sfter SharePoint has determined from the List schema xml and the ContentType FormUrls element which aspx page should be used it opens up that page. Each of those pages contain a ListFormWebPart which does all the job of rendering the list item. SharePoint 2007 still supports legacy forms and you can use the UseLegacyForm attribute of the Form element in the List schema xml to tell SharePoint you want to use the CALM forms. In such a case the rendering of the List Form will be done from the unmanaged SPRequest class, which will return the HTML to be rendered.

In most of the cases no legacy forms are used and then the ListFormWebPart needs to determine which Form Template has to be used with the List Form. The Form Templates provide a way to further customize the list forms without changing the aspx pages. This is what SharePoint does as it uses exactly the same NewForm.aspx page for all new forms but uses different Form Templates for a standard list and for a document library. The Form Templates also allow to customize the list forms for items in the same list but for different content types.

The Form Templates are defined as an XmlDocument per content type and all content types that inherit from the Item content type also inherit the Form Templates as defined in the Item Content Type. All forms use a Form Template called ListForm.

 <FormTemplates xmlns=”http://schemas.microsoft.com/sharepoint/v3/contenttype/forms”&gt;

    <Display>ListForm</Display>

    <Edit>ListForm</Edit>

    <New>ListForm</New>

</FormTemplates>

However not all forms in SharePoint use these Form Templates. For example they are redefined by the Document content type (and all content types that inherit from Document) and they use a Form Template called DocumentLibraryForm instead of ListForm. 

The next question is where those Form Templates come from and the answer is they usually come from the 12\TEMPLATE\CONTROLTEMPLATES\DefaultTemplates.ascx page. The form templates are simply RenderingTemplate web controls and define the sets of controls to be added to the List Form. The Form Templates are matched by their ID attribute and it is actually possible to overwrite an existing template or define your own Form Templates and deploy them in an ascx page in  12\TEMPLATE\CONTROLTEMPLATES.

The code below shows the standard ListForm templates used by all generic lists:

<SharePoint:RenderingTemplate ID=”ListForm” runat=”server”>

      <Template>

            <SPAN id=’part1′>

                  <SharePoint:InformationBar ID=”InformationBar1″ runat=”server”/>

                  <wssuc:ToolBar CssClass=”ms-formtoolbar” id=”toolBarTbltop” RightButtonSeparator=”&nbsp;” runat=”server”>

                              <Template_RightButtons>

                                    <SharePoint:NextPageButton ID=”NextPageButton1″ runat=”server”/>

                                    <SharePoint:SaveButton ID=”SaveButton1″ runat=”server”/>

                                    <SharePoint:GoBackButton ID=”GoBackButton1″ runat=”server”/>

                              </Template_RightButtons>

                  </wssuc:ToolBar>

                  <SharePoint:FormToolBar ID=”FormToolBar1″ runat=”server”/>

                  <TABLE class=”ms-formtable” style=”margin-top: 8px;” border=0 cellpadding=0 cellspacing=0 width=100%>

                  <SharePoint:ChangeContentType ID=”ChangeContentType1″ runat=”server”/>

                  <SharePoint:FolderFormFields ID=”FolderFormFields1″ runat=”server”/>

                  <SharePoint:ListFieldIterator ID=”ListFieldIterator1″ runat=”server”/>

                  <SharePoint:ApprovalStatus ID=”ApprovalStatus1″ runat=”server”/>

                  <SharePoint:FormComponent ID=”FormComponent1″ TemplateName=”AttachmentRows” runat=”server”/>

                  </TABLE>

                  <table cellpadding=0 cellspacing=0 width=100%><tr><td class=”ms-formline”><IMG SRC=”/_layouts/images/blank.gif” width=1 height=1 alt=””></td></tr></table>

                  <TABLE cellpadding=0 cellspacing=0 width=100% style=”padding-top: 7px”><tr><td width=100%>

                  <SharePoint:ItemHiddenVersion ID=”ItemHiddenVersion1″ runat=”server”/>

                  <SharePoint:ParentInformationField ID=”ParentInformationField1″ runat=”server”/>

                  <SharePoint:InitContentType ID=”InitContentType1″ runat=”server”/>

                  <wssuc:ToolBar CssClass=”ms-formtoolbar” id=”toolBarTbl” RightButtonSeparator=”&nbsp;” runat=”server”>

                              <Template_Buttons>

                                    <SharePoint:CreatedModifiedInfo ID=”CreatedModifiedInfo1″ runat=”server”/>

                              </Template_Buttons>

                              <Template_RightButtons>

                                    <SharePoint:SaveButton ID=”SaveButton2″ runat=”server”/>

                                    <SharePoint:GoBackButton ID=”GoBackButton2″ runat=”server”/>

                              </Template_RightButtons>

                  </wssuc:ToolBar>

                  </td></tr></TABLE>

            </SPAN>

            <SharePoint:AttachmentUpload ID=”AttachmentUpload1″ runat=”server”/>

      </Template>

</SharePoint:RenderingTemplate>

You can probably recognize all the controls you have seen many times in those NewForm.aspx pages. There are many “interesting” controls in that form, such as the ListFieldIterator, InitContentType, SaveButton, AttachmentUpload. A detailed information on how to customize your ListForms, include custom fields and customize the Form Templates could be found in the following MSDN article.

Issues with Document Libraries and _layouts/Upload.aspx

There is one specific issue when using a NewForm with a ListFormWebPart in a Document Library. You will be unconditionally redirected to the _layouts/Upload.aspx page so you can upload the file first. After the file is uploaded you will be redirected to the EditForm. So what does this mean:

– Well first of all the “List Form” and “Form Template” of  the NewForm are actually not used because you will be unconditionally redirected to the Upload.aspx page. Of course you have the alternative of creating a custom New Form that _does not_ contain a ListFormWebPart. However all the SharePoint Field controls require to be inside a ListFormWebPart so you won’t be able to use those. You can actually use the SharePoint Field controls without a ListFormWebPart but you will need to create a new SPContext object in the Page.OnInit() event using reflection and set the ItemContext property of the SharePoint Field web control. No need to say that this will probably not be supported by Microsoft.

– Secondly the ItemAdded and ItemAdding event receivers will be initiated from the Upload.aspx page after the file is uploaded and a blank list item has been created. However this happens before you are redirected to the Edit Form and before the user has filled in the metadata of the document. Even worse the ItemAdded event receiver is executed asynchronously and there is no guarantee when it will be run. Depending on how loaded is the server and how quick the user clicks “Check In” on the Edit Form, the ItemAdded event receiver can be actually executed after the Edit Form has been submitted! Also if you rely the ItemAdded event receiver to set some default values for the item after it has been created, which values  you expect to be displayed in the Edit Form, it is quite possible that the ItemAdded will be executed after the Edit Form has rendered the old values of the item (before they have been changed by the event receiver).

– Lastly if you press “Cancel” on the Edit Form after the file has been already uploaded the file and the list item may remain in a strange state where it is only visible to the person who has uploaded the file and is even not visible to the system account. This happens when the document library is in a “Force Check Out” mode. See more in this post.