Executing long running process from a web site


I recently had to build a report which accepts one xls excel file, churns through a bunch the data looking stuff up on other systems and performing calculations, then spits out another xls excel sheet the other end. The whole process takes about 15 to 20 minutes to run and at present is a simple command line application which accepts a couple of parameters (input and output filenames) and I fire up manually- obviously a situation which is no good moving forward.

All our internal business systems are web forms/ mvc running from local IIS boxes so ideally I would like to add a screen where the user can submit their xls file, hit go, and then get the result back 15/ 20 minutes later- but I don’t want to tie up one of the threads in the aspnet worker thread pool for the entire duration. Some Googling revealed the best approach appears to be to write a windows service which hosts the long running process then get the web app to call across to that to kick off the task.

Implementation

This is to be a standard Windows Service which will run my long running process, hosting a WCF service which the website can use to call into it with. If you;ve not done this before, you can follow the steps on the MSDN ; msdn.microsoft.com/en-us/library/ms733069.aspx.

Create your WCF Service which will respond to requests from the website. In my instance I wanted to be able to start the operation, passing a byte[] containing the xls file, request status updates, then finally request the resulting xls file, again as a byte[]. For the sake of this example, the WCF service is the one which is doing all the work with regards spawning the worker thread and tracking updates from the actual worker class, and the worker class is a type called “Calculator”.

    using System.ServiceModel;
    [ServiceContract(Namespace = "http://Moneybarn.VIVS.FleetRevaluation")]
    public interface IMyWCFService
    {
        [OperationContract]
        string Start(byte[] excel_file, string user_name);

        [OperationContract]
        Progress GetProgress();

        [OperationContract]
        byte[] GetResult();
    }

    using System.Threading;
    public class MyWCFService: IMyWCFService
    {
        private static string CurrentUserInstance;
        private static Calculator MyCalculatorInstance;
        private static Progress LastUpdate;
        private static Thread WorkerThread;
        private static bool IsComplete;
        private static string LastError;
        private static byte[] FinishedFile;

        public string Start(byte[] excel_file, string user_name)
        {
            if (WorkerThread == null)
            {
                CurrentUserInstance = user_name;
                MyCalculatorInstance = new Calculator(excel_file);

                MyCalculatorInstance.OnProgress += new OnProgressEventHandler(MyCalculatorInstance_OnProgress);
                MyCalculatorInstance.OnComplete += new OnCompleteEventHandler(MyCalculatorInstance_OnComplete);
                MyCalculatorInstance.OnError += new OnErrorEventHandler(MyCalculatorInstance_OnError);

                WorkerThread = new Thread(new ThreadStart(MyCalculatorInstance.Start));
                WorkerThread.Start();

                return null; // no news is good news!
            }
            else
            {
                return "Instance already running for user " + CurrentUserInstance;
            }
        }

        public Progress GetProgress()
        {
            return LastUpdate;
        }

        public string GetError()
        {
            return LastError;
        }

        public byte[] GetResult()
        {
            if (IsComplete)
            {
                byte[] buff = FinishedFile;

                MyCalculatorInstance = null;
                WorkerThread = null;
                CurrentUserInstance = string.Empty;
                LastUpdate = null;
                FinishedFile = null;
                IsComplete = false;

                return buff;
            }
            else
                return null;
        }

        private void MyCalculatorInstance_OnError(string error)
        {
            LastError = error;
        }

        private void MyCalculatorInstance_OnComplete(byte[] the_file)
        {
            FinishedFile = the_file;
            IsComplete = true;
        }

        private void MyCalculatorInstance_OnProgress(int stage, float percent)
        {
            LastUpdate = new Progress() { 
                Stage = stage, 
                Percent = (int)(percent * 100) 
            };
        }
    }

Add a new class called ProjectInstaller- this will handle registering your service with Windows

    using System.ComponentModel;
    using System.Configuration.Install;
    using System.ServiceProcess;

    [RunInstaller(true)]
    public class ProjectInstaller : Installer
    {
        private ServiceProcessInstaller process;
        private ServiceInstaller service;

        public ProjectInstaller()
        {
            process = new ServiceProcessInstaller();
            process.Account = ServiceAccount.LocalSystem;
            service = new ServiceInstaller();
            service.ServiceName = "MyWindowsService";  // change this!
            Installers.Add(process);
            Installers.Add(service);
        }
    }

Add the actual service body; add a new “Windows Service” file to the project. This is the meat of the service, and contains no real logic- it’s only job is to fire up the ServiceHost for your WCF service.

    using System.ServiceModel;
    using System.ServiceProcess;

    public class MyWindowsService : ServiceBase
    {
        public ServiceHost serviceHost = null;

        public MyWindowsService()
        {
            // Name the Windows Service
            ServiceName = "MyWindowsService";
        }

        public static void Main()
        {
            ServiceBase.Run(new MyWindowsService());
        }

        // Start the Windows service.
        protected override void OnStart(string[] args)
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
            }

            // The type of your WCF service
            serviceHost = new ServiceHost(typeof(MyWCFService));

            serviceHost.Open();
        }

        protected override void OnStop()
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
                serviceHost = null;
            }
        }
    }

Finally there is a little bit of config to pop into the app.config for the WCF endpoints- I’ve created a custom binding here to allow me to receive larger files over the WCF call than the default limits allow (in this case, up to 10 meg);

  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="bigWSHttpBinding" 
                 maxBufferPoolSize="10485760" maxReceivedMessageSize="10485760"> <!-- 10 MB limit -->
          <readerQuotas maxDepth="10485760" maxStringContentLength="10485760" maxArrayLength="10485760" maxBytesPerRead="10485760" />
        </binding>
      </wsHttpBinding>
    </bindings>
    <services>
      <service behaviorConfiguration="MyWCFServiceBehavior"
        name="Demo.MyWCFService">
        <endpoint address="" binding="wsHttpBinding" bindingConfiguration="bigWSHttpBinding"
          contract="Demo.IMyWCFService" />
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="http://localhost:8000/MyWCFService/service" />
          </baseAddresses>
        </host>
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="MyWCFServiceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

I also created a couple of little batch files which I mark as “Copy to output directory” for installing and removing the service;

rem install.bat
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe
net start MyWindowsService
rem remove.bat
net stop MyWindowsService
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe /u

That’s the Windows service part built. You can now build, and jump to the bin output folder and run the install.bat file from the command line- with any luck (more likly with a little debugging!) you will be up and running. The second part is even easier;

Jump over to your web app, add a service reference to the url of the base address from your app config above then it’s just a matter of exposing those web service methods so that you can call them from your page with an ajax call- so in MVC I created a controller with Start, GetProgress & GetResult methods which returned JsonResults called straight from the client with jQuery.

  1. #1 by Jeroen de Graaff on May 4, 2013 - 14:13

    Hi,
    The code looks great and thank you very much.
    It got me going…

    May I ask if you could provide a sample of the script that you used in MVC to get the status and results back? Because that’s the part that’s still troubling me..

    With regards,

    Jeroen

  2. #2 by shawson on May 31, 2013 - 13:54

    Hey,

    Yea, there’s nothing too it really- as you will have consumed the service already, I just created an action on my controller which returns a JsonResult which can be hit using an http get- this action just calls the GetProgress method on the underlying WCF service and returns the result. On the html on the front end, there is a piece of jquery which just does an ajax call to this action every second and updates a progress bar.

(will not be published)