Archive for February, 2013

Executing long running process from a web site

I recently had to build a report which accepts one xls excel file, churns through a bunch the data looking stuff up on other systems and performing calculations, then spits out another xls excel sheet the other end. The whole process takes about 15 to 20 minutes to run and at present is a simple command line application which accepts a couple of parameters (input and output filenames) and I fire up manually- obviously a situation which is no good moving forward.

All our internal business systems are web forms/ mvc running from local IIS boxes so ideally I would like to add a screen where the user can submit their xls file, hit go, and then get the result back 15/ 20 minutes later- but I don’t want to tie up one of the threads in the aspnet worker thread pool for the entire duration. Some Googling revealed the best approach appears to be to write a windows service which hosts the long running process then get the web app to call across to that to kick off the task.

Implementation

This is to be a standard Windows Service which will run my long running process, hosting a WCF service which the website can use to call into it with. If you;ve not done this before, you can follow the steps on the MSDN ; msdn.microsoft.com/en-us/library/ms733069.aspx.

Create your WCF Service which will respond to requests from the website. In my instance I wanted to be able to start the operation, passing a byte[] containing the xls file, request status updates, then finally request the resulting xls file, again as a byte[]. For the sake of this example, the WCF service is the one which is doing all the work with regards spawning the worker thread and tracking updates from the actual worker class, and the worker class is a type called “Calculator”.

    using System.ServiceModel;
    [ServiceContract(Namespace = "http://Moneybarn.VIVS.FleetRevaluation")]
    public interface IMyWCFService
    {
        [OperationContract]
        string Start(byte[] excel_file, string user_name);

        [OperationContract]
        Progress GetProgress();

        [OperationContract]
        byte[] GetResult();
    }

    using System.Threading;
    public class MyWCFService: IMyWCFService
    {
        private static string CurrentUserInstance;
        private static Calculator MyCalculatorInstance;
        private static Progress LastUpdate;
        private static Thread WorkerThread;
        private static bool IsComplete;
        private static string LastError;
        private static byte[] FinishedFile;

        public string Start(byte[] excel_file, string user_name)
        {
            if (WorkerThread == null)
            {
                CurrentUserInstance = user_name;
                MyCalculatorInstance = new Calculator(excel_file);

                MyCalculatorInstance.OnProgress += new OnProgressEventHandler(MyCalculatorInstance_OnProgress);
                MyCalculatorInstance.OnComplete += new OnCompleteEventHandler(MyCalculatorInstance_OnComplete);
                MyCalculatorInstance.OnError += new OnErrorEventHandler(MyCalculatorInstance_OnError);

                WorkerThread = new Thread(new ThreadStart(MyCalculatorInstance.Start));
                WorkerThread.Start();

                return null; // no news is good news!
            }
            else
            {
                return "Instance already running for user " + CurrentUserInstance;
            }
        }

        public Progress GetProgress()
        {
            return LastUpdate;
        }

        public string GetError()
        {
            return LastError;
        }

        public byte[] GetResult()
        {
            if (IsComplete)
            {
                byte[] buff = FinishedFile;

                MyCalculatorInstance = null;
                WorkerThread = null;
                CurrentUserInstance = string.Empty;
                LastUpdate = null;
                FinishedFile = null;
                IsComplete = false;

                return buff;
            }
            else
                return null;
        }

        private void MyCalculatorInstance_OnError(string error)
        {
            LastError = error;
        }

        private void MyCalculatorInstance_OnComplete(byte[] the_file)
        {
            FinishedFile = the_file;
            IsComplete = true;
        }

        private void MyCalculatorInstance_OnProgress(int stage, float percent)
        {
            LastUpdate = new Progress() { 
                Stage = stage, 
                Percent = (int)(percent * 100) 
            };
        }
    }

Add a new class called ProjectInstaller- this will handle registering your service with Windows

    using System.ComponentModel;
    using System.Configuration.Install;
    using System.ServiceProcess;

    [RunInstaller(true)]
    public class ProjectInstaller : Installer
    {
        private ServiceProcessInstaller process;
        private ServiceInstaller service;

        public ProjectInstaller()
        {
            process = new ServiceProcessInstaller();
            process.Account = ServiceAccount.LocalSystem;
            service = new ServiceInstaller();
            service.ServiceName = "MyWindowsService";  // change this!
            Installers.Add(process);
            Installers.Add(service);
        }
    }

Add the actual service body; add a new “Windows Service” file to the project. This is the meat of the service, and contains no real logic- it’s only job is to fire up the ServiceHost for your WCF service.

    using System.ServiceModel;
    using System.ServiceProcess;

    public class MyWindowsService : ServiceBase
    {
        public ServiceHost serviceHost = null;

        public MyWindowsService()
        {
            // Name the Windows Service
            ServiceName = "MyWindowsService";
        }

        public static void Main()
        {
            ServiceBase.Run(new MyWindowsService());
        }

        // Start the Windows service.
        protected override void OnStart(string[] args)
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
            }

            // The type of your WCF service
            serviceHost = new ServiceHost(typeof(MyWCFService));

            serviceHost.Open();
        }

        protected override void OnStop()
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
                serviceHost = null;
            }
        }
    }

Finally there is a little bit of config to pop into the app.config for the WCF endpoints- I’ve created a custom binding here to allow me to receive larger files over the WCF call than the default limits allow (in this case, up to 10 meg);

  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="bigWSHttpBinding" 
                 maxBufferPoolSize="10485760" maxReceivedMessageSize="10485760"> <!-- 10 MB limit -->
          <readerQuotas maxDepth="10485760" maxStringContentLength="10485760" maxArrayLength="10485760" maxBytesPerRead="10485760" />
        </binding>
      </wsHttpBinding>
    </bindings>
    <services>
      <service behaviorConfiguration="MyWCFServiceBehavior"
        name="Demo.MyWCFService">
        <endpoint address="" binding="wsHttpBinding" bindingConfiguration="bigWSHttpBinding"
          contract="Demo.IMyWCFService" />
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="http://localhost:8000/MyWCFService/service" />
          </baseAddresses>
        </host>
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="MyWCFServiceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

I also created a couple of little batch files which I mark as “Copy to output directory” for installing and removing the service;

rem install.bat
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe
net start MyWindowsService
rem remove.bat
net stop MyWindowsService
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe /u

That’s the Windows service part built. You can now build, and jump to the bin output folder and run the install.bat file from the command line- with any luck (more likly with a little debugging!) you will be up and running. The second part is even easier;

Jump over to your web app, add a service reference to the url of the base address from your app config above then it’s just a matter of exposing those web service methods so that you can call them from your page with an ajax call- so in MVC I created a controller with Start, GetProgress & GetResult methods which returned JsonResults called straight from the client with jQuery.

2 Comments

My 8 Week Game Entry!

For the last 8 weeks I’ve been beavering away on a game to enter into the 8 week game competition! The last one (back in 2010) I won! (with Manic Spaceman, a Manic Miner clone, set in.. you guessed it… space)! So the pressure is on this time round with more competitors, some of which have experience in the games industry.

You can try out “The Distant Future (The Year 2000)” Here!

As last time, I had the choice of either XNA windows game, or a browser based html5/javascript/canvas effort- I went for the latter. I decided to make a game in the vein of the original classic Syndicate from the Amiga made by bullfrog. The competition states you have to do all the coding yourself, but other assets are up to you, which is great because I suck at trying to draw little people and animate them; so I managed to get the original sprites from Syndicate, along with some sound effects- for the levels I decided to use real cities, suing imagery from Bing Maps (which, to my surprise, were actually much better than the Google maps photography). I built a very basic level editor using jQuery, knockout and EaselJS allowing me to define walkable and non-walkable regions, and also define where buildings were so my game could make the levels pseudo-3d, where your characters would appear to walk behind buildings and trees etc and produce JSON data which could be dropped into the main game. (You can have a play with that here, there’s not a great deal you can do with it to be honest, but it was a great help during development).

The game’s main mechanic, from a technical standpoint, is taking the flat 2d Bing Map and making it into a 3d city your guys can walk around. This is achieved by taking the buildings as defined in the level editor, then snipping those pieces of the map out and placing them onto a separate layer, then it’s just a matter of figuring out what order to draw everything in- should each little guy be in front of or behind each building- I think it’s quite effective at giving the illusion of depth, though could do with a little refining, given more time!

As a base for the game, unlike my last 8 week game where everything was written from scratch, I used the CreateJS set of libraries to underpin everything. This is a great set of open source libraries which covers a whole bunch of stuff but the parts I used were EaselJS for the canvas itself, SoundJS to handle html5 audio, and PreloadJS which handles the preloading of various assets. This made it a bit more of a level playing field with the XNA guys who have a lot of this great stuff built into the framework. I found create js to be a great library, abstracting out lots of repetitive bits of code which are boring to write, but while still letting you manipulate the canvas directly when you need to. (EaselJS in particular has some great bitmap caching features in there too which let you dynamically build up textures as you go which I put to good use with the blood splatters and bullet holes!)

Getting the guys to find their way round the city proved quite complex- I used Dijkstra’s algorithm for this which used my array of non-walkable regions built in the level editor as the node graph and calculates the shortest region. I actually ported some code from C to JavaScript for this- the original being written by Darel Finley and had to make some adjustments to allow it to read from my data structures. This also proved to be quite an expensive calculation which would freeze the browser for a moment so I employed Web Workers, which are new to HTML5 and effectively bring threading to JavaScript for the first time, which solved this issue.

Everything else was pretty straight forward- the interface is all built with HTML5/ CSS/ jQuery and utilises a class I wrote called layout manager which reshuffles all the little grey boxes to fit together when they all change size. I also reused my Mini Map class to generate the level previews you see on the briefing screen.

The competition is judged by voting on the Facebook event page which is open to the general public next week- 11th Feb 2013 onwards. You can check out the competition details and the other entries on the Facebook page found at www.facebook.com/events/165946213543847

No Comments