Archive for category .net

Executing long running process from a web site

I recently had to build a report which accepts one xls excel file, churns through a bunch the data looking stuff up on other systems and performing calculations, then spits out another xls excel sheet the other end. The whole process takes about 15 to 20 minutes to run and at present is a simple command line application which accepts a couple of parameters (input and output filenames) and I fire up manually- obviously a situation which is no good moving forward.

All our internal business systems are web forms/ mvc running from local IIS boxes so ideally I would like to add a screen where the user can submit their xls file, hit go, and then get the result back 15/ 20 minutes later- but I don’t want to tie up one of the threads in the aspnet worker thread pool for the entire duration. Some Googling revealed the best approach appears to be to write a windows service which hosts the long running process then get the web app to call across to that to kick off the task.

Implementation

This is to be a standard Windows Service which will run my long running process, hosting a WCF service which the website can use to call into it with. If you;ve not done this before, you can follow the steps on the MSDN ; msdn.microsoft.com/en-us/library/ms733069.aspx.

Create your WCF Service which will respond to requests from the website. In my instance I wanted to be able to start the operation, passing a byte[] containing the xls file, request status updates, then finally request the resulting xls file, again as a byte[]. For the sake of this example, the WCF service is the one which is doing all the work with regards spawning the worker thread and tracking updates from the actual worker class, and the worker class is a type called “Calculator”.

    using System.ServiceModel;
    [ServiceContract(Namespace = "http://Moneybarn.VIVS.FleetRevaluation")]
    public interface IMyWCFService
    {
        [OperationContract]
        string Start(byte[] excel_file, string user_name);

        [OperationContract]
        Progress GetProgress();

        [OperationContract]
        byte[] GetResult();
    }

    using System.Threading;
    public class MyWCFService: IMyWCFService
    {
        private static string CurrentUserInstance;
        private static Calculator MyCalculatorInstance;
        private static Progress LastUpdate;
        private static Thread WorkerThread;
        private static bool IsComplete;
        private static string LastError;
        private static byte[] FinishedFile;

        public string Start(byte[] excel_file, string user_name)
        {
            if (WorkerThread == null)
            {
                CurrentUserInstance = user_name;
                MyCalculatorInstance = new Calculator(excel_file);

                MyCalculatorInstance.OnProgress += new OnProgressEventHandler(MyCalculatorInstance_OnProgress);
                MyCalculatorInstance.OnComplete += new OnCompleteEventHandler(MyCalculatorInstance_OnComplete);
                MyCalculatorInstance.OnError += new OnErrorEventHandler(MyCalculatorInstance_OnError);

                WorkerThread = new Thread(new ThreadStart(MyCalculatorInstance.Start));
                WorkerThread.Start();

                return null; // no news is good news!
            }
            else
            {
                return "Instance already running for user " + CurrentUserInstance;
            }
        }

        public Progress GetProgress()
        {
            return LastUpdate;
        }

        public string GetError()
        {
            return LastError;
        }

        public byte[] GetResult()
        {
            if (IsComplete)
            {
                byte[] buff = FinishedFile;

                MyCalculatorInstance = null;
                WorkerThread = null;
                CurrentUserInstance = string.Empty;
                LastUpdate = null;
                FinishedFile = null;
                IsComplete = false;

                return buff;
            }
            else
                return null;
        }

        private void MyCalculatorInstance_OnError(string error)
        {
            LastError = error;
        }

        private void MyCalculatorInstance_OnComplete(byte[] the_file)
        {
            FinishedFile = the_file;
            IsComplete = true;
        }

        private void MyCalculatorInstance_OnProgress(int stage, float percent)
        {
            LastUpdate = new Progress() { 
                Stage = stage, 
                Percent = (int)(percent * 100) 
            };
        }
    }

Add a new class called ProjectInstaller- this will handle registering your service with Windows

    using System.ComponentModel;
    using System.Configuration.Install;
    using System.ServiceProcess;

    [RunInstaller(true)]
    public class ProjectInstaller : Installer
    {
        private ServiceProcessInstaller process;
        private ServiceInstaller service;

        public ProjectInstaller()
        {
            process = new ServiceProcessInstaller();
            process.Account = ServiceAccount.LocalSystem;
            service = new ServiceInstaller();
            service.ServiceName = "MyWindowsService";  // change this!
            Installers.Add(process);
            Installers.Add(service);
        }
    }

Add the actual service body; add a new “Windows Service” file to the project. This is the meat of the service, and contains no real logic- it’s only job is to fire up the ServiceHost for your WCF service.

    using System.ServiceModel;
    using System.ServiceProcess;

    public class MyWindowsService : ServiceBase
    {
        public ServiceHost serviceHost = null;

        public MyWindowsService()
        {
            // Name the Windows Service
            ServiceName = "MyWindowsService";
        }

        public static void Main()
        {
            ServiceBase.Run(new MyWindowsService());
        }

        // Start the Windows service.
        protected override void OnStart(string[] args)
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
            }

            // The type of your WCF service
            serviceHost = new ServiceHost(typeof(MyWCFService));

            serviceHost.Open();
        }

        protected override void OnStop()
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
                serviceHost = null;
            }
        }
    }

Finally there is a little bit of config to pop into the app.config for the WCF endpoints- I’ve created a custom binding here to allow me to receive larger files over the WCF call than the default limits allow (in this case, up to 10 meg);

  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="bigWSHttpBinding" 
                 maxBufferPoolSize="10485760" maxReceivedMessageSize="10485760"> <!-- 10 MB limit -->
          <readerQuotas maxDepth="10485760" maxStringContentLength="10485760" maxArrayLength="10485760" maxBytesPerRead="10485760" />
        </binding>
      </wsHttpBinding>
    </bindings>
    <services>
      <service behaviorConfiguration="MyWCFServiceBehavior"
        name="Demo.MyWCFService">
        <endpoint address="" binding="wsHttpBinding" bindingConfiguration="bigWSHttpBinding"
          contract="Demo.IMyWCFService" />
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="http://localhost:8000/MyWCFService/service" />
          </baseAddresses>
        </host>
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="MyWCFServiceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

I also created a couple of little batch files which I mark as “Copy to output directory” for installing and removing the service;

rem install.bat
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe
net start MyWindowsService
rem remove.bat
net stop MyWindowsService
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe /u

That’s the Windows service part built. You can now build, and jump to the bin output folder and run the install.bat file from the command line- with any luck (more likly with a little debugging!) you will be up and running. The second part is even easier;

Jump over to your web app, add a service reference to the url of the base address from your app config above then it’s just a matter of exposing those web service methods so that you can call them from your page with an ajax call- so in MVC I created a controller with Start, GetProgress & GetResult methods which returned JsonResults called straight from the client with jQuery.

2 Comments

Application timing out on live (under load) when making WCF Calls

Had an issue with a maintenance release of a system this week to UAT; It was a new WCF web service which supplies valuation data about vehicles- adapted an existing system to consume this service instead of it’s original local database- it worked fine in dev, but on UAT under load testing everything worked perfectly… for about 10 minutes!

As soon as the client system got under load from multiple connections we begun seeing timeout exceptions and the application just generally hanging, but only on pages which made WCF calls to the new service. After some perusal of the ELMAH error logs it became apparent that it was indeed our implementation of the WCF clients to blame- a quick peek into the code revealed that we were creating the WCF service proxy client, then executing the method we wanted, but then NOT calling the .Close method against the proxy! So the first 16 connections (the default number of connections allowed with the stock WCF config) were coming in and everything went fine, but after that all the connections were used up, just hanging there still open!

So… don’t forget to close you connections when you’re done- especially if you’ve come from using ASMX services where you didn’t have to worry about such things!

No Comments

Unexpected null reference exceptions when using NSubstitute

I recently got some unexpected null reference exceptions in a test I was writing using MSTest and NSubstitute; I was setting up my mock to return a new instance of an object with a bunch of parameters set- my code looked like this;

iv_client.GetValuation("G1|12345", 10000, 2008, "08").Returns(new WebServiceResponse()
{
    Result = false,
    ErrorEnum = IdResponseCode.NoDataFound
});

However when access the value of my GetValuation call in the code it was always null? The solution it seems is to define the result in it’s own variable and then return that, rather than define it inline- like so;

var valuation = new WebServiceResponse()
{
    Result = false,
    ErrorEnum = IdResponseCode.NoDataFound
};

iv_client.GetValuation("G1|12345", 10000, 2008, "08").Returns(valuation);

There’s probably a good reason for this… but I don’t know what it is! Anyone care to shed some light on this in the comments?

1 Comment

WCF Service with message based security validating against AspNet Membership Provider

This took me days of fiddling, but when I eventually sussed it was quite easy! WCF services have to be secured, and you have have a few options;

  • Message security
    The messages are encrypted
  • Transport security
    The connection is secured (SSL)
  • Message & Transport
    Both of the above

I want to use message based security (so no SSL) and I wanted AspNet Membership (as used in forms authentication by default) to manage the users allowed to use the service.

The key to message based security, is an X.509 certificate installed on the server. You can buy a cert from a trusted certificate authority (like Verisign) or you can make your own self signed cert. I went for the self signed certificate option, using the awesome SelfCert tool on the pluralsite blog as this is a system used internally- for an externally facing public service you’de probably want a proper certificate from a recognised certificate authority.

Drop the tool on your server, and fire it up- give it a common name (CN=) of whatever you want- for the sake of the example we’ll call it MyCert. I put the certificate in the “My” store.

Once that’s done you need to make sure the user account of whatever process hosts your WCF service has access to read the certificate. In this example, I’m running the WCF service in a web site hosted by IIS 7, in an app pool called “WCFDemo.shawson.co.uk”- so the identity is “IIS APPPool\wcfdemo.shawson.co.uk”. To assign permissions you can use another handy tool called winhttpcertcfg – once installed on your server, fire up the command line and run;

C:\Program Files (x86)\Windows Resource Kits\Tools>winhttpcertcfg -g -c LOCAL_MACHINE\My -s MyCert -a WCFDemo.shawson.co.uk

This will come back and tell you permissions have been assigned and everything is good. Ok so lets build the service.

I created a new MVC3 project (this doesn’t really matter) and added a WCF service to the project called TestService.svc. This had a single method called Hello which accepted a string “name” and returned a string. I setup the aspnet membership and added a single account called tester with a password of tester1.

Once thats all sorted, the critical bit is in the web config, which is where most of the WCF magic happens- so the server side config for the service looks like this;

 <system.serviceModel>
    
    <services>
      <!-- the service end point- this ties everything together for the service -->
      <service name="WCFCertificateTest.Services.TestService">
        <endpoint address="/"
                  binding="wsHttpBinding"
                  bindingConfiguration="MessageSecurity"
                  contract="WCFCertificateTest.Services.ITestService" />
      </service>
    </services>

    <bindings>
      <wsHttpBinding>
        <binding name="MessageSecurity" >

          <!-- specifies we cant message, not transport security -->
          <security mode="Message">
            <message clientCredentialType="UserName"/>
          </security>

        </binding>
      </wsHttpBinding>
    </bindings>
    
    <behaviors>
      <serviceBehaviors>
        
        <behavior name="">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceCredentials>
            
            <!-- this is the bit which tells WCF to use the ASPNet sql membership -->
            <userNameAuthentication
              userNamePasswordValidationMode="MembershipProvider"
              membershipProviderName="AspNetSqlMembershipProvider" /> 
            
            <!-- this next bit tells the server which certificate we should be using the encrypt the messages -->
            <serviceCertificate
              findValue="MyCert"
              storeLocation="LocalMachine"
              storeName="My"
              x509FindType="FindBySubjectName"/> 
            
          </serviceCredentials>
        </behavior>
        
      </serviceBehaviors>
    </behaviors>
    
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true"/>
  </system.serviceModel>

The server side is now ready. Make sure you run this on the same machine we created the certificate on earlier- fire it up!

Now for the client– this is the really easy bit.

For the purpose of this demo, I created a windows console app. Add the service reference.

Enter the url for your service, and select it, then enter a namespace for it (I chose TestService) then click ok. This will add the reference to your client app and write a bunch of stuff into the config. In my demo console app I got this;

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.serviceModel>
        <bindings>
            <wsHttpBinding>
                <binding name="WSHttpBinding_ITestService">
                    <security>
                        <message clientCredentialType="UserName" />
                    </security>
                </binding>
            </wsHttpBinding>
        </bindings>
        <client>
            <endpoint address="http://wcfdemo.shawson.co.uk/Services/TestService.svc"
                binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_ITestService"
                contract="TestService.ITestService" name="WSHttpBinding_ITestService">
                <identity>
                    <certificate encodedValue="AwAAAAEAAAAUAAAAyb+3RrSzF0j7Lm7TONYkIpPPJosgAAAAAQAAAK4EAAAwggSqMIICkqADAgECAhAwLsfZzjx....QGqKDw4P1sScwBANFBjdRSrKzNyVR7b5gDU3geiPsXvS3an6kxZ" />
                </identity>
            </endpoint>
        </client>
    </system.serviceModel>
</configuration>

The critical part of this is the big string of jibberish in the identity certificate. When you add the reference it goes to the server and grabs this string, which is unique to the certificate we created earlier. If you are running this from local, and later deploy to a production server with a new certificate (even if it has the same common name), make sure you update the service reference, as this string will be different!

We can now make the call from the application- The asp net membership cerdentials are passed over using the ClientCredential.Username element (lines 3-4). Because we’re using a self signed certificate not for a trusted root authority we add a line (line 5 in the example below) to change how dot net validates the certificate- basically telling it not to worry about verifying the certificate.

            TestService.TestServiceClient client = new TestService.TestServiceClient();

            client.ClientCredentials.UserName.UserName = "tester";
            client.ClientCredentials.UserName.Password = "tester1";
            client.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None;

            Console.WriteLine(client.Hello("Shaw"));
            Console.ReadLine();

And voila! A working WCF service using message based security. You can download the tools referenced above (just in case they become unavailable in the future) SelfCert & winhttpcertcfg

No Comments

MVC3/Webforms app using forms authentication – CSS & Images folder not accessible before login

This is very usually embarrassingly simple. You have a site which is locked down using forms authentication, but your login page needs the css and images- so you add settings to your web config telling the app only authenticated users can come in, but no one else can- then you add a couple of exceptions for your specific folders- like so;

<configuration>
	<system.web>
		<authentication mode="Forms" >
			<forms loginUrl="login.aspx" name=".ASPNETAUTH" protection="None" path="/" timeout="20" >
			</forms>
		</authentication>
<!-- This section denies access to all files in this application except for those that you have not explicitly specified by using another setting. -->
		<authorization>
			<deny users="?" /> 
		</authorization>
	</system.web>
<!-- This section gives the unauthenticated user access to the Default1.aspx page only. It is located in the same folder as this configuration file. -->
		<location path="default1.aspx">
		<system.web>
		<authorization>
			<allow users ="*" />
		</authorization>
		</system.web>
		</location>
<!-- This section gives the unauthenticated user access to all of the files that are stored in the Subdir1 folder.  -->
		<location path="subdir1">
		<system.web>
		<authorization>
			<allow users ="*" />
		</authorization>
		</system.web>
		</location>
</configuration>

This is documented on the MSDN on this article.

I just deployed an existing site to a new dev server and found everything ran fine, except the css and images would not load on the login page, despite me having the exception in the web.config.

After lots of Googling (most articles simply explain that you need the above exception’s in your config!) I found this little gem which explained I needed to;

  1. Open the IIS7.5 control panel
  2. Select the application
  3. double-click “Authentication”
  4. select “Anonymous Authentication”, then Edit
  5. change it to use the Application Pool Identity. Make sure that user has permissions on the folder that contains the site

No Comments

Switching from ObjectContext to DbContext in your Entity Framework EDMX File

Tres easy and much cleaner classes turfed out the other end;

http://blogs.msdn.com/b/adonet/archive/2011/03/15/ef-4-1-model-amp-database-first-walkthrough.aspx

2 Comments

Import & Export break points in Visual Studio 2010

“This will change your life”

… reassured my colleague Mr Carmicheal.

Basically it’s possible in VS2010 (Pro, Premium or Ultimate) to Import & Export sets of break points for projects, handy if you have pieces of code you often have to step through for some reason or another.

The original article can be found here: http://msdn.microsoft.com/en-us/library/dd293657.aspx

In case that page goes AWOL as MSDN pages often do, this is what it said;

To export all breakpoints that match the current search criteria

  1. In the Breakpoints window toolbar, click the Export all breakpoints matching current search criteria icon.

    The Save As dialog box appears.

  2. In the Save As dialog box, type a name in the File name box.

    This is the name of the XML file that will contain the exported breakpoints.

  3. Note the folder path shown at the top of the dialog box. To save the XML file to a different location, change the folder path shown in that box, or click Browse Folders to browse for a new location.
  4. Click Save.

To export selected breakpoints

  1. In the Breakpoints window, select the breakpoints you want to export.

    To select multiple breakpoints, hold down the CTRL key and click additional breakpoints.

  2. Right-click in the breakpoints list, and choose Export selected.

    The Save As dialog box appears.

  3. In the Save As dialog box, type a name in the File name box.

    This is the name of the XML file that will contain the exported breakpoints.

  4. The folder path is shown at the top of the dialog box. To save the XML file to a different location, change the folder path shown in that box, or click Browse Folders to browse for a new location.
  5. Click Save.

To import breakpoints

  1. In the Breakpoints window toolbar, click the Import breakpoints from a file icon.

    The Open dialog box appears.

  2. In the Open dialog box, browse to the directory where your file is located, and then type the file name or select the file from the file list.
  3. Click OK.

1 Comment

Conditional logic in ListView ItemTemplate with DataBinder.Eval

Embarrasingly simple, but something I always seem to forget, and then never blog! I needed to show a tick in a repeater when a row value was -1: Simple! In MVC, with the razor syntax and strongly typed views I’ve grown to love, this is a pinch o’ the proverbial piss- In WebForms rreviously I often resort to setting up an ItemDataBound event handler which hides or shows an image, but to be honest I could not be bothered- this seems such overkill for something as simple as this. Anyway, after some faffing I reminded myself the easiest method is to use an ‘in-line if’ in the ItemTemplate, like so;

<ItemTemplate>
        <tr>
            <td><%# Eval("CarReg") %></td>
            ...
            <td>
                <%# (DataBinder.Eval(Container.DataItem, "NewRepeat") != null && DataBinder.Eval(Container.DataItem, "NewRepeat").Equals(-1) ? "<img src=\"../images/tick.png\" alt=\"tick\" />" : "<img src=\"../images/cross.png\" alt=\"cross\" /")%>
            </td>

No Comments

Grabbing data from the registry on a remote machine using dot net and WMI

I recently had to write an app which, given a computer name, would grab their telephone extension from a registry key (For the purpose of the code example below, I just grab the CommonFilesDir key from the windows node. There’s a couple of ways you can achieve this- either using RegistryKey.OpenRemoteBaseKey which gave me a whole bunch of permissions issues, plus you need the remote registry and remote administration services active on the server/client OR you can use the Windows Management Instrumentation (WMI) service. I took this route, as it’s a service which is active by default on all of our machines on the domain, and it was easier to get the permissions right.

Before I paste the code, a GOTCHA to beware of, straight from the MSDN here;

The registry provider is hosted in LocalService—not the LocalSystem. Therefore, obtaining information remotely from the subtree HKEY_CURRENT_USER is not possible

ConnectionOptions options = new ConnectionOptions();
options.Impersonation = ImpersonationLevel.Impersonate;
options.EnablePrivileges = true;
options.Username = "<domain_admin_username>";
options.Password = "<domain_admin_password>";

// http://msdn.microsoft.com/en-us/library/system.management.managementscope.aspx
ManagementScope ms = new ManagementScope("\\\\<computer name>\\root\\default", options);          
   
ms.Connect();

// http://msdn.microsoft.com/en-us/library/windows/desktop/aa390788(v=vs.85).aspx
ManagementClass mc = new ManagementClass("stdRegProv");
mc.Scope = ms;

ManagementBaseObject inParams = mc.GetMethodParameters("GetStringValue"); 
// there are other methods for grabbing other reg types- see 
// http://msdn.microsoft.com/en-us/library/windows/desktop/aa393664(v=VS.85).aspx

inParams["hDefKey"] = RegHive.HKEY_LOCAL_MACHINE;
inParams["sSubKeyName"] = "SOFTWARE\\Microsoft\\Windows\\CurrentVersion";
inParams["sValueName"] = "CommonFilesDir";

ManagementBaseObject outParams =
        mc.InvokeMethod("GetStringValue", inParams, null);

if (outParams["ReturnValue"].ToString() == "0")
{
    Console.WriteLine(outParams["sValue"]);
}
else
{
    Console.WriteLine("Error " + outParams["ReturnValue"] + " please refer to http://msdn.microsoft.com/en-us/library/ms681382%28v=3Dvs.85%29.aspx");
}

Console.ReadLine();

This code requires this enum..

public enum RegHive : uint
{
  HKEY_CLASSES_ROOT = 0x80000000,
  HKEY_CURRENT_USER = 0x80000001,
  HKEY_LOCAL_MACHINE = 0x80000002,
  HKEY_USERS = 0x80000003,
  HKEY_CURRENT_CONFIG = 0x80000005
}

If you receive a COM error back “The RPC server is unavailable”, make sure the target machine’s firewall isn’t blocking the call- this will open the port;
netsh firewall set service RemoteAdmin

No Comments

IIS Express hanging when launched from visual studio

I recently built up a new machine at work and grabbed all my latest projects out of SVN only to find that one of them no longer seemed to start.

Since checkout I hadn’t changed any code, got a clean build, but when the browser window opened it just hung, with no error.

I eventually found this was because I was using the UrlRewrite module on the site (with the rules in the web.config file) but had forgotten to install this component on my IIS on the new machine. An error message would have been nice!

No Comments