Archive for category Web Services

Executing long running process from a web site

I recently had to build a report which accepts one xls excel file, churns through a bunch the data looking stuff up on other systems and performing calculations, then spits out another xls excel sheet the other end. The whole process takes about 15 to 20 minutes to run and at present is a simple command line application which accepts a couple of parameters (input and output filenames) and I fire up manually- obviously a situation which is no good moving forward.

All our internal business systems are web forms/ mvc running from local IIS boxes so ideally I would like to add a screen where the user can submit their xls file, hit go, and then get the result back 15/ 20 minutes later- but I don’t want to tie up one of the threads in the aspnet worker thread pool for the entire duration. Some Googling revealed the best approach appears to be to write a windows service which hosts the long running process then get the web app to call across to that to kick off the task.

Implementation

This is to be a standard Windows Service which will run my long running process, hosting a WCF service which the website can use to call into it with. If you;ve not done this before, you can follow the steps on the MSDN ; msdn.microsoft.com/en-us/library/ms733069.aspx.

Create your WCF Service which will respond to requests from the website. In my instance I wanted to be able to start the operation, passing a byte[] containing the xls file, request status updates, then finally request the resulting xls file, again as a byte[]. For the sake of this example, the WCF service is the one which is doing all the work with regards spawning the worker thread and tracking updates from the actual worker class, and the worker class is a type called “Calculator”.

    using System.ServiceModel;
    [ServiceContract(Namespace = "http://Moneybarn.VIVS.FleetRevaluation")]
    public interface IMyWCFService
    {
        [OperationContract]
        string Start(byte[] excel_file, string user_name);

        [OperationContract]
        Progress GetProgress();

        [OperationContract]
        byte[] GetResult();
    }

    using System.Threading;
    public class MyWCFService: IMyWCFService
    {
        private static string CurrentUserInstance;
        private static Calculator MyCalculatorInstance;
        private static Progress LastUpdate;
        private static Thread WorkerThread;
        private static bool IsComplete;
        private static string LastError;
        private static byte[] FinishedFile;

        public string Start(byte[] excel_file, string user_name)
        {
            if (WorkerThread == null)
            {
                CurrentUserInstance = user_name;
                MyCalculatorInstance = new Calculator(excel_file);

                MyCalculatorInstance.OnProgress += new OnProgressEventHandler(MyCalculatorInstance_OnProgress);
                MyCalculatorInstance.OnComplete += new OnCompleteEventHandler(MyCalculatorInstance_OnComplete);
                MyCalculatorInstance.OnError += new OnErrorEventHandler(MyCalculatorInstance_OnError);

                WorkerThread = new Thread(new ThreadStart(MyCalculatorInstance.Start));
                WorkerThread.Start();

                return null; // no news is good news!
            }
            else
            {
                return "Instance already running for user " + CurrentUserInstance;
            }
        }

        public Progress GetProgress()
        {
            return LastUpdate;
        }

        public string GetError()
        {
            return LastError;
        }

        public byte[] GetResult()
        {
            if (IsComplete)
            {
                byte[] buff = FinishedFile;

                MyCalculatorInstance = null;
                WorkerThread = null;
                CurrentUserInstance = string.Empty;
                LastUpdate = null;
                FinishedFile = null;
                IsComplete = false;

                return buff;
            }
            else
                return null;
        }

        private void MyCalculatorInstance_OnError(string error)
        {
            LastError = error;
        }

        private void MyCalculatorInstance_OnComplete(byte[] the_file)
        {
            FinishedFile = the_file;
            IsComplete = true;
        }

        private void MyCalculatorInstance_OnProgress(int stage, float percent)
        {
            LastUpdate = new Progress() { 
                Stage = stage, 
                Percent = (int)(percent * 100) 
            };
        }
    }

Add a new class called ProjectInstaller- this will handle registering your service with Windows

    using System.ComponentModel;
    using System.Configuration.Install;
    using System.ServiceProcess;

    [RunInstaller(true)]
    public class ProjectInstaller : Installer
    {
        private ServiceProcessInstaller process;
        private ServiceInstaller service;

        public ProjectInstaller()
        {
            process = new ServiceProcessInstaller();
            process.Account = ServiceAccount.LocalSystem;
            service = new ServiceInstaller();
            service.ServiceName = "MyWindowsService";  // change this!
            Installers.Add(process);
            Installers.Add(service);
        }
    }

Add the actual service body; add a new “Windows Service” file to the project. This is the meat of the service, and contains no real logic- it’s only job is to fire up the ServiceHost for your WCF service.

    using System.ServiceModel;
    using System.ServiceProcess;

    public class MyWindowsService : ServiceBase
    {
        public ServiceHost serviceHost = null;

        public MyWindowsService()
        {
            // Name the Windows Service
            ServiceName = "MyWindowsService";
        }

        public static void Main()
        {
            ServiceBase.Run(new MyWindowsService());
        }

        // Start the Windows service.
        protected override void OnStart(string[] args)
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
            }

            // The type of your WCF service
            serviceHost = new ServiceHost(typeof(MyWCFService));

            serviceHost.Open();
        }

        protected override void OnStop()
        {
            if (serviceHost != null)
            {
                serviceHost.Close();
                serviceHost = null;
            }
        }
    }

Finally there is a little bit of config to pop into the app.config for the WCF endpoints- I’ve created a custom binding here to allow me to receive larger files over the WCF call than the default limits allow (in this case, up to 10 meg);

  <system.serviceModel>
    <bindings>
      <wsHttpBinding>
        <binding name="bigWSHttpBinding" 
                 maxBufferPoolSize="10485760" maxReceivedMessageSize="10485760"> <!-- 10 MB limit -->
          <readerQuotas maxDepth="10485760" maxStringContentLength="10485760" maxArrayLength="10485760" maxBytesPerRead="10485760" />
        </binding>
      </wsHttpBinding>
    </bindings>
    <services>
      <service behaviorConfiguration="MyWCFServiceBehavior"
        name="Demo.MyWCFService">
        <endpoint address="" binding="wsHttpBinding" bindingConfiguration="bigWSHttpBinding"
          contract="Demo.IMyWCFService" />
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="http://localhost:8000/MyWCFService/service" />
          </baseAddresses>
        </host>
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="MyWCFServiceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="true"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

I also created a couple of little batch files which I mark as “Copy to output directory” for installing and removing the service;

rem install.bat
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe
net start MyWindowsService
rem remove.bat
net stop MyWindowsService
c:\windows\microsoft.net\Framework\v4.0.30319\InstallUtil MyWindowsService.exe /u

That’s the Windows service part built. You can now build, and jump to the bin output folder and run the install.bat file from the command line- with any luck (more likly with a little debugging!) you will be up and running. The second part is even easier;

Jump over to your web app, add a service reference to the url of the base address from your app config above then it’s just a matter of exposing those web service methods so that you can call them from your page with an ajax call- so in MVC I created a controller with Start, GetProgress & GetResult methods which returned JsonResults called straight from the client with jQuery.

2 Comments

WCF Service with message based security validating against AspNet Membership Provider

This took me days of fiddling, but when I eventually sussed it was quite easy! WCF services have to be secured, and you have have a few options;

  • Message security
    The messages are encrypted
  • Transport security
    The connection is secured (SSL)
  • Message & Transport
    Both of the above

I want to use message based security (so no SSL) and I wanted AspNet Membership (as used in forms authentication by default) to manage the users allowed to use the service.

The key to message based security, is an X.509 certificate installed on the server. You can buy a cert from a trusted certificate authority (like Verisign) or you can make your own self signed cert. I went for the self signed certificate option, using the awesome SelfCert tool on the pluralsite blog as this is a system used internally- for an externally facing public service you’de probably want a proper certificate from a recognised certificate authority.

Drop the tool on your server, and fire it up- give it a common name (CN=) of whatever you want- for the sake of the example we’ll call it MyCert. I put the certificate in the “My” store.

Once that’s done you need to make sure the user account of whatever process hosts your WCF service has access to read the certificate. In this example, I’m running the WCF service in a web site hosted by IIS 7, in an app pool called “WCFDemo.shawson.co.uk”- so the identity is “IIS APPPool\wcfdemo.shawson.co.uk”. To assign permissions you can use another handy tool called winhttpcertcfg – once installed on your server, fire up the command line and run;

C:\Program Files (x86)\Windows Resource Kits\Tools>winhttpcertcfg -g -c LOCAL_MACHINE\My -s MyCert -a WCFDemo.shawson.co.uk

This will come back and tell you permissions have been assigned and everything is good. Ok so lets build the service.

I created a new MVC3 project (this doesn’t really matter) and added a WCF service to the project called TestService.svc. This had a single method called Hello which accepted a string “name” and returned a string. I setup the aspnet membership and added a single account called tester with a password of tester1.

Once thats all sorted, the critical bit is in the web config, which is where most of the WCF magic happens- so the server side config for the service looks like this;

 <system.serviceModel>
    
    <services>
      <!-- the service end point- this ties everything together for the service -->
      <service name="WCFCertificateTest.Services.TestService">
        <endpoint address="/"
                  binding="wsHttpBinding"
                  bindingConfiguration="MessageSecurity"
                  contract="WCFCertificateTest.Services.ITestService" />
      </service>
    </services>

    <bindings>
      <wsHttpBinding>
        <binding name="MessageSecurity" >

          <!-- specifies we cant message, not transport security -->
          <security mode="Message">
            <message clientCredentialType="UserName"/>
          </security>

        </binding>
      </wsHttpBinding>
    </bindings>
    
    <behaviors>
      <serviceBehaviors>
        
        <behavior name="">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceCredentials>
            
            <!-- this is the bit which tells WCF to use the ASPNet sql membership -->
            <userNameAuthentication
              userNamePasswordValidationMode="MembershipProvider"
              membershipProviderName="AspNetSqlMembershipProvider" /> 
            
            <!-- this next bit tells the server which certificate we should be using the encrypt the messages -->
            <serviceCertificate
              findValue="MyCert"
              storeLocation="LocalMachine"
              storeName="My"
              x509FindType="FindBySubjectName"/> 
            
          </serviceCredentials>
        </behavior>
        
      </serviceBehaviors>
    </behaviors>
    
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true"/>
  </system.serviceModel>

The server side is now ready. Make sure you run this on the same machine we created the certificate on earlier- fire it up!

Now for the client– this is the really easy bit.

For the purpose of this demo, I created a windows console app. Add the service reference.

Enter the url for your service, and select it, then enter a namespace for it (I chose TestService) then click ok. This will add the reference to your client app and write a bunch of stuff into the config. In my demo console app I got this;

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.serviceModel>
        <bindings>
            <wsHttpBinding>
                <binding name="WSHttpBinding_ITestService">
                    <security>
                        <message clientCredentialType="UserName" />
                    </security>
                </binding>
            </wsHttpBinding>
        </bindings>
        <client>
            <endpoint address="http://wcfdemo.shawson.co.uk/Services/TestService.svc"
                binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_ITestService"
                contract="TestService.ITestService" name="WSHttpBinding_ITestService">
                <identity>
                    <certificate encodedValue="AwAAAAEAAAAUAAAAyb+3RrSzF0j7Lm7TONYkIpPPJosgAAAAAQAAAK4EAAAwggSqMIICkqADAgECAhAwLsfZzjx....QGqKDw4P1sScwBANFBjdRSrKzNyVR7b5gDU3geiPsXvS3an6kxZ" />
                </identity>
            </endpoint>
        </client>
    </system.serviceModel>
</configuration>

The critical part of this is the big string of jibberish in the identity certificate. When you add the reference it goes to the server and grabs this string, which is unique to the certificate we created earlier. If you are running this from local, and later deploy to a production server with a new certificate (even if it has the same common name), make sure you update the service reference, as this string will be different!

We can now make the call from the application- The asp net membership cerdentials are passed over using the ClientCredential.Username element (lines 3-4). Because we’re using a self signed certificate not for a trusted root authority we add a line (line 5 in the example below) to change how dot net validates the certificate- basically telling it not to worry about verifying the certificate.

            TestService.TestServiceClient client = new TestService.TestServiceClient();

            client.ClientCredentials.UserName.UserName = "tester";
            client.ClientCredentials.UserName.Password = "tester1";
            client.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None;

            Console.WriteLine(client.Hello("Shaw"));
            Console.ReadLine();

And voila! A working WCF service using message based security. You can download the tools referenced above (just in case they become unavailable in the future) SelfCert & winhttpcertcfg

No Comments

WCF Services with both JSON and SOAP endpoints

A quick example of how to set this up in the web.config.

Services, in this example, can be accessed

  • using JSON via localhost/Services/MyService.svc/HelloWorld or..
  • using SOAP on localhost/Services/MyService.svc/soap/HelloWorld
  <system.serviceModel>
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true" aspNetCompatibilityEnabled="true" />

    <services>
      <service behaviorConfiguration="WebServiceBehavior" name="Demo.Web.Services.MyService">
        <endpoint address="" binding="webHttpBinding" contract="Demo.Web.Services.IMyService" behaviorConfiguration="jsonBehavior" />
        <endpoint address="soap" binding="basicHttpBinding" contract="Demo.Web.Services.IMyService"/>
      </service>
    </services>
    
    <behaviors>
      <endpointBehaviors>
        <behavior name="jsonBehavior">
          <webHttp defaultOutgoingResponseFormat="Json" />
        </behavior>
      </endpointBehaviors>
      
      <serviceBehaviors>
        <behavior name="WebServiceBehavior">
          <serviceMetadata httpGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    
    <bindings>
      <webHttpBinding>
        <binding crossDomainScriptAccessEnabled="false" name="httpBinding" />
      </webHttpBinding>
    </bindings>
    
  </system.serviceModel>

No Comments

Twitter OAuth Registration

Any app’s you create now which need to post to twitter programmatically, will need to do so using oAuth, for which you will need to register your app with twitter- this can be done @ http://dev.twitter.com/apps/new Once registered you will be taken to a page with your consumer keys (a key value and a secret value)- to get your access token, click the button on the right of that page (a token and a secret) and you should now have all 4 values you need to perform oAuth authentication prior to interacting with twitter.

No Comments

MSDeploy Setup Guide & How to Call Server Side Method Using jQuery/Ajax

Couple of interesting items popped up in the RRS feed’s this morning.

Automating Deployment with Microsoft Web Deploy – ScottGu’s Blog.

Calling Server Side Method Using jQuery/Ajax – Karan’s Blog.

No Comments

Why ASMX web services are not an excuse anymore with WCF 4.0 – Pablo M. Cibraro (aka Cibrax)

Why ASMX web services are not an excuse anymore with WCF 4.0 – Pablo M. Cibraro (aka Cibrax).

No Comments

Getting started with WCF and REST web services

I’m building some REST WCF services at the moment and found this great guide online www.robbagby.com/rest/rest-in-wcf-blog-series-index

The HiREST stuff shows you how to create fully fledged REST services utilising all of the HTTP verbs.

Heres a quick summary of the most useful bits;

The service itself is just added to the project as a normal WCF service. The following needs adding to the Web.config to setup the end

points;

<system.serviceModel>
  <behaviors>
    <endpointBehaviors>
      <behavior name="AJAXFriendly">
        <enableWebScript />
      </behavior>
      <behavior name="RESTFriendly">
        <webHttp />
      </behavior>
    </endpointBehaviors>
</behaviors>

<services>
    <service      name="MyNamespace.AjaxServiceClassName">
      <endpoint   address=""
        behaviorConfiguration="AJAXFriendly"
        binding="webHttpBinding"
        contract="CatalogService" />
    </service>
    <service      name="MyNamespace.RESTServiceClassName">
      <endpoint   address=""
        behaviorConfiguration="RESTFriendly"
        binding="webHttpBinding"
        contract="MyNamespace.IRESTServiceInterfaceName" />
    </service>
  </services>
</system.serviceModel>

Remember the class names in the web.config need to be fully qualified.

An example class for passing around the service;

[DataContract]
public class ProductData
{
    [DataMember]
    public int ProductId;

    [DataMember]
    public string ProductName;

    [DataMember]
    public string Description;

    [DataMember]
    public decimal Price;

    [DataMember]
    public string ProductImage;
}

The service itself is just a normal WCF service so consists of its usual two parts- the interface and the class. Here’s an example interface

demoing the various access methods and return serialisations;

    [ServiceContract]
    public interface IRESTServiceInterfaceName
    {
        // responds to a GET request to www.mysite.com/Servicename.svc/products/
        // return is serialised to JSON  
        [OperationContract]
        [WebGet(UriTemplate = "products/", ResponseFormat = WebMessageFormat.Json)]
        List<FlotChartSeries> GetProducts();

        // responds to a GET request to www.mysite.com/Servicename.svc/products/{product_id}
        // return is serialised to JSON  
        [OperationContract]
        [WebGet(UriTemplate = "products/{product_id}", ResponseFormat = WebMessageFormat.Json)]
        List<FlotChartSeries> GetProductById(string product_id);

        // responds to a GET request to www.mysite.com/Servicename.svc/products?brand={brand}
        // return is serialised to JSON  
        [OperationContract]
        [WebGet(UriTemplate = "products?brand={brand}", ResponseFormat = WebMessageFormat.Json)]
        List<FlotChartSeries> GetProductsByBrand(string brand);

	// responds to a POST, but still passing a bunch of parameters into the querystring
	// no return type
        [WebInvoke(Method = "POST", UriTemplate = "product/{product_id}/rate?score={score}")]
        [OperationContract]
        void PostProductAppRating(string product_id, string score);

	// accepts a POST with a JSON encoded product data, to allow for an update
	// no return
	[WebInvoke(Method = "POST", UriTemplate = "product/{product_id}", RequestFormat = WebMessageFormat.Json)]
	void PostProductUpdate(string product_id, ProductData data);

	//responds to a PUT- passing a querystring variable for the filename- it expects the body of the push request to be a file stream
	// no return type
        [WebInvoke(Method = "PUT", UriTemplate = "products/{product_id}/desktopimage?filename={file_name}")]
        [OperationContract]
        void PutProductImage(string product_id, string file_name, Stream fileContents);
	
    }

Note the variables in the curly braces must match the real parameters to the method signitures for the value to be automatically mapped.

Responses are generally done using HTTP response codes. At the top of each method you need to grab the current WebOperationContext which

will allow you to set the return codes;

    WebOperationContext ctx = WebOperationContext.Current;

    ctx.OutgoingResponse.StatusCode = System.Net.HttpStatusCode.OK;		// .. if everything is ok!
    ctx.OutgoingResponse.StatusCode = System.Net.HttpStatusCode.BadRequest;	// on error
    ctx.OutgoingResponse.StatusCode = System.Net.HttpStatusCode.Created;	// after an insert

    // all of the http response codes are mapped here

    ctx.OutgoingResponse.SetStatusAsNotFound(); // if record cannot be found or similar

, ,

No Comments

Properties getting “k__BackingField” appended to their name in the WSDL file

I was finding k__backingField was being appended to all my object properties when exposed via my a WCF service, in the WSDL.  The solution it turns out was simple- just had to make my class a [DataContract] and mark the properties as [DataMember].

via WCF Data Contracts and “k__BackingField” Property Naming – Nathan Bridgewater.

No Comments

Making your WCF Service compatible with legacy .net 1.1 applications

I’m building an error tracking service which all our future web project will report to, so we can track and tag all our various systems problems from one place- this is currently done with email which is a bit of a nightmare!

According to Microsoft, traditional ASMX web services are now considered “Legacy technology” (!) so I thought I would buite the bullet and build the new services using WCF.

This was fairly painless until I tried to consume the web service in some old .net 1.1 web apps- when trying to add the web reference I received this error message;

Web ReferenceslocalhostReference.map(1): Custom tool warning: DiscoCodeGenerator unable to initialize code generator.  No code generated.

I found a great article over on the MSDN – and all it takes is a small change to the web.config, fiddling with, my old friend, the httpBindings.

I had to swap out the default bindings put in by .net;

        <endpoint address="" binding="wsHttpBinding" contract="HachetteErrorTracker.IErrorLog">
					<identity>
						<dns value="localhost"/>
					</identity>
				</endpoint>

For this one;

<endpoint
           address=""
           binding="basicHttpBinding" bindingNamespace="http://errortracker.localhost/"
           contract="HachetteErrorTracker.IErrorLog"
        />

How to: Configure WCF Service to Interoperate with ASP.NET Web Service Clients.

No Comments

The provided URI scheme ‘http’ is invalid; expected ‘https

Going from dev to live, where the dev system referenced dev version of web services, but the live system has to reference live versions which are HTTPS i recieved this error;

The provided URI scheme ‘http’ is invalid; expected ‘https

This is resolved simply by updating the web.config file and setting the security tag’s mode attribute from None to Transport;

<bindings>
<wsHttpBinding>
<binding name="WSHttpBinding_IWSHttpService" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
allowCookies="false">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="None" proxyCredentialType="None"
realm="" />
<message clientCredentialType="None" negotiateServiceCredential="true"
algorithmSuite="Default" establishSecurityContext="true" />
</security>
</binding>
</wsHttpBinding>
</bindings>

The provided URI scheme ‘http’ is invalid; expected ‘https.

8 Comments