Posts Tagged timeout

Synchonous Yield messing up my foreach loop!

I recently got stung while debugging a colleagues code, by my lack of understanding of the Yield keyword! We had a foreach loop which looks like this;

foreach (PurchaseOrderLine aLine in PurchaseOrderLine.LoadExtractLines(extractTime, systemId)) {
  //do some processing stuff...
  PurchaseOrder.MarkAsExported(lastId, extractTime);
}

The LoadExtractLines returns an IEnumerable and uses yield to return each line- the stored proc it runs is quite intensive- the whole thing looks like this;

public static IEnumerable<PurchaseOrderLine> LoadExtractLines(DateTime cutOff,int systemId)
        {
            using (SqlConnection conn = new SqlConnection(SingleAccess.Instance.ConnectionToUse))
            {
                conn.Open();
                using (SqlCommand cmd = new SqlCommand("Get_PurchaseOrderLinesToExtract", conn))
                {
                    cmd.CommandType = CommandType.StoredProcedure;
                    cmd.Parameters.AddWithValue("@CutOff", cutOff.ToString("dd MMM yyyy HH:mm:ss"));
                    cmd.Parameters.AddWithValue("@SystemId", systemId);
                    using (SqlDataReader reader = cmd.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            yield return LoadLine(reader);
                        }
                    }
                }
            }
        }

MarkAsExported runs quite an intensive update procedure on the database. As more and more data came into the system we started to see Sql Timeouts, and upon running a trace noticed something strange. Logging the RPC:Start and Completed events of the stored procs, the Get_PurchaseOrderLinesToExtract proc which feed’s the for loop was starting, then the update was starting before the Get_ had finished- the two were running side by side, causing the timeout’s!

Turns out the foreach loop started the moment it received it’s first row, yielded back from the LoadExtractLine method- which in retrospect does make sense! The solution was to convert the Load method to populate a local List<> then return the whole thing once complete, removing the yield statement alltoghether and forcing the process to wait for the entire result set to be complete before starting the loop.

, ,

2 Comments

WordPress import from WordPress.org – 500 Server Errors & Timeout’s!

I’ve recently setup a wordpress blog to use when we’re on holiday to post pictures and let people know how we’re getting on. I had an existing hosted blog up at shawandtashinjapan.wordpress.com which I wanted to import to my new self hosted blog, but ran into a whole world of pain while trying to run the built in importer.

I ran an export from the wordpress.org hosted blog which gave me an xml file which contained wordpresses own variant of rss. I then selected import from my new wordpress’s tools menu and gave it the xml file- all went well but when i asked it to download the attachments it all started to go to shit!

There were two main issues which i had to fix- the first was wordpress goes mental when it hits a 302 redirect when trying to grab an image, but there is a fix available for the download function in wordpress;

From : http://en.forums.wordpress.com/topic/downloading-images-for-import?replies=4

1052c1052
< //ini_set("display_errors", true);
---
>
1061,1069c1061
< $response = wp_remote_request($url, $options);
< $headers = wp_remote_retrieve_headers($response);
<
< $cd = (string)$response['response']['code'];
<
< if ($cd == '301') {
< $response = wp_remote_request($headers['location'], $options);
< $headers = wp_remote_retrieve_headers($response);
< }
---
> $response = wp_remote_request($url, $options);
1074c1066
< //$headers = wp_remote_retrieve_headers( $response );
---
> $headers = wp_remote_retrieve_headers( $response );
1077d1068
<

Then i’m hit with timeouts coming from the FastCGI module- I’m running PHP 5.2 under IIS7 on Windows 2008 Web Edition- the error I was now getting when trying to run the import was;

The FastCGI process exceeded configured activity timeout

I found the fix for this one here: http://forums.iis.net/t/1076662.aspx. It was down to the FastCGI module timeout being hit, so it was just a matter of increasing that from the command line i executed;

%windir%\system32\inetsrv\appcmd set config -section:system.webServer/fastCgi /[fullPath='C:\php5\php-cgi.exe'].activityTimeout:600

In the end, I STILL didn’t get all the images over as some of them were quite large and the wordpress file server timed out while the script was trying to grab them, but i thought i would post this none the less, to save anyone else having to experience the evening i just had.

ergh…

, , ,

No Comments