Friday, December 21, 2012

WaitN and HTTPWatch – Latency Measurements using Automated UI Testing

I wanted the measure the latency of a website from different geo locations. Some of the calls were made by JavaScript and hence UI actions were required instead of a simple playback of HTTP requests.
So there were two major goals
  1. Automating the UI actions
  2. Capturing the HTTP traces for these actions
There are a lot of options available for UI automation but it was important that the framework I chose is compatible with the toll I use to capture the HTTP traces.
 
So the first task was to determine which tool to be used for capturing the traces. The tool needed to have the ability of starting the trace. Stopping the trace and saving the trace in an automated fashion. HTTPWatch comes to rescue here. The basic edition is available for free but you will need to buy a license for advanced operations. See http://www.httpwatch.com/
 
Now, I had to determine an UI automation framework which works seamlessly with HTTPWatch. WaitN (pronounced as What-In) can be used with HTTPWatch. There is a sample available on the HTTPWatch site, see http://blog.httpwatch.com/2008/10/30/using-httpwatch-with-watin/
 
I wrote a console app using WatiN and HTTPWatch and could run it from different geo locations to collect the results. Sweet!

Wednesday, December 19, 2012

SQL / SSRS - Selecting 95% and 99% (percentile) values

I was trying to generate a SSRS report where I wanted to display the 95% and 99% (percentile) values to remove the outliers.
Unfortunately there isn't a percentile function in SSRS unlike excel.

So I had to use SQL to derive these values.

To derive 95% value, select top 5% by ordering the counters of the instance in desc order. Now order these 5% in ascending order and select top 1
Select @95P = (select top 1 t.coulmnname from (select top 5 percent coulmnname from tablename order by columnname desc)t order by t.coulmnname asc)

To derive 99% value, select top 1% by ordering the counters of the instance in desc order. Now order these 1% in ascending order and select top 1
Select @99P = (select top 1 t.coulmnname from (select top 1 percent coulmnname from tablename order by columnname desc)t order by t.coulmnname asc)

Thursday, December 6, 2012

Fiddler - Simulate Modem Speeds


There is an easy trick in Fiddler to simulate modem speeds. You can go to Rules->Performance->Simulate Modem Speeds to enable it.
It can also be customized by going to Rules->Customize Rules
This will open CustomRules.js where you will find
if (m_SimulateModem) {
            // Delay sends by 300ms per KB uploaded.
            oSession["request-trickle-delay"] = "300";
            // Delay receives by 150ms per KB downloaded.
            oSession["response-trickle-delay"] = "150";
        }
The values for request-trickle-delay and response-trickle-delay can be modified as per your need.

Now make a request in browser to experience the delay

Monday, November 26, 2012

VS 2010 Test Controller / SQL Server 2012 - Could not find stored procedure 'sp_dboption'

While configuring VS 2010 Test Controller with SQL Server 2012 as Load Test Repository, I was getting an error:

Microsoft.VisualStudio.TestTools.ConfigCore.TestControllerHelper.CreateAndUpgradeLoadTestSchemaIfRequired(String loadTestConnectionString, String directoryContainingSchemaFile)
E, 2012/11/26, 09:35:41.787, Microsoft.VisualStudio.TestTools.WebStress.LoadTestException: An error occurred while attempting to create the load test results repository schema: Could not find stored procedure 'sp_dboption'.
Could not find stored procedure 'sp_dboption'.


Found the same issue when I tried to create the Load Test Repository manually by executing the script C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\loadtestresultsrepository.sql

Then I looked for the corresponding VS 2012 script on a different machine at C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\loadtestresultsrepository.sql which worked.

Looking at the difference, the fix was to replace exec sp_dboption with ALTER DATABASE

Example:

Replace exec sp_dboption N'LoadTest2010', N'autoclose', N'false' with ALTER DATABASE [LoadTest2010] SET AUTO_CLOSE OFF

Tuesday, November 6, 2012

Selenium - Automating UI Testing for Websites using Visual Studio Unit Tests


Selenium is a well-known open source tool for UI automation of websites across a range of browsers. To know more, please refer http://seleniumhq.org/
A huge benefit of using Selenium is that you don’t need an interactive desktop session to run the automated UI tests.
Here we will discuss about integrating Selenium with Visual Studio Unit Tests written in C#
Test
·         Open Bing http://www.bing.com

·         Search for James Bond

·         Verify the results contain a Wikipedia link for James Bond
Steps
·         Download the IE Driver (needed only for running tests in IE) and the C# Client Drivers from http://seleniumhq.org/download/

·         Create a Unit Test Project in Visual Studio using File-New-Project->Test->Test Project

·         Add a folder called Dependencies and add the following files with Copy Always to Output:

o   WebDriver.dll

o   IEDriverServer.exe

o   Ionic.Zip.dll

o   Newtonsoft.Json.dll

·         Add a Reference to WebDriver.dll

·         Make sure Enable Deployment is checked in the Test Settings File, see http://msdn.microsoft.com/en-us/library/ms182475(v=vs.90).aspx

·         Import the following namespaces

using Microsoft.VisualStudio.TestTools.UnitTesting;

using OpenQA.Selenium;

using OpenQA.Selenium.IE;

using System.Drawing.Imaging;

·         Create a Test Class with DeploymentItem Attribute   
[TestClass]
[DeploymentItem("Dependencies\\IEDriverServer.exe")]
public class IETests

·         Create a web driver in Class Initialize and Quit in Class Cleanup
private static InternetExplorerDriver _webDriver;       

[ClassInitialize()]
public static void MyClassInitialize(TestContext testContext)
{
            _webDriver = new InternetExplorerDriver(new InternetExplorerOptions() {IntroduceInstabilityByIgnoringProtectedModeSettings = true});

       }
 
[ClassCleanup()]
public static void MyClassCleanup()
       {
            _webDriver.Quit();
       }
 
       private TestContext testContextInstance;    

       public TestContext TestContext
       {
            get
            {
                return testContextInstance;
            }
            set
            {
                testContextInstance = value;
            }
       } 

·         Create a Test Method
[TestMethod]
       public void TestBing()
       {
            // Test Case Steps
            _webDriver.Navigate().GoToUrl("http://www.bing.com");

            IWebElement search = _webDriver.FindElement(By.Name("q"));

            IWebElement go = _webDriver.FindElement(By.Name("go")); 

            search.SendKeys("james bond");

            go.Click(); 

            // Save screenshot if required
            Screenshot screenshot = _webDriver.GetScreenshot();

            screenshot.SaveAsFile("Result.png", ImageFormat.Png); 

            // Verfication
            IWebElement msWebsite =
                _webDriver.FindElement(
                    By.XPath(

                        "//a[@href='http://en.wikipedia.org/wiki/James_Bond']")); 

            Assert.IsNotNull(msWebsite, "Could not find wikipedia link for James Bond");
       } 

·         To test with Firefox, we can similarly create a Firefox web driver

private static FirefoxDriver _webDriver;

 Troubleshooting
If you see the error Class Initialization method Selenium.IETests.MyClassInitialize threw exception. System.InvalidOperationException: System.InvalidOperationException: Unexpected error launching Internet Explorer. Protected Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or disabled) for all zones. (NoSuchDriver).

This could be if you instantiated the IE Driver using new InternetExplorerDriver()
The fix is to either use new InternetExplorerDriver(new InternetExplorerOptions() IntroduceInstabilityByIgnoringProtectedModeSettings = true})

Or enable Protected Mode setting for all zones. Go to Tools->Internet Option->Security and check “Enable Protected Mode” for all zones.

Monday, October 29, 2012

Azure Table Storage - One of the request inputs is not valid

I was getting this error while getting the count of the total number of entities for a given Partition Key in a Azure Storage Table:

System.Data.Services.Client.DataServiceQueryException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="
http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
  <code>InvalidInput</code>
  <message xml:lang="en-US">One of the request inputs is not valid.RequestId:xxxTime:xxx</message>
</error>


This was occuring when I was trying to get the count like this:
int count = (from e in tableContext.CreateQuery<MyEntity>(_myTable)
                     where e.PartitionKey == interalId
                     select e).Count();


I was hoping for it to work since I was able to get the entire List using the same approach:
(from e in tableContext.CreateQuery<MyEntity>(_myTable)
                     where e.PartitionKey == interalId
                     select e).ToList();


However, I didn't want to first get the entire List and then take the count which would be lot of size overhead over the network. Finally I was able to get the count by first creating a query and then calling Execute:
CloudTableQuery<MyEntity> query = (from e in tableContext.CreateQuery<MyEntity>(_myTable)
                     where e.PartitionKey == interalId
                     select e).AsTableServiceQuery();
int count = query.Execute().Count();

Azure Table Storage - One of the request inputs is out of range

I was getting this error while saving an entity in Azure Table Storage using the method SaveChangesWithRetries() in Microsoft.WindowsAzure.StorageClient.TableServiceContext

System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="
http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
  <code>OutOfRangeInput</code>
  <message xml:lang="en-US">One of the request inputs is out of range.
RequestId:xxxTime:xxx</message>
</error>


I was setting the Primary Key and the Row Key correctly so wasn't sure about the root cause of the error.
Then I found that one of the properties of the entity was of Type DateTime which was not being set.
After I set the property to a value, say DateTime.Now, the entity got saved successfully.

Friday, October 26, 2012

VS 2010 – Generating a TRX file for a list of Tests


I came across a situation where I needed to generate a TRX file dynamically for a set of tests. This TRX would then to be used to rerun the set of tests.

To create a utility for this, I needed 3 pre-requisites:

·         List of test names

·         A TRX file template with one test

·         VSMDI file which has the test case details

The TRX template looks like this:

<?xml version="1.0" encoding="UTF-8"?>

<TestRun id="75ef9a27-b8ce-4c81-a2bb-fb3e26088e32" name="xxx" runUser="xxx" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">

          <TestSettings name="Local" id="3402c4f6-3b10-4472-9f2c-14a883c33075">

                   <Description>These are default test settings for a local test run.</Description>

                   <Deployment enabled="false" runDeploymentRoot="anujc_ANUJDEV1 2012-10-09 17_43_06">

                   </Deployment>

                   <Execution hostProcessPlatform="MSIL">

                             <TestTypeSpecific>

                                      <UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">

                                                <AssemblyResolution applicationBaseDirectory="xxx ">

                                                          <TestDirectory useLoadContext="true" />

                                                          <RuntimeResolution>

                                                                   <Directory path=" xxx " includeSubDirectories="true" />

                                                          </RuntimeResolution>

                                                </AssemblyResolution>

                                      </UnitTestRunConfig>

                             </TestTypeSpecific>

                             <AgentRule name="LocalMachineDefaultRole">

                             </AgentRule>

                   </Execution>

          </TestSettings>

          <Times creation="2012-10-09T17:43:06.8818053-07:00" queuing="2012-10-09T17:43:08.3979569-07:00" start="2012-10-09T17:43:08.5789750-07:00" finish="2012-10-09T18:01:43.5194579-07:00" />

          <ResultSummary outcome="Failed">

                   <Counters total="1" executed="1" passed="0" error="0" failed="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="1" inProgress="0" pending="0" />

          </ResultSummary>

          <TestDefinitions>

                   <UnitTest name="DefaultTestName" storage=" xxx">

                             <Description>DefaultDescription</Description>

                             <Execution id="e58026f9-1eb9-4f96-8fc9-e672dce1e6c7" />

                             <Properties>

                                      <Property>

                                                <Key>JobID</Key>

                                                <Value>133069</Value>

                                      </Property>

                             </Properties>

                             <TestMethod codeBase=" xxx" adapterTypeName="Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter, Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" className=" xxx, Federation4.5, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" name="DefaultTestName" />

                   </UnitTest>

          </TestDefinitions>

          <TestLists>

                   <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />

          </TestLists>

          <TestEntries>

                   <TestEntry testId="5099c33a-7e6b-4dff-8872-c0b6f614bbed" executionId="e58026f9-1eb9-4f96-8fc9-e672dce1e6c7" testListId="19431567-8539-422a-85d7-44ee4e166bda" />

          </TestEntries>

          <Results>

                   <UnitTestResult executionId="e58026f9-1eb9-4f96-8fc9-e672dce1e6c7" testId="5099c33a-7e6b-4dff-8872-c0b6f614bbed" testName="DefaultTestName" computerName=" xxx" duration="00:00:15.4887422" startTime="2012-10-09T17:43:08.7109882-07:00" endTime="2012-10-09T17:43:25.0076177-07:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Completed" testListId="19431567-8539-422a-85d7-44ee4e166bda" relativeResultsDirectory="e58026f9-1eb9-4f96-8fc9-e672dce1e6c7">

                             <Output>

                                      <StdOut>Fake Log</StdOut>

                             </Output>

                   </UnitTestResult>

          </Results>

</TestRun>

Now I created one method which reads the test properties from the VSMDI

static string GetTestcaseIdFromVsmdi(string testcase)

        {

            XmlDocument doc = new XmlDocument();

            doc.Load(oldVsmdi);

            XmlNamespaceManager namespaceManager = new XmlNamespaceManager(new NameTable());

            namespaceManager.AddNamespace("ns", "http://microsoft.com/schemas/VisualStudio/TeamTest/2010");

 

            XmlNode testLink = doc.SelectSingleNode("//ns:TestLink[@name='" + testcase + "']", namespaceManager);

            return testLink.Attributes["id"].Value;

        }

The below method creates a TRX for the list of test names

static void CreateNewTrx(List<string> selectedTests)

        {

            XmlDocument doc = new XmlDocument();

            doc.Load(oldTrx);

            XmlNamespaceManager namespaceManager = new XmlNamespaceManager(new NameTable());

            namespaceManager.AddNamespace("ns", "http://microsoft.com/schemas/VisualStudio/TeamTest/2010");

 

            XmlNode unitTest = doc.SelectSingleNode("//ns:UnitTest", namespaceManager);

            XmlNode testEntry = doc.SelectSingleNode("//ns:TestEntry", namespaceManager);

            XmlNode unitTestResult = doc.SelectSingleNode("//ns:UnitTestResult", namespaceManager);

 

            XmlNode counters = doc.SelectSingleNode("//ns:ResultSummary/ns:Counters", namespaceManager);

            counters.Attributes["total"].Value = selectedTests.Count.ToString();

            counters.Attributes["executed"].Value = selectedTests.Count.ToString();

            counters.Attributes["completed"].Value = selectedTests.Count.ToString();

 

            foreach (string selectedTest in selectedTests)

            {

                string testId = GetTestcaseIdFromVsmdi(selectedTest);

                string executionId = Guid.NewGuid().ToString();

                if (doc.SelectSingleNode("//ns:UnitTest[@name='DefaultTestName']", namespaceManager) != null)

                {

                    unitTest.Attributes["name"].Value = selectedTest;

                    unitTest.Attributes["id"].Value = testId;

                    unitTest.SelectSingleNode("ns:Execution", namespaceManager).Attributes["id"].Value = executionId;

                    unitTest.SelectSingleNode("ns:TestMethod", namespaceManager).Attributes["name"].Value = selectedTest;

                    testEntry.Attributes["testId"].Value = testId;

                    testEntry.Attributes["executionId"].Value = executionId;

                    unitTestResult.Attributes["executionId"].Value = executionId;

                    unitTestResult.Attributes["testName"].Value = selectedTest;

                    unitTestResult.Attributes["testId"].Value = testId;

                }

                else

                {

                    XmlNode newUnitTest = unitTest.Clone();

                    newUnitTest.Attributes["name"].Value = selectedTest;

                    newUnitTest.Attributes["id"].Value = testId;

                    newUnitTest.SelectSingleNode("ns:Execution", namespaceManager).Attributes["id"].Value = executionId;

                    newUnitTest.SelectSingleNode("ns:TestMethod", namespaceManager).Attributes["name"].Value = selectedTest;

                    unitTest.ParentNode.AppendChild(newUnitTest);

 

                    XmlNode newtestEntry = testEntry.Clone();

                    newtestEntry.Attributes["testId"].Value = testId;

                    newtestEntry.Attributes["executionId"].Value = executionId;

                    testEntry.ParentNode.AppendChild(newtestEntry);

 

                    XmlNode newUnitTestResult = unitTestResult.Clone();

                    newUnitTestResult.Attributes["executionId"].Value = executionId;

                    newUnitTestResult.Attributes["testName"].Value = selectedTest;

                    newUnitTestResult.Attributes["testId"].Value = testId;

                    unitTestResult.ParentNode.AppendChild(newUnitTestResult);

                }

            }

 

            doc.Save(newTrx);

        }

Now you are ready to run the tests from the TRX.

The only thing to note is that while running the tests from the Test Results window, don’t check any test. Just click Run and Enjoy!