Category: test tools

A Lifehacker post pointed me to a useful Windows utility for quickly testing characters not found on the standard keyboard – CatchChar.

CatchChar runs in the system tray and is accessed via a keyboard combination. When activated, a popup menu appears with characters you have defined. This is a very quick way to access a set of characters when testing. It can be downloaded at http://helpingthings.com/index.php/insert-unicode-characters

The following two sites are great resources when you need test images:

lorempixum.com allows you to set a size, choose between color and grayscale, and select an image that meets those settings.
dummyimage.com also allows you to choose a size, text, background and foreground colors and three formats. A wide variety of pre-defined formats are also available.

I was recently researching various mock frameworks for Java. We settled on mockito. I will post more on mockito at another time.

As I was reading about JMock and EasyMock, I had a sense of deja vu. I came across the following report my team developed over 3 years ago. It appears that the report still appears correct:

During my day to day testing, I often need to generate large blocks of text. I often find the text of such public domain classics like Moby Dick or The Adventures of Tom Sawyer. Then there is the old stand by – Lorem Ipsum. The site www.lipsum.com is a useful site to generate different lengths of this text.

My new favorite site for this type of work is www.blindtextgenerator.com. Not only does it provide lorem ipsum it provide the following additional options: cicero (latin and english), Li Europan lingues (Occidental – an international auxiliary language and english), Far far away, Werther, Kafka, Pangrams, a-z A-Z 123.

Check it out.

A couple of weeks ago, I realized I needed to combine multiple testng-results.xml files into to do some consolidated reporting. I did some research and came up with a solution that works for me. Recently, the testng-users Google group had a thread that asked the same thing. This post explains how I did it – not necessarily the best way but works for me now.

Basically, I use an XSLT stylesheet to merge the various XML files into a single file. As you will see below, the solution I used only merges 2 files at a time so I create a loop that merges each file into a master file. Let’s start with the master ant task:


...
<taskdef resource="net/sf/antcontrib/antlib.xml"/>
...
<target name="merge-results">
    <delete dir="${report.dir}/testng_summary_report"/>
    <delete file="${report.dir}/start-merge.xml" />
    <delete file="${report.dir}/testng-merge.xml" />

    <mkdir dir="${report.dir}/testng_summary_report"/>
    <copy
        file="empty-testng-results.xml"
        tofile="${report.dir}/testng-merge.xml" />

    <for param="xmlFile">
        <path>
            <fileset dir="${report.dir}" >
                <include name="**/testng-results.xml" />
            </fileset>
        </path>


        <sequential>

            <copy
                file="${report.dir}/testng-merge.xml"
                toFile="${report.dir}/start-merge.xml" />
                <xslt style="testng-merge.xsl"
                    destdir="${report.dir}"
                    in="${report.dir}/start-merge.xml"
                    out="${report.dir}/testng-merge.xml">

                    <param name="with" expression="@{xmlFile}" />
                </xslt>
            <delete file="${report.dir}/start-merge.xml" />
        </sequential>
    </for>
</target>



After some initial cleanup, the ant task copies the "starting" XML file (empty-testng-results.xml). It looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<testng-results>
    <reporter-output>
    </reporter-output>
</testng-results>


The task then uses the antlib for command to loop through all of the testng-results.xml files and applies the XSLT transform to each in turn. I did not write the XSLT stylesheet. I found it at http://www2.informatik.hu-berlin.de/~obecker/XSLT/merge/merge.xslt.html. For convenience, I include it here:


<!--
   Merging two XML files
   Version 1.6
   LGPL (c) Oliver Becker, 2002-07-05
   obecker@informatik.hu-berlin.de
-->
<xslt:transform
    xmlns:xslt="http://www.w3.org/1999/XSL/Transform"
    xmlns:m="http://informatik.hu-berlin.de/merge"
    version="1.0"
    exclude-result-prefixes="m">

<!-- Normalize the contents of text, comment, and processing-instruction
     nodes before comparing?
     Default: yes -->
<xslt:param name="normalize" select="'yes'" />

<!-- Don't merge elements with this (qualified) name -->
<xslt:param name="dontmerge" />

<!-- If set to true, text nodes in file1 will be replaced -->
<xslt:param name="replace" select="false()" />

<!-- Variant 1: Source document looks like
     <?xml version="1.0"?>
     <merge xmlns="http://informatik.hu-berlin.de/merge">
        <file1>file1.xml</file1>
        <file2>file2.xml</file2>
     </merge>         
     The transformation sheet merges file1.xml and file2.xml.
-->
<xslt:template match="m:merge">
   <xslt:variable name="file1" select="string(m:file1)" />
   <xslt:variable name="file2" select="string(m:file2)" />
   <xslt:message>
      <xslt:text />Merging '<xslt:value-of select="$file1" />
      <xslt:text />' and '<xslt:value-of select="$file2" />'<xslt:text />
   </xslt:message>
   <xslt:if test="$file1='' or $file2=''">
      <xslt:message terminate="yes">
         <xslt:text>No files to merge specified</xslt:text>
      </xslt:message>
   </xslt:if>
   <xslt:call-template name="m:merge">
      <xslt:with-param name="nodes1" select="document($file1,/*)/node()" />
      <xslt:with-param name="nodes2" select="document($file2,/*)/node()" />
   </xslt:call-template>
</xslt:template>

<!-- Variant 2:
     The transformation sheet merges the source document with the
     document provided by the parameter "with".
-->
<xslt:param name="with" />

<xslt:template match="*">
   <xslt:message>
      <xslt:text />Merging input with '<xslt:value-of select="$with" />
      <xslt:text>'</xslt:text>
   </xslt:message>
   <xslt:if test="string($with)=''">
      <xslt:message terminate="yes">
         <xslt:text>No input file specified (parameter 'with')</xslt:text>
      </xslt:message>
   </xslt:if>

   <xslt:call-template name="m:merge">
      <xslt:with-param name="nodes1" select="/node()" />
      <xslt:with-param name="nodes2" select="document($with,/*)/node()" />
   </xslt:call-template>
</xslt:template>

<!-- ============================================================== -->

<!-- The "merge" template -->
<xslt:template name="m:merge">
   <xslt:param name="nodes1" />
   <xslt:param name="nodes2" />

   <xslt:choose>
      <!-- Is $nodes1 resp. $nodes2 empty? -->
      <xslt:when test="count($nodes1)=0">
         <xslt:copy-of select="$nodes2" />
      </xslt:when>
      <xslt:when test="count($nodes2)=0">
         <xslt:copy-of select="$nodes1" />
      </xslt:when>

      <xslt:otherwise>
         <!-- Split $nodes1 and $nodes2 -->
         <xslt:variable name="first1" select="$nodes1[1]" />
         <xslt:variable name="rest1" select="$nodes1[position()!=1]" />
         <xslt:variable name="first2" select="$nodes2[1]" />
         <xslt:variable name="rest2" select="$nodes2[position()!=1]" />
         <!-- Determine type of node $first1 -->
         <xslt:variable name="type1">
            <xslt:apply-templates mode="m:detect-type" select="$first1" />
         </xslt:variable>

         <!-- Compare $first1 and $first2 -->
         <xslt:variable name="diff-first">
            <xslt:call-template name="m:compare-nodes">
               <xslt:with-param name="node1" select="$first1" />
               <xslt:with-param name="node2" select="$first2" />
            </xslt:call-template>
         </xslt:variable>

         <xslt:choose>
            <!-- $first1 != $first2 -->
            <xslt:when test="$diff-first='!'">
               <!-- Compare $first1 and $rest2 -->
               <xslt:variable name="diff-rest">
                  <xslt:for-each select="$rest2">
                     <xslt:call-template name="m:compare-nodes">
                        <xslt:with-param name="node1" select="$first1" />
                        <xslt:with-param name="node2" select="." />
                     </xslt:call-template>
                  </xslt:for-each>
               </xslt:variable>
      
               <xslt:choose>
                  <!-- $first1 is in $rest2 and 
                       $first1 is *not* an empty text node  -->
                  <xslt:when test="contains($diff-rest,'=') and not($type1='text' and normalize-space($first1)='')">
                     <!-- determine position of $first1 in $nodes2
                          and copy all preceding nodes of $nodes2 -->
                     <xslt:variable name="pos" select="string-length(substring-before( $diff-rest,'=')) + 2" />
                     <xslt:copy-of select="$nodes2[position() &lt; $pos]" />
                     <!-- merge $first1 with its equivalent node -->
                     <xslt:choose>
                        <!-- Elements: merge -->
                        <xslt:when test="$type1='element'">
                           <xslt:element name="{name($first1)}" namespace="{namespace-uri($first1)}">
                              <xslt:copy-of select="$first1/namespace::*" />
                              <xslt:copy-of select="$first2/namespace::*" />
                              <xslt:copy-of select="$first1/@*" />
                              <xslt:call-template name="m:merge">
                                 <xslt:with-param name="nodes1" select="$first1/node()" />
                                 <xslt:with-param name="nodes2" select="$nodes2[position()=$pos]/node()" />
                              </xslt:call-template>
                           </xslt:element>
                        </xslt:when>
                        <!-- Other: copy -->
                        <xslt:otherwise>
                           <xslt:copy-of select="$first1" />
                        </xslt:otherwise>
                     </xslt:choose>
      
                     <!-- Merge $rest1 and rest of $nodes2 -->
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2[position() &gt; $pos]" />
                     </xslt:call-template>
                  </xslt:when>

                  <!-- $first1 is a text node and replace mode was
                       activated -->
                  <xslt:when test="$type1='text' and $replace">
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2" />
                     </xslt:call-template>
                  </xslt:when>

                  <!-- else: $first1 is not in $rest2 or
                       $first1 is an empty text node -->
                  <xslt:otherwise>
                     <xslt:copy-of select="$first1" />
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2" />
                     </xslt:call-template>
                  </xslt:otherwise>
               </xslt:choose>
            </xslt:when>

            <!-- else: $first1 = $first2 -->
            <xslt:otherwise>
               <xslt:choose>
                  <!-- Elements: merge -->
                  <xslt:when test="$type1='element'">
                     <xslt:element name="{name($first1)}" namespace="{namespace-uri($first1)}">
                        <xslt:copy-of select="$first1/namespace::*" />
                        <xslt:copy-of select="$first2/namespace::*" />
                        <xslt:copy-of select="$first1/@*" />
                        <xslt:call-template name="m:merge">
                           <xslt:with-param name="nodes1" select="$first1/node()" />
                           <xslt:with-param name="nodes2" select="$first2/node()" />
                        </xslt:call-template>
                     </xslt:element>
                  </xslt:when>
                  <!-- Other: copy -->
                  <xslt:otherwise>
                     <xslt:copy-of select="$first1" />
                  </xslt:otherwise>
               </xslt:choose>

               <!-- Merge $rest1 and $rest2 -->
               <xslt:call-template name="m:merge">
                  <xslt:with-param name="nodes1" select="$rest1" />
                  <xslt:with-param name="nodes2" select="$rest2" />
               </xslt:call-template>
            </xslt:otherwise>
         </xslt:choose>
      </xslt:otherwise>
   </xslt:choose>
</xslt:template>

<!-- Comparing single nodes: 
     if $node1 and $node2 are equivalent then the template creates a 
     text node "=" otherwise a text node "!" -->
<xslt:template name="m:compare-nodes">
   <xslt:param name="node1" />
   <xslt:param name="node2" />
   <xslt:variable name="type1">
      <xslt:apply-templates mode="m:detect-type" select="$node1" />
   </xslt:variable>
   <xslt:variable name="type2">
      <xslt:apply-templates mode="m:detect-type" select="$node2" />
   </xslt:variable>

   <xslt:choose>
      <!-- Are $node1 and $node2 element nodes with the same name? -->
      <xslt:when test="$type1='element' and $type2='element' and local-name($node1)=local-name($node2) and namespace-uri($node1)=namespace-uri($node2) and name($node1)!=$dontmerge and name($node2)!=$dontmerge">
         <!-- Comparing the attributes -->
         <xslt:variable name="diff-att">
            <!-- same number ... -->
            <xslt:if test="count($node1/@*)!=count($node2/@*)">.</xslt:if>
            <!-- ... and same name/content -->
            <xslt:for-each select="$node1/@*">
               <xslt:if test="not($node2/@* [local-name()=local-name(current()) and namespace-uri()=namespace-uri(current()) and .=current()])">.</xslt:if>
            </xslt:for-each>
         </xslt:variable>
         <xslt:choose>
            <xslt:when test="string-length($diff-att)!=0">!</xslt:when>
            <xslt:otherwise>=</xslt:otherwise>
         </xslt:choose>
      </xslt:when>

      <!-- Other nodes: test for the same type and content -->
      <xslt:when test="$type1!='element' and $type1=$type2 and name($node1)=name($node2) and ($node1=$node2 or ($normalize='yes' and normalize-space($node1)= normalize-space($node2)))">=</xslt:when>

      <!-- Otherwise: different node types or different name/content -->
      <xslt:otherwise>!</xslt:otherwise>
   </xslt:choose>
</xslt:template>

<!-- Type detection, thanks to M. H. Kay -->
<xslt:template match="*" mode="m:detect-type">element</xslt:template>
<xslt:template match="text()" mode="m:detect-type">text</xslt:template>
<xslt:template match="comment()" mode="m:detect-type">comment</xslt:template>
<xslt:template match="processing-instruction()" mode="m:detect-type">pi</xslt:template>

</xslt:transform>


At the end of this process, the merged results will be found in a file named testng-merge.xml.

One of the neat things about TestNG is the ability to annotate individual tests as belonging to zero or more groups. At execution time, it is possible to tell the test runner to run all tests in a particular group. In addition, tests in certain groups can be excluded so you can run all tests in a particular group except for those that are marked as broken.

The Apache Harmony project apparently uses TestNG. On their wiki, they describe their testing conventions for using groups. They have groups to identify tests that are operating system specific, processor architecture specific, feature specific, environment requirements, current state and test type. Worth checking out.