Sunday, April 13, 2014

Web Applications that test themselves

When it comes to test (mobile) web applications it's pretty much ends with using a WebDriver technology.
I might sound a bit disappointed but actually I'm not. It's a good technology that enables the developers to write tests that automate the browser from a separate controlling process. Those tests can act like a real user would and for sure it's a big contribution for the web applications' echo system. There are many pros using that kind of technology but still I can think we can do better.

What it is good for?

Let's think for a second that we actually can make the web applications tests themselves. Else than it's cool what it is good for?

  • For starter there's no need for any drivers installations. If the web applications will test themselves they are pretty much working like they are normally do, running in their clients. 
  • Another thing is about testing the actual application code and not its output. In that way we can really spot the real problems from the source code and externalize the real errors nature. Furthermore there are many changes in the application's source code that leads to massive DOM changes that makes it hard to spot, what is the error source and on what module we should be focused.
  • The developer is working on his application's source project, creating his test as part of his code, in that way he does not need to go to a separate environment creating another project and using different set of API's for creating his tests. 
  • You can say: "If I'm coding my tests in my application's source code, I'm messing up my code". We can take care of that, you might heard of comments and annotation technologies
  • Well, and how about recording my actions? In this kind of technology it gets a bit problematic, it's not impossible but still it's not there just yet. Any technology has it's cons and this might be one of them, for now.
It turns out that it might be a good technology after all. In my initial blog's post "Web Automation Testing Solution", I have introduced catjs framework, you can read all about it. In this section I want to explain the basic usage and a bit behind the scenes.

Let's see how we can make our web application to test itself.

Create the actual tests

First thing we need to code some tests and that we can achieve by adding annotation within comments. For example if we have a JQM based application with an input control and we want it to set a data into that input control we can add a comment like so:

The input control in our HTML file:
<input type="text" name="serach" id="search" value="">

The annotation snippet in our JavaScript file is:
      @@name searchTest
      @@jqm setText("search", "hello world");
      @@assert equal('search', 'hello world', 'search input !== "hello world"')

This is a simple example for you to understand the idea since catjs supports much complex situations.
Creating the test using annotation within comments is great, but for making the application to test itself  we need to create a test deployment, for that we are going to use catjs framework.

Install catjs

catjs framework was created with Node.js technology. If you are not familiar with Node.js just yet, install it first.
Node Packaged Module (NPM) is the official package manager for Node.js that is automatically bundled and installed with the Node.js environment.  After installing Node.js you'll get the 'npm' comand in your path.
Then in your terminal put the following command: npm -g install catjs

From that point you'll get to have "catcli" command in your path. with that command you can create your test deployment but first you'll need to create a catjs project.

Initial catjs project

catjs has a built-in initialization system for creating an initial project. For creating your first project, open your terminal and change your directory to where your parent application folder exists.

Now, run the following command: catcli -i
catjs will guide you through the project creation process, filling the required properties of your first project.

Most of the properties has defaults options, the most important property is the name of your project and the your application path. Since catjs has to be run from where its project is created ('cat-project' folder) you might want to put a relative path e.g. ./../[APP_FOLDER_NAME]

Creating "myapp" project

Creating your application test deployment

At this point you should have two main folders: 
  • Your Application's folder
  • cat-project folder
    • includes catproject.json (catjs's project) 
From your terminal change your directory to where your 'cat-project' folder exists and run the following command: catcli -b

Now, we have a new test deployment including all we need for the application to test itself.

Thats's it! Let the application test itself

Since the new test deployment includes the application with the test resources included, now we can let the application test itself. It means that all we have to do is deploy the new test deployment and run the application like the user does. Go to your device or browser and set the new deployment address.

CatJS Technical Details

CatJS Server

catjs has a built in Node.js server that will be automatically configured. Using the -s flag at the command line you'll be able to runt & deploy your test deployment on catjs server. 

Its a good practice to create your application with a standalone mode. It means that it could be running with fake data or test data, in such mode you can focus on testing your application user interface. catjs can then run your application ans test it with a simple command: catcli -bs

The process behind the scenes 

catjs has an internal Task manager (Grunt is supported) that can be addressed using the command line or running catjs as a module. (catjs Task manager can be customized, see the docs for more details)

When running the command: catcli -b , what catjs does is scanning your application resources, clone the application, scan the files and looking for its annotations. Interpret it, generates the tests functionality and pack all of the tests resources.

Hello world project

Part of catjs initialization system is to generate an example project.
When running the command: catcli --init example it generates a new project containing a demo application. The command also builds the  test deployment and run the server.

Creating "hello-world" project

Building "hello-world" project

Project's folder structure

In the following picture you can see the folder structure after initiating the example project:

  • CAT project folder
    • lib - Core library resources
    • src - The generated test project's sources
      • common - Test common sources
      • config - CAT configuration files
        • cat.json - The test project
        • testdata.json - The test data bundle (fake data)
    • target - The target folder is the actual new test deployment folder (Deploy the hello-world folder and see how the application test itself)
  • Configuration files
    • cat.json - This file contains the tests names to be actually tested. In case you have lots of test cases and you wish to run specific test scenarios, in this file you can configure what kind of test to run. (see the docs for more details)
    • testdata.json - This is the test data that you can use for your tests. You can put whatever data you like and use catjs API to address it.
  • CAT project
    • catproject.json - When running catjs command line interface (catcli) it performs actions according to the catproject.json configuration.


This is a new approach and methodology that can be embraced by those developers that wishes to really test their work while developing and maintain the quality from Day 1.

Next Post
On the next post we will deal with CAT runner that will make it much easier for running your application on multiple devices and browsers. 


  1. We have a dedicated in-house team for SEO, Web Design Company, Web development Company, Web developer, Web Designer and SEO Delhi that can create effective solutions to enhance brand.

    1. This comment has been removed by the author.

  2. This constantly amazes me just how blog owners such as your self can find the time as well as the dedication to keep on crafting superb blog posts. Your website is good and one of my personal must read weblogs. I just had to thank you.
    Professional Web Design Company

    1. Thank you for your kind words :)
      Well, it might sounds phony, but everything I do is really to make life easier and efficient, if it will help everybody it's even better. This and the fun of innovation and creation is the thing that drives me. Reading comments like yours makes me keep going. Thank you.

      BTW recently I wrote another article that was published in Method & Tools Magazine -

    2. This comment has been removed by the author.

  3. Very nice and helpful information has been given in this article. I like the way you explain the things. Keep posting. Thanks..
    Web Designing Companies

  4. This is Great wonderful information.. Great ideas Thanks for sharing..

    Web Application Development India

  5. Thanks for sharing such beneficial information with us. Your article is truly extremely agreeable while perusing and please continue sharing more.

    Web Application Development | JavaScript Error Fix | Website Fix

  6. Nice work and great style of presenting information about the Red Teaming it's a good work.
    Red Teaming