End to End Testing with Protractor

Tweet about this on TwitterShare on LinkedIn15Share on Google+6Share on Reddit0Buffer this pageFlattr the authorEmail this to someonePrint this page

Getting Started with Protractor

In this post, we will see how to setup protractor and run a few E2E tests. First, We are going to scaffold an Angular application using Yeoman and then add Protractor tests to them (as of today, there are no E2E tests included in generator-angular, in future if they do get included, you can still follow this post).

Special thanks to my dear friend Ali Baig for helping me better understand Selenium and Webdriver.

So, Let’s get started.

Contents

Understanding Testing & Testing Scope

We test stuff because we do not want to end up building something like this,best-funny-pictures_safe-gunwhen we actually wanted to build this

p38_layout_840This is the same with testing software too. We don’t want to deliver something that was not promised like bugs, issues or “unexpected” features.

When you take an application or any piece of software, there are 2 levels of testing that is always needed.

    1. Unit Testing
    2. Integration or End to End Testing

In Unit testing all we do is check if a small entity of code does what it claims to do. Let’s take an example of simple string manipulation.

Requirement: Provided two strings need to be concatenated and the result needs to be capitalized

Code:

Unit Test

The above code is said to be working fine if individual pieces of code match their entry and exit criteria

entry_exit

This is all we do in Unit testing.

Integration Test 

Technically at this point we pretty much are sure that our function is going to spit out the expected value. But there may be a chance that the methods might not have been invoked properly or network issues (not in this example) or due to some “unknown reason”, there was an error. In that case the complete system will breakdown. Integration test checks that.

Generally integration tests are considered to be more costly than Unit tests, as they involve testing practically all layers of the application.

In this way with a combo of Unit Tests & Integration test, we can pretty much be sure that our code quality is ~99% (let’s leave 1% to ‘to err is human‘).

Understanding Testing Scope

Depending on your application, your budget, your QA standards, your timelines, you will need to pick the scope of testing for your application. What do I mean by scope?

Lets take a look at a simple Angular application. The application will consist of many layers right from client side to server side like

end2end

It is up to you on where you want to draw a line. You can simple test if

1. All the pages are loading fine, that would be testing controllers, views and routes

2. The filters are working fine, that would be testing controllers, views and filters

3. A service loads the data and populate a list of results, that would be testing services, controllers and views

and so on.

The simplest form of testing an Angular app is to test only the client side (unit tests). This make sense because as UI devs, we have built only the client side.

But if you want to be 99% sure that your app is working fine, you will test it end to end with the server and DB integration.

You can also run these test when new features are added or when the application is migrated from one environment to another. And the best of all, you know whom to blame when things do not work as expected.

What is Selenium and Selenium Webdriver?

 Selenium automates browsers

In simple terms Selenium has the power to open a browser you specify, navigate to a page and browse around. This kind of a framework is very powerful when you want to automate testing a web application.

Now, imagine if Selenium can not only open and close pages, but interact with them like a user does using a test script, Wouldn’t that be awesome? This is exactly what a Webdriver is for.

Webdriver is a simple API that wraps user interactions in simple methods. You can use the API to open a login page, fill in the credentials and submit the page. And then check if the user has successfully logged in.

You can read more about Selenium here and WebDriver here.

What is Protractor?

Protractor is an end to end test framework for AngularJS applications built on top of WebDriverJS. Protractor runs tests against your application running in a real browser, interacting with it as a user would.

Protractor is built on top of Webdriverjs, so you can do anything that can be done on Webdriverjs and more. Protractor has a simple config file that consists of the Selenium config, browser details, our spec file paths and a few jasmine options. A sample config file would look like

Protractor has a smart API, on top of Selenium Webdriver, that will help us in interacting with elements on pages easily.

For example:

or

Now, lets put all the above concepts to work and write a few tests.

Setting up an Angular App

We are going to use a Yeoman generator named generator-angular for scaffolding a simple app on which we will be configuring Protractor and running a few E2E tests.

First, create a new folder named myNgApp and then open terminal/prompt here. Then run

Windows Mac/*nix

npm install -g yo generator-angular 

  sudo npm install -g yo generator-angular

to install yeoman, angular generator and then run

yo angular

You will be asked a few questions and you can fill it up like

Yeoman will go ahead and scaffold a new project and its dependencies for you. Once the installation is done, if you do not see node_modules folder run npm i, and if you do not see app/bower_components folder run  npm i && bower i  to install the dependencies manually.

Then to run the app execute

grunt serve

This will launch the application in your default browser.

Our main code lies in app/scripts folder. The app.js consists of the Angular Module code and defines the routes. app/scripts/controllers folder consist of our controllers. app/index.html consists of the base page, with our ng-app and ng-view directives. app/views folder consists of views for each route.

Setup & Run Protractor

Moving on to the more exciting part, we will be setting up Protractor for our app.

First, let us install Protractor, run

npm i protractor --save-dev

We are going to set it up as local dev dependency. Once the setup is done, I recommend you checking out the installation at node_modules/protractor

Screen Shot 2014-05-25 at 7.26.23 pm

This folder consists of all the things you would need to run E2E tests. Do notice the protractor/bin folder, this consists of a webdriver-manager (– a simple wrapper to manage selenium) and the protractor task itself.

And, it ships with a well documented sample config file (protractor/referenceConf.js), that has all the available options. You can checkout the example and docs folder when you need some reference. But do not modify anything here.

You can setup Selenium server in 2 ways, and then run tests.

Approach 1 : Referring to an existing Selenium server

Now, lets create a new file at the root of myNgApp folder and name it protractor-e2e.js and fill it up like

This is our protractor config file.

Running protractor involves a 3 step process using this approach. In the current terminal/prompt, execute the below steps

Update & Start the Selenium server 

Step Windows Mac/*nix
Update Driver node node_modules/protractor/bin/webdriver-manager update    ./node_modules/protractor/bin/webdriver-manager update
Start Driver   node node_modules/protractor/bin/webdriver-manager start    ./node_modules/protractor/bin/webdriver-manager start

Updating the driver is more of a one time thing.
Run the application
Open a new terminal and start the application by running

grunt serve 

Run tests

Open a new terminal and run

Windows Mac/*nix
node /node_modules/protractor/bin/protractor protractor-e2e.js    ./node_modules/protractor/bin/protractor protractor-e2e.js

And you should see an error saying that

Spec patterns did not match any files

This means that we have successfully setup Protractor to run with running copy of Selenium.

Approach 2 : Running a standalone Selenium server

You can update the config file like

Note on line 12, we add the Selenium jar file and on line 16 we set the port.

If you have not yet run webdriver update, you can do so by running

Windows Mac/*nix
node node_modules/protractor/bin/webdriver-manager update    ./node_modules/protractor/bin/webdriver-manager update

Updating the driver is more of a one time thing.

Then make sure the app is running (by executing grunt serve ) and the run the tests by opening a new terminal and executing

Windows Mac/*nix
node node_modules/protractor/bin/protractor protractor-e2e.js     ./node_modules/protractor/bin/protractor protractor-e2e.js 

And you should see an error saying that

Spec patterns did not match any files

This means that we have successfully setup Protractor to run with a Standalone version of Selenium.

You can pick any one of the above approaches as per your convenience. For this post, we will follow the second approach.

Now, let us setup the tests. Create a new folder named e2e inside the test folder. Create a file named app-spec.js inside the e2e folder where we will be writing the tests.

Open app-spec.js and add the below specs.

A simple test to check, if there is no hash in the URL, the app redirects to the home page. The tests are written in Jasmine. You can check out their site for syntax.

browser is exposed by Protractor globally. In the above test, we are navigating to index.html page. Remember the baseUrl config in protractor-e2e.js? This is how Selenium knows where to find index.html. Internally,  browser.get('index.html'); is same as  browser.get('http://localhost:9000/index.html');.

Next, we use the getLocationAbsUrl() to fetch the path and check if the current page URL is actually the default path mentioned in /app/scripts/app.js. 

To run this test, back to terminal/prompt and run

Windows Mac/*nix
node node_modules/protractor/bin/protractor protractor-e2e.js    ./node_modules/protractor/bin/protractor protractor-e2e.js

And you should see chrome launch and the test will get executed. And you should see a log like

Screen Shot 2014-05-25 at 9.35.04 pm

PS: I am using safari because of this defect in Chrome 36.

Simple and easy. Let us add a couple more tests. Open the app-spec.js and update as below

The second test is going to check if the appName on the top left corner of the page is same as “myNgApp”. Using the element(by.css()), we select the element and then getText() (getText() returns a promise). Then we assert if the returned text is same as what it is expected to be.

The last test will check if the menu items on the top right corner of the page sums up to 3.

You can run the tests by executing

Windows Mac/*nix
node node_modules/protractor/bin/protractor protractor-e2e.js    ./node_modules/protractor/bin/protractor protractor-e2e.js

and you should see

Screen Shot 2014-05-25 at 10.35.39 pm

Add a Grunt Task to Run tests

There are a couple of Grunt task runners for protractor like grunt-protractor-runner and grunt-protractor-webdriver. You can also checkout this post to setup Grunt with Yeoman. But I somehow did not find much difference between the approach we followed till now and the one we do when we setup a Task.

Add a Login feature and Test it

We will use the generator-angular’s sub-generator to scaffold a new route. Back to terminal/prompt and run

yo angular:route login

This will scaffold us a View, a Controller and update our routes. To test this, run

grunt serve

And then navigate to  http://127.0.0.1:9000/#/login and you should a simple message. Now, let’s add some login functionality. Open views/login.html and add the below code

Then open the controllers/login.js  and add the below code, that will take care of validation

Thats it. Our simple login form is ready.

PS : I am not validating things like valid email, valid password. I am just checking if the email is [email protected] & password is 1234. If this is true, I set a scope variable named isLoggedIn to true, else false.

Back to the browser and you should see

Screen Shot 2014-05-26 at 2.59.21 amFill in the username and password (1234) and you should see a message like

Screen Shot 2014-05-26 at 2.59.27 am

Now, lets test this feature. Create a new file named login-spec.js inside the test/e2e folder and add the below code

We have added the below test

    1. Redirect to /login when location hash is login
    2. Show the signin panel on page load
    3. Should not authenticate a user when the credentials do not match
    4. Should successfully authenticate a user when the credentials match

To run the tests, execute

Windows Mac/*nix
node node_modules/protractor/bin/protractor protractor-e2e.js    ./node_modules/protractor/bin/protractor protractor-e2e.js

You should see

Screen Shot 2014-05-26 at 3.29.03 am

Simple and easy.

Protractor and Sauce labs

Finally, we will run the same tests on Sauce Labs. If you have not created an account yet, create one here. You get a super awesome free starter package, just enough to give you a feel of Sauce Labs.

For us to test our app on Sauce Labs, we need to either host our app on a public domain or else, you can install Sauce Labs connect. For this example, we will be hosting the app on Github and then test it. You can access the same app we have built here.

Now, to run our tests on Sauce Labs, all you need to do is update the config with your Sauce Labs User & Key details. To get your SauceKey, navigate to your accounts page and on the bottom left hand side you will have the key. And the User is your Username.

The updated config file would be (fill in your sauceUser and sauceKey)

Do notice that I have updated the baseUrl to point to GitHub. And since our app is have a context (ngTestApp) we need to update our beforeEach of login-spec.js to

Thats all, now all you need to is run the test from your terminal/prompt

Windows Mac/*nix
node node_modules/protractor/bin/protractor protractor-e2e.js    ./node_modules/protractor/bin/protractor protractor-e2e.js

You can see the logs rolling. Simultaneously you can also checkout Sauce Labs tests page to see the live execution

Screen Shot 2014-05-26 at 3.59.38 am

Screen Shot 2014-05-26 at 3.59.57 am

Screen Shot 2014-05-26 at 4.00.59 am

Thats it. Hope this post gave you an idea on how to get started with Protractor and run a few End to End test.


Thanks for reading! Do comment.
@arvindr21

Tweet about this on TwitterShare on LinkedIn15Share on Google+6Share on Reddit0Buffer this pageFlattr the authorEmail this to someonePrint this page
  • vasavi

    Hi, I have a case where I want to change the defaultTimeoutInterval and allScriptsTimeout dynamically from the script to run that particular script for longer time. Can you suggest any solution?

  • Nikhil

    I am working with grunt a protractor to automate the test for that I have written protractor task in grunt.js file.
    But while running this task unable to pass the command line argument fro grunt task to app-speck.js method.

    I have getcookie(username,password) function of javascript in getcookies.js file.

    ‘use strict';
    var getcookie = function(username,password){
    //body of the getcookie method.
    };

    I am calling this function using grunt for that I have written task in grunt.js file.

    module.exports = function(grunt) {

    require(‘matchdep’).filterDev(‘grunt-*’).forEach(grunt.loadNpmTasks);
    grunt.initConfig({

    pkg: grunt.file.readJSON(‘package.json’),

    protractor: {

    options: {
    keepAlive: false,
    configFile: “./config/protractorConf.js”
    },
    getcookie: {
    options: {
    args: {
    specs: [‘getcookie.js’]

    }

    }
    }

    }
    });

    //Note: Create task and add it to register task. In order to run single task comment all other tasks and run
    grunt.registerTask(‘getcookie’, [‘protractor:getcookie’]);
    };
    I want to pass the arguments username and password to the getcookie function from the grunt file. How can I achieve this?

  • Kapil Chokhawala

    I have been working with Selenium Webdriver for creating a Keyword based framework (separate file for object locators, test case in excel file, creating reports using log4j., etc).I like the clarity in managing objects, TCs/Scenarios, reports, etc. in separate files. It also helps, as anyone else who is non tech., can write TCs.

    My question here is, can I use Protractor with my existing java based Selenium webdriver framework ? Will it add value ?

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Hello Kapil, I don’t think I am the right person to answer that question. The tutorial you see is purely from testing an Angular app perspective and nothing more. You can check with Selenium forums or create a question on Stack overflow. ~Thanks.

  • Yokesh Varadhan

    i am using mac system and i get this type of err when i run my test
    [launcher] Process exited with error code 1

  • Luis Troya

    Hi. How I leave the batch open when the testing had finished? It closed after finish e2e testings and I cannot see the results

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Hello Luis, Generally from the console where you have launched the test case, you should see the results there itself. Is it not happening that way?

      • Luis Troya

        I found the mistake I had. I was using the angular-seed repo to testIng protractor. U was executing the e2e-test.sh file, it works fine but it close so quickly and dont let me see the results well. The problem was I executed the file as a normal file, when I opened in from my console it didn’t close.

        Thanks for the article. It was very usefuk to help me to understand more about testing.

        • http://thejackalofjavascript.com/ Arvind Ravulavaru

          Glad I could be of help! Thanks.

  • David

    Hi, I’m using the second approach to configure protractor using the option seleniumServerJar but I’m getting the following error on Ubuntu:

    David-HP ~/apps/sandbox/angular-protractor $ ./node_modules/protractor/bin/protractor protractor.conf.js

    Using the selenium server at http://localhost:4444/wd/hub

    [launcher] Running 1 instances of WebDriver

    ERROR – Unable to start a WebDriver session.

    /home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/selenium-webdriver/http/index.js:145

    callback(new Error(message));

    ^

    Error: ECONNREFUSED connect ECONNREFUSED

    at ClientRequest. (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/selenium-webdriver/http/index.js:145:16)

    at ClientRequest.EventEmitter.emit (events.js:95:17)

    at Socket.socketErrorListener (http.js:1547:9)

    at Socket.EventEmitter.emit (events.js:95:17)

    at net.js:441:14

    at process._tickCallback (node.js:415:13)

    ==== async task ====

    WebDriver.createSession()

    at Function.webdriver.WebDriver.acquireSession_ (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:149:22)

    at Function.webdriver.WebDriver.createSession (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/webdriver.js:123:30)

    at Builder.build (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/selenium-webdriver/builder.js:294:22)

    at DriverProvider.getNewDriver (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/lib/driverProviders/driverProvider.js:38:7)

    at Runner.createBrowser (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/lib/runner.js:180:37)

    at /home/david/apps/sandbox/angular-protractor/node_modules/protractor/lib/runner.js:246:21

    at _fulfilled (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/q/q.js:797:54)

    at self.promiseDispatch.done (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/q/q.js:826:30)

    at Promise.promise.promiseDispatch (/home/david/apps/sandbox/angular-protractor/node_modules/protractor/node_modules/q/q.js:759:13)

    [launcher] Process exited with error code 1

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Hello David, Can you try this solution : http://stackoverflow.com/a/27493806/1015046

      • David

        Thank you Arvind. I tried that solution but didn’t solve the problem

        • http://thejackalofjavascript.com/ Arvind Ravulavaru

          Oh! Can you try

          (as mentioned here http://stackoverflow.com/a/21124867/1015046 )

          • David

            Running “./node_modules/protractor/bin/webdriver-manager
            start” with and without the “–standalone” flag works, but then I have to open a new terminal window to run “./node_modules/protractor/bin/protractor
            protractor-e2e.js”. Is there a way to just run a command that starts the server and then run the test automatically?

          • http://thejackalofjavascript.com/ Arvind Ravulavaru

            You can probably have a shell script or a gulp shell command that will do the both for you in one command. Does that make sense?

          • David

            Yes, thank you Arvind

  • Kiry Chiang

    so useful. thanks for your sharing!

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Thanks!

  • mint

    thanks a lot brother. your tutorial is the only one i found that works.

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Thanks mint!

  • Anusha Nilapu

    Here e2e means testing the server code also?

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Hi Anusha, We do not test the server code per say, we just check if the server responds as expected when we request a resource.

      • Anusha Nilapu

        Thanks Aravind. Could you suggest which is best tool for node.js testing ?

        • http://thejackalofjavascript.com/ Arvind Ravulavaru

          Hi Anusha, What exactly are you planning to test in your Node.js project?

          To test units of node code, you can use node unit. If you are planning to test a Node based web application, you can use Karma + Jasmine to do that.

          Thanks.

          • Anusha Nilapu

            Thank you Aravind. I have to test nod code so i will going to use node unit.

          • http://thejackalofjavascript.com/ Arvind Ravulavaru

            Great. Thanks.

  • https://geshan.blogspot.com/ Geshan Manandhar

    did you face any timeout issues with saucelabs?

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      Not really. Is there a specific issue?

  • Joons

    How do you structure your Protractor test vs your Karma test in your directory structure?

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      I generally have a test folder where I have e2e and unit tests folder and I maintain it in that way, if that is what you are asking.

  • psahni

    How to mock request. I want to test REST api, but i don’t want to hit actual Urls

  • Priyabrat

    Any pointers on how to handle the print dialog box. I needed to open the dialog and then able to cancel it to go back to the parent page. Does not seem to get the cancel part working even though the print dialog box is open. I am using mac os.

    • http://thejackalofjavascript.com/ Arvind Ravulavaru

      @priyabrat Interesting. How are you opening the print dialog box?

      • Priyabrat

        @Arvind in the app we are using $window.print to open the print dialog box. So in the spec.js i am clocking on the print button to open up the dialog box. However, i cant get to the cancel button on the opened overlay. One option is to press the escape key when the dialog box is opened, but i believe that is possible with Robot class in the java side not in the javascript since my test files are all js files. So any suggestions on this??

        • http://thejackalofjavascript.com/ Arvind Ravulavaru

          @disqus_yN3oTB2gGh:disqus I was thinking on the same lines of the escape key, but we need to have the control of the popup window to issue the ‘esc’ key stroke. So I don’t think this would work.

          I have raised an issue with the protractor folks here: https://github.com/angular/protractor/issues/937. Let us see what they propose.

        • http://thejackalofjavascript.com/ Arvind Ravulavaru

          I was thinking on the same lines of the escape key, but we need to have the control of the popup window to issue the ‘esc’ key stroke. So I don’t think this would work.

          I have raised an issue with the protractor folks here: https://github.com/angular/pro…. Let us see what they propose.

        • http://thejackalofjavascript.com/ Arvind Ravulavaru

          I was thinking on the same lines of the escape key, but we need to have the control of the popup window to issue the ‘esc’ key stroke. So I don’t think this would work.

          I have raised an issue with the protractor folks here: https://github.com/angular/protractor/issues/937 Let us see what they propose.