End-to-End testing with codecept (mobile & web)

Introduction — picking the right tool

Finding a good framework is not always an easy task. Especially in javascript. There often many options. And always a new cool kid saying he is the best… This new framework will save your life!

Perhaps I’m doing too much. But it sometimes feels like it to me.
It’s like every tool and framework got a “features page”. But no “drawback/pitfalls page”. And when I pick a new tool know I’m supposed to gain from the get-go (it’s advertised). Yet I will have to discover by-myself those drawbacks. And it’s costly.

Yes, it costly. learning that framework, playing with it, deciding if will use it in production, coding some real-life application of it… Then and only then, I discover what issues I brought with that “cool” tool I choose…

So! All that to say that I made a good discovery lately. I will share gladly it with you. And tell you about some drawback I discovered.

what’s behind the trees ?

Codecept

https://codecept.io/ is an End-to-End tool. It’s an abstraction on top of various frameworks like appium, protractor, pupeteer, webdriver… This testing tool considers framework like protactor as a webdriver — Not as testing framework. And therefore focus on providing you a testing framework to allow to write things like this:

I.amOnPage("/login"); // visit url /login
I.fillField("username", "lsmod"); // fill input with name=password
I.fillField("password", "STRONGPASSWORD");
I.click("login"); // click on the button login (matching "login" text)
I.see("Hello lsmod!"); // wait some and check if we are logged

It looks simple. No .then() .catch(). No await, No retry neither. Definitely no browser, page, getElement and such.

Some of my favorites stuff that’s just working:

I.attachFile("form input[name=avatar]", "data/avatar.jpg");
I.checkOption("#agree");
I.selectOption("Choose Plan", "Monthly"); // select by label

I just type what my test does and what I want to see on the screen. I don’t have to bother with async & browser stuff. Codecept do the waiting & retying for me.

The good

First of all, I used codecept with pupeteer on frontend projects (yew.rs & react) and with appium on a react-native app. And it’s working!

Tests are a lot less verbose than with pupeteer or cypress for example. You can focus on writing your text. In the end, it’s easier to write and to read.

Even though I’m testing a mobile app & web project my tests looks very much alike.

One other good thing is that you can write custom helpers. If you don’t find what you need, likeI.login(); or I.intercepHttpResponse();to catch HTTP queries made by your app and compare response with what’s displayed on the screen, you can easily write an helper for that.

Also, tests inside a scenario aren’t launched in parallel just one after another. It can make your life easier to do stuff like:

Feature('Admin Dashboard');Scenario('Add a new user', (I) => {
I.amOnPage("/admin/new_user");
I.fillField("username", "newbie41");
I.fillField("password", "helloCodecept");
I.click("create");
}
// this test will be executed after adding an user
Scenario('Display recently added users', (I) => {
I.amOnPage("/admin/dashboard");
I.see("user account recently created");
I.see("newbie41");
}

In Javascript, testing frameworks generally launches tests in the order they want. Not in the order you wrote it. That doesn’t really help testing a feature while keeping your code divided.

I mean async unit test doesn’t bother me. Because I’m testing isolated elements. But when performing End-to-End test I want to run “scenarios” where previous actions often matters. And the solution to that is often to write a big test case doing various steps to verify a feature.

The bad

I had to create some helper! And therefore I had to learn about pupeteer. Codecept helper allows you to access pupeteer, protractor, or whatever “webdriver” you choose to use behind the scene.

Slower test. As tests don’t run in parallel the overall execution time is greater.

The UI https://codecept.io/ui/ is still in beta. It’s a cypress alternative. But I got good hopes for it. Meanwhile, reports displayed in the terminal do the job.

The ugly

I still haven’t found a way to select the nth element in a<select> list.

When using codecept with appium I encounter a bug. My test was working like this:

I.fillField("~new-comment", "Codecept isn't super stable yet"); // ~ say that I want to select an element based on it's accessiblity label
I.click("~submit");

Few weeks later (probably after a npm install) it didn’t work anymore. I had to use tap instead of click

I.fillField("~new-comment", "Codecept isn't super stable yet"); // ~ say that I want to select an element based on it's accessiblity label
I.tap("~submit");

I also had a scrolling issue…

My test wouldn’t “see” some text because It was offscreen. I tried scrolling to it but no luck… I had to click to some element on top of it to make appium scroll to it (not an issue with pupeteer tho).

Not really much else to complain about!

Bonus: intercept HTTP queries

I said that I had to write a custom helper.
When I was testing a website, I wanted to be sure the user’s information where displayed on the landing page. But I didn’t know for sure what this information would be because I had no control over the web API.

A “Profile” is returned by the API through GET /profile. So I had the idea to capture the http request made by the app when loading the landing page and compare it with what’s displayed.

I don’t recommend this way of testing but it can be handy sometimes (When you got no real testing API with a deterministic database).

Any way, codecept allows you to add “Helpers” like the following:

const Helper = codecept_helper;
const puppeteer = require("puppeteer");

class HttpIntercepterHelper extends Helper {
async interceptHttpResponse(url) {
const httpInspect = async () => {
const { page } = this.helpers.Puppeteer; // pupeteer is accessible within Helpers

return new Promise((resolve, reject) => {
page.on("response", response => {
if (response.url().endsWith(url)) {
response.json().then(json => {
resolve({
method: response.request().method(),
url: response.url(),
body: json,
status: response.status()
});
});
}
});
});
};
return await httpInspect();
}
}

module.exports = HttpIntercepterHelper;

That you can use like this:

Scenario("I Sign in & see my nickname in greetings", async I => {
I.login();
// when loggin-in the app make an API call to /profil to fetch user details
const response = await I.interceptHttpResponse("/profil");
const profil = response.body;

I.waitForElement("#greetings");
// app displaying fetched nickname ?
I.see(`Welcome back ${profil.nickname}!!`);
});

Conclusion

I really like codecept. It’s not super mature yet. But it’s enough to save a lot of time & headaches. You should try it for yourself!

We use it in production at work now.

Yet another good discovery, made during a personal project used at work, at no cost for my employer. That’s was I talking about in my previous article: https://medium.com/@lsmod/homeworking-7bd0a44047cb

Thank you for reading. If you want to encourage me to keep on sharing, please give me some clap!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store