An introspective on Test Automation Questions.

My Previous post started February with a retrospective. So let’s do this week about Introspective! Also, Warning, there is a lot of tenants this article.

I will share with you my open questions. Those are not simple question more of philosophical type. I don’t have either enough evidence to make concrete answer, or I have an answer that I am not satisfied with.

The First Question is around ROI

Is the time I spent on making Automation bring enough value to justify its cost?

The more I think about it the problem with Automation is a condensed problem with testing. And the Dilemma of ROI of testing and automation is not new Dilma.
So my question can be generalized to:

How I can track the ROI of my Automation?

And that is basically a subset of

How I can track the ROI of testing?

I am not only one thinking about hell I am far from the Brightest one talking about it. Evil Tester talked about it not too long ago. He is not found of ROI

Brent Jensen was wondering this in the same year I started working in IT. Apparently, his post was born from discussions he had with James Whittaker. Whose answer was „Get rid of all the testers then their value will be obvious”. This is an interesting scenario to entertain. But Brent explains in his post and in AB Testing podcast
(Around 16-minute mark) why it is a wrong assumption .

And to be honest looking deep enough the actual question that lies here is:

How can I prove my tasting has value?
How can I prove that what I do is actually the best thing I can do to help my team right now?

Fortunately here I am starting to see Light in the darkness I fill that Modern Testing may hold the answer for me. But I haven’t yet arrived at it.


This is another question that I am strongly quite often, for last 2-3 month I am doing research into this subject but I am far from ready to say I have answers.

What really should be tested by Integration Testing?

Should API Testing be treaded as Integration Testing?

Where is the End to End API and Integration testing?

Why all actual code example of Integration testing is either API Testing or contract testing.

And This Leads me to my last Issue.

I think it was best described by Alan page and response to him on Twitter (warning both twits are little vulgar).

My question here is:

How can I improve my automation to test better?
Why I am still doing so much UI automation while I strongly agree with his points?

That all!

Those are questions that haunt me.
I’ve decided that finding answer that satisfies me for at least two of those is my goal for this blog for the current year.

Ok, now we finished this strange post.
What about you? What testing questions are still bothering you?

If you want to read more my musing you can check it here!

Ten post ma 3 komentarzy

  1. szwendacz

    Hmm this is quite interesting „tools can make you a fuck off awesome exploratory tester” – do you have an idea what tools? I don’t have an account on twitter so I can’t ask personally.

    1. admin

      For starters, Postman can be used to automate part of the testing process for example data preparation and allows for quite a fast generation of new data.
      I am using Fiddler a lot in my exploratory testing.

      And in past In my previous job, we have written a few tools which we referred to „QAdoThats” – those were small tools helping us with testing, like data generation, or observing some behaviour. Which wasn’t easly checkable.

      I won’t mention DeveloperTools in the browser.
      There is also bug magnet plugin for browsers :).

      1. szwendacz

        Sorry for the late response. I didn’t receive a notification about your answer.
        Anyway, bug magnet looks interesting!

Dodaj komentarz