Testing Tour Stop #5 : Exploring and pairing on AI and ML with Tariq King

I had my fifth session on my #TestingTour with Tariq King and discussed all about AI and ML. I met Tariq King at Hustef, Budapest in October this year where I was speaking as well. The conference theme was #inspire #invest #innovate and I also got the chance to attend Tariq’s keynote. This theme and Tariq’s keynote had a huge impact in terms of motivation and inspiration. So now you can imagine why I was so super excited to get a chance to pair with Tariq and learn about AI.


We had already planned before the session that we would have some introduction to AI and ML and then we would use Jypyter Notebook application for some hands-on testing.

Tariq started off with an introduction to AI and at this point, I could still not believe that I'm learning from him. I told him that I tried to find out about AI quiet a lot of time by reading articles but could not understand the concept so treat me as completely new to this topic.

Tariq started off by explaining about AI. He mentioned that AI is really a broad field, It is all about how computers simulate and about having intelligent behaviour. He mentioned that when we try to learn about AI we can see a lot of mention about intelligent agents. Intelligent agents are robots or an entity that can perceive and act.

Tariq gave an example of my #TestingTour, the decision of how I wanted to do was by pairing with different people. Or I could have asked help from people about how I could do my #TestingTour.  Another example, If I walk into the room its dark, and I can know its dark.
So these bots are like built computer programs that can interact with the environment with sensors.

And he gave an example of how we could use AI for exploratory testing by building these agents that try to perceive and do exploratory tasks. At this point, I was getting more clarity about this topic and I compare this to automation checks we perform using any tool for ex Selenium. We provide instructions by writing scripts for it to perform the task and give us back the results. This was more of a question and Tariq mentioned that yes we do provide instructions but not what exactly it needs to do but we provide high-level instructions.

Now I had another question in my mind at this point, do we have to build this bot according to what we need that bot to do so we can it in our testing? And this is where Tariq mentioned without even me asking this question, that we don’t have to build the bots to use it to test it.
So this is where I thought, when we using automated testing we use some tools based on the type of project/product we are working on and we don’t have to build the automation tool from scratch for us to start the automation.
Different types of Intelligent agent:
  • Reflex based
  • Model-based
  • Utility-based
  • Goal-based
  • Learning-based
Before going into each of the Intelligent agents, Tariq gave an example of his 2-year-old daughter, I really liked all the real-life example’s he was giving me which was making these concepts so clear. So coming back to the example, his 6-year-old daughter does kart-wheeling and the 2-year-old tries to copy doing that. Kids see something once and they try to follow by example. Similarly, these bots need some data which they can perceive and act on it. So instead of trying to tell the bot that its a raindrop, it feels soft and providing its size and colour, we provide examples. The more examples we give, the better it gets at it.

So here’s another example, when we look at any random login screen we know its a login screen even if dint had any title or even there was no text prepopulated in those text fields that its username and password. How do we know, its because we have seen a lot of examples of login screens. At this point, I had a question in my mind about how we could provide these examples to the bot but I was sure I'll get an answer to this question further in this session.

Now coming back to Reflex based agents, they have reflex action. It performs actions based on the current situation. For example, if we are driving then our actions depend on the environment to decide what needs to done and when. It’s an intelligent agent, that can observe its environment then take actions.

Model-based agents work by following the conditions or rules. For example, we can build the model while testing - Go to homepage> Add an item to the basket> Go to the shopping cart, they use the goal. For example, when the light goes off because I know the room I can navigate, so similarly model-based agents, they follow the model to perform an action.

Goal-based agents are kind of a problem-solving agent who follows or choose their actions in order to achieve goals.  Utility-based agents make decisions based on past and current perspectives, making decisions any moment and how happy we are. For example, you have to do some tidy up at home and clean the dishes, and you decide to go for shopping instead as this makes you feel happy and you decide to do all other tasks after this.

Learning agents has a learning element. It gets better the more it sees. When we first use the bots they are not good but more it learns the better it gets. For example, the self-driving cars when new they are not that good but the more miles it runs the better it gets.
How can we train the bots, Yes I had the same question. So Tariq mentioned that the data is very important and data is kind of examples for training the bots. For example, how to teach an agent to categorise between different flowers. The answer is we can get different flower images which are data for training. Another example could be, having different example images of login screens or shopping carts to train the bots. I compared this data and example images to the test cases/scenarios which we use while we perform testing.

Now was the fun part to actually run the test and see it. We wanted to train the bot to identify different flowers and we added data for 3 different flowers to make it more precise as the agents need training and consumes the data to perceive and then performs the actions and then gives the results.  We did a test run with 60% as a training data set and 40% to test. We tried to run another test by splitting the data randomly into set of 67% for training and the remaining as test. And it shows the results like this

We have to keep training these bots, and over time it gets better. It’s not like once we have a set of data and images and train the bots and job is done. It’s just like automation, we need to keep maintaining tests and updating the tests, maintain the code to get better value and results.

These are cool resources which Tariq shared so I can go further and learn more about it AI and ML.

  • https://cloud.google.com/vision/automl/docs/
  • https://teachablemachine.withgoogle.com/
  • https://scikit-learn.org/stable/
  • Numpy and Pandas


  • AI is an intelligent computer program that can perceive and simulate.
  • We need to train the bots with a lot of different examples of data for it to perform the test and provide the results.
  • We don’t have to provide exact instructions to the bots to perform a test, we just need to provide examples of data so the bots can be trained/learn so it can provide the predictions by going through the patterns. For ex, we could use AI to prioritise test cases/scenarios. 
  • People were worried that manual testing is going to be dead when automation was introduced. But this was not the reality. Similarly, I saw a lot of news and hype that bots are going to replace testers. After some understanding about AI I strongly felt that even if we use AI for testing, we need testers to train the bots by analysing what type of data we need to provide to train the bots.
  • It's similar to automation, we need to understand the product, come up with test scenarios that need to be automated and update the tests based on new features. Similarly, we need to first understand what type of data we need to train/learn the bots so we can use them for testing.
  • Executed a sample test to see how AI works and got few resources which I can use to learn more about it.
This is just an introduction what I learnt about AI and there's a lot more to explore and learn.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.