Friday, December 11, 2015

QA Tools Part 2 - Stress Testing Mobile Apps

UI/Application Exerciser Monkey


Monkey is a program that runs on device and generates events such as clicks, touches or gestures. Monkey can be used to stress test applications.
For using monkey you need to install the ADB from Android SDK. ADB is the Android Debug Bridge which communicates with emulators and Android powered devices. You will find the adb under sdk > Platform tools.
Before running monkey, you need to get the package name so that your monkey stress test is run only on the AUT. For this we need to use Android Asset Packaging tool. You will find AAPT under sdk > build tools > 21.1.2 – aapt.exe
For pulling out the package name we need to use in cmd after redirecting to the path of the AAPT:
aapt d badging <File name>
This will give the package name along with other details

d -->  dump
badging --> prints the label and the icon for the app declared in the apk

Once package name is available execute monkey with adb:
adb shell monkey –p <package name> -v <event count>

p --> will run monkey test only on the specified package
v --> will make the results of the monkey test more verbose
s--> seed, will ensure that the same events are triggered next time so it’s easier to reproduce.

 



UI AutoMonkey


UI AutoMonkey is a simple stress testing tool for iOS applications. You can test on your app with a barrage of events like taps, swipes, device rotation.
For using UI AutoMonkey you need a mac machine with Xcode installed.
  1. Then you need to open the xcodeproject of the app from Xcode
  2. Profile > Products, build your application
  3. This will launch the app in simulator and also launch the Instruments template picker
  4. Select UI automation template
  5. Copy paste the js file in scripts
  6. Download it from here: https://github.com/jonathanpenn/ui-auto-monkey/blob/master/UIAutoMonkey.js
  7. At the top of the script, you'll see a JavaScript dictionary of configuration settings:
config: {
  numberOfEvents: 1000,
  delayBetweenEvents: 0.05,    // In seconds

  // Events are triggered based on the relative weights here.
  // The event with this highest number gets triggered the most.
  eventWeights: {
    tap: 30,
    drag: 1,
    flick: 1,
    orientation: 1,
    clickVolumeUp: 1,
    clickVolumeDown: 1,
    lock: 1,
    pinchClose: 10,
    pinchOpen: 10,
    shake: 1
  },

  // Probability that touch events will have these different properties
  touchProbability: {
    multipleTaps: 0.05,
    multipleTouches: 0.05,
    longPress: 0.05
  }
},

  1. Hit play from bottom of the Instruments window to start the script
  2. It will execute the stress test on the app on the simulator
You can configure the config file as per requirement.

Android Screen Recording using ADB


Screenrecord is a shell utility for recording display of a device. The utility records the screen in an MPEG-4 file.
adb shell screenrecord /sdcard/probwithlogin.mp4
Stop the screen recording by ctrl+c, max time limit is 3 minutes or timelimit set by –time—limit. The tool is recording the video in native display resolution and orientation.
Some points to consider:
  • This will work on Android 4.4 +
  • Audio is not recorded with video file

QA Tools Part 1

QA has several testing tools at their disposal, we generally use tools to automate repetitive, monotonous tasks. We also use tools to quickly perform checks and validations which would take a human hours to do.

#1 Xenu

Xenu is a very fast and reliable link checker mainly used for finding out broken links. Large websites with deep links are a good target for Xenu. Its an open source tool so you can download it from here: http://home.snafu.de/tilman/xenulink.html

Best thing about Xenu besides it being free is that it has a simple interface and supports SSL websites. You can customize and ask Xenu to hit a list of URLs within minutes you will get results.

xenu reddit.PNG


# 2 Bug Magnet

Bug Magnet is  a nice Google Chrome extension, that will help you with your web exploratory testing. The extension is called Bug Magnet and is developed by Gojko Adzic.

Gojko describes his tool with the following text: “Bug Magnet provides convenient access to common problematic values and edge cases, so you can keep them handy and access them easily during exploratory testing sessions. Just right-click an input field!”
The tool provides the following features:

•    Lorems | Text in different charsets and languages
•    Text size | With or without white spaces
•    Names | Different names like NULL or Chloë Rømer
•    E-Mail addresses | Valid or non valid addresses
•    Numbers | Large numbers
•    Whitespace | Leading white spaces, tabs or newlines
•    Format exploits | SQL injection or broken HTML

It provides easy and useful test cases that you could check on every form and input field of a web application. It is also a very good reminder of things like white spaces, charsets and format exploits. However, the tool has two limitations. The first one is, that it is a Chrome Extension so it works only with Chrome. The second limitation is that it works on multi-frame pages but only if they are on the same web domain.
http://gojko.github.io/bugmagnet/

# 3 SQL Inject Me

An Add in for Firefox, SQL inject helps in testing Injection vulnerabilities. You can choose what tests you want to run and based on focus your attacks. Word of caution, you might not want to run this on your prod website.

SQL Inject.PNG

Will talk about more tools in my next post about QA tools. Please leave a comment if you have any questions/suggestions or any tools that you use and find them useful.

Wednesday, April 15, 2015

Mnemonics in Software testing

Good Evening,

There is a lot of functional testing being done out in the field. Testers run their test cases, give sign offs and the product is shipped off with everyone hoping for the best. Post, we see a lot of bugs being reported from end users and beta testers. The first thing I recommend is to use mnemonics to aid our thinking and expand beyond the rigmarole of functional testing.

Wikipedia defines mnemonics as "A common mnemonic for remembering lists is to create an easily remembered acronym, or, taking each of the initial letters of the list members, create a memorable phrase in which the words with the same acronym as the material. Anyone can create their own mnemonics to aid the memorization of novel material."

Mnemonics can be simple thinking models for testers, each letter stands for an approach or consideration for testing software. You can use it to help your brain change testing perspectives, and to find important problems quickly. Some tester who catch important bugs quickly do so not just because they are experienced but they have easy ways to remember stuff that's important to them in different contexts. Mnemonics can be very useful heuristics and oracles. Below I have listed a few:


RCRCRC for regression testing by KN Johnson

  • R- Recent, means what new features/enhancements have been added. what testing around those changes should I think about?
  • C - Core, means what are the critical functions, what should I test to ensure they work
  • R - Risky, what features are inherently risky? areas that I want to safeguard by testing
  • C - Configuration sensitive, what testing I do based on environment?
  • R - Repaired,  What testing I want to do based on fixed defects?
  • C - Chronic, What areas have broken in the past and are prone to bugs? I want to test those areas

I SLICED UP FUN - by Jonathan Kohl for Mobile app testing

  • I - Inputs into the device
  • S - Store
  • L - Large vs Small screen
  • I - Interactions/Interruptions
  • C- Communication
  • E - Ergonomics
  • D - Data
  • U - Usability
  • P - Platforms
  • F - Functional Testing
  • U - User Scenarios
  • N - Networks

Can I use this? for usability testing by David Greenless

  • C- Compare Product/features
  • A - Accessibility
  • N - Navigation
  • I - Intuitive
  • U - Users
  • S - Standard
  • E - Emotions
  • T - Trunk Test
  • H - Heuristics
  • I - Instructions and Help text
  • S - Satisfaction


Ben Simo's error handling mnemonic - FAILURE

  • F - Functional
  • A- Appropriate
  • I - Impact
  • L - Log
  • U - UI
  • R - Recovery
  • E - Emotions


San Francisco Depot(SFDPOT) is a simple and effective test strategy tool created by James Bach.

  • S - Structure
  • F - Function
  • D - Data
  • P - Platform
  • O- Operations
  • T - Time


I haven't gone into the detail of each mnemonic because you can invoke google for help and read/hear about them directly from the horse's mouth. The good thing about mnemonics is that you can also create your own or add/edit to the ones you like.

Wednesday, April 1, 2015

Mobile Application Eco System

Mobile Applications have been in existence for over a decade now. Many of us have seen 'Palm pilot', and how Blackberry in its prime was used as the only platform for e-mail communication. Even today many enterprises user Blackberry smartphones for their e-mail. Mobile revolution started  when Apple introduced the iPhone, with iOS in 2008 and released their SDK for developers to develop apps for the iPhone. The revolution was followed by the release of Google's open source mobile OS called Android during 2008/2009.

Smartphone market is unlike the PC market, where Microsoft controls 80% of the market share. In the world of mobile though, its Android and iOS who dominate the market. Android has close to 77% market share and iOS around 20%, source: http://www.idc.com/prodserv/smartphone-os-market-share.jsp

Understanding the SmartPhone OS

iOS is promoted by Apple, its a Unix based platform(a stripped down version of Mac OS X). Objective C was the supported programming language for developing both Mac and iOS based applications. This has recently been replaced by Swift. Latest iOS release is iOS 8. Apple controls both mobile device hardware and software. iOS powers multiple devices like iPhone, iPad, iPad mini and iPod. Since iOS is not open source, customization is very limited.

Android is promoted by Google as an open source project from the Linux family. Android apps are created using Java(version of Java specific to android). Latest OS release is Lollipop. Android being an open source project allows OEM to control the size of the device, this increases various device sizes supported by many OEM providers like Samsung, Sony, HTC etc. Almost anything can be customized on Android.

Both Apple and Google provide developers with SDK to enable them to develop mobile applications. All the SDK related tools are provided free by both providers. For iOS the developers can get SDK for free but it will work only on a Mac OS. Apple does not provide a means to develop for iOS using Windows.

Mobile Apps and their Lifecycle

Mobile applications follow development/deployment and distribution lifecycle which are completely different from typical web/windows based application life cycle. Most of the web apps are deployed within the organization's premise and infrastructure. For mobile apps targeted to end customers, they are developed and tested within the enterprise but deployed outside the enterprise and distribution is taken care by OS provider. Mobile applications once tested and developed are deployed to App stores of the OS providers for their approval. Once approved they are available for download from the respective app stores, end customers can directly download the application from application stores on their mobile devices.

  • Apple has the Apple App Store.
  • Google has Play Store
  • Blackberry has App World


Mobile Application Developer License

If the developer completed development and wishes to deploy the app on app store then she requires the developer licence.



App stores share your revenue, in case of Apple it keeps 30 percent of your revenue. This policy is stated right on the developer program web page. Apple deals with all downloads, credit card clearing and refund requests. If your app is a paid one, then Apple pays the developer once a month after a four week delay. This delay gives apple time to process any refunds. Developer also has to earn $150 before they get paid.