Testing and You

The Art of Software Communication

Created by Tim Klever

Who is this guy?

Tim Klever

  • Testing Zealot
  • Lifelong Nerd & PHP Enthusiast
  • From Phoenix, AZ
  • Purple Belt in Brazilian Jiu Jitsu

Contact: tim@timisawesome.com - @timklever

What's this talk all about

Two Parts:

  • The Theory of (Testing) Everything
  • The Nitty Gritty



  1. a procedure intended to establish the quality, performance, or reliability of something, especially before it is taken into widespread use.


  1. take measures to check the quality, performance, or reliability of (something), especially before putting it into widespread use or practice.

Production code is
being tested

Manual Testing

Manual Testing is:

  • Slow
  • Expensive
  • Unreliable

But it does work!

We can do a better job!

Types of Automated Tests

  • Unit Tests
  • Behavior Tests
  • Characterization Tests
  • Contract Tests
  • Integration Tests
  • Performance Tests
  • Always More...

3 types of software bugs

  • Incorrect Implementation - Unit Tests
  • Incorrect Specification - Behavior Tests
  • Missing Specifications - Behavior Tests

Things that can not be achieved without Automated Testing

  • Continuous Integration (CI)
  • Continuous Deployment (CD)
  • Reasonable Refactoring
  • Agile / Extreme Programming Practices
  • Emergent Design
Take away my people, but leave my factories, and soon grass will grow on the factory floors.
Take away my factories, but leave my people, and soon we will have a new and better factory.
- Andrew Carnegie
Take away my test suites, but leave my source code, and soon the codebase will be expensive and bitrotten.
Take away my source code, but leave my test suites, and soon we will have a new and better codebase.

Tests Facilitate
about Software

Behavior Testing

  • Describes a development effort that provides value
  • Uses "Domain Specific Language"
  • Automated tests against human readable "ubiquitous language"

Ubiquitous Domain What?

Feature: Serve coffee
    In order to earn money
    Customers should be able to
    buy coffee at all times

    Scenario: Buy last coffee
    Given there are 1 coffees left in the machine
    And I have deposited 1 dollar
    When I press the coffee button
    Then I should be served a coffee

Running Behavior Tests


PHP implementation of Cucumber

Executes a series of "steps" defined by the keywords Given, When & Then

Brief Introduction to Unit Testing


Arrange - Act - Assert

Test Functionality in isolation in order to eliminate outside variables

Basic Test

class SomeTest extends PHPUnit_Framework_TestCase()
    public function testCanCount() {
        // Arrange
        $counter = new Counter();

        // Act

        // Assert
        $this->assertEquals(5, $counter->getCount());

Test Doubles

When we are writing a test in which we cannot (or chose not to) use a real depended-on component (DOC), we can replace it with a Test Double. The Test Double doesn't have to behave exactly like the real DOC; it merely has to provide the same API as the real one so that the SUT thinks it is the real one
- Gerard Meszaros

Mocking Dependencies

class SomeTest extends PHPUnit_Framework_TestCase()
    public function testCanDouble() {
        // Arrange

        $fetcher = $this->getMock('SomeFetchingInterface');


        $doubler = new Doubler($fetcher);

        // Act
        $output = $doubler->double();

        // Assert
        $this->assertEquals(10, $output);

Unit Testing Best Practices

  • BEWARE over mocking
  • Don't mock what you don't own (sometimes)
  • Mock Interfaces Not Implementations (if possible)
  • Do not explicitly test non public methods
  • Use the minimum amount of assertions per test (1 is best)
  • BEWARE integration tests posing as unit tests

Code Coverage

How comprehensively are we testing our code?

  • Line Coverage
  • Branch Coverage
  • Path Coverage

Line Coverage is a filthy liar!


Mutation Testing

Test our Test's Testing


Attempts to simulate branch coverage

Not all mutations are harmful

Double Loop TDD

Characterization Testing

A method to describe the ACTUAL behavior of EXISTING software


Uses XDebug execution trace data to test existing untested software

Thank You!

public function testContactInformationForSpeaker()