Mock testing, Spy Test, Software Stub : What are the
differences
In one phase of our project, we needed to test our UI behaviors
when the layers below it (facades) were not implemented yet. We had
to put something or a test double in the place of lower layers and
run our test on the UI layer. There are different types of
test doubles used for different purposes:
- Dummy objects: Usually are just used to fill parameter
lists and are passed around but never actually used.
- Fake objects: actually have working implementations, but
usually take some shortcut which makes them not suitable for
production (an in memory database is a good
example).
- Stubs: provide canned answers to calls made during the test,
usually not responding at all to anything outside what's programmed
in for the test. They are sometimes called fake stub. Stubs may
also record information about calls, such as an email gateway stub
(Software Stub) that remembers the messages it 'sent', or maybe
only how many messages it 'sent' (Spy Test or Spy Object).
- Mocks (Mock Testing): objects pre-programmed with
expectations which form a specification of the calls they are
expected to receive. They are sometime named fake mock.
Mock testing in Java
As mock testing in java (mock objects) are often described
poorly and most of developers use them without noticing the main
purpose or definition, we have a discussion with our colleagues
about what is mock testing and when we should choose one of mock
software, stub software, dummy class or fake objects . In
particular, mock testing were confused with stubs. We finally
reviewed mocks fakes and stubs and found out that, of these kinds
of doubles, only mocks insist upon behavior verification. The other
doubles can, and usually do, use state verification. Mocks actually
do behave like other doubles during the exercise phase (Act
Setction), as they need to make the SUT believe it's talking with
its real collaborators - but in software testing mock test differs
in the setup and the verification phase.
So the main difference between mocks vs stubs and fakes is in
how test results are verified: a distinction between state
verification and behavior verification. As we trying to verify the
state of UI so behavior verification is out!
Finally we chose the fake objects to have a working
implementation of lower layers. It was needed to setup these fake
objects somehow that easily be replaced with the real lower layers
and starting tests. Anyway, when the real layers implemented and
tested carefully, we will use them as the real DOC. We had 2
challenges to overcome for using fake objects instead of real DOC
for test:
- Fake objects should be deploys with the same JNDI name as the
real DOC.
- Fake Objects should be easily deployed in Test setup.
As we have used the Arquillian and JUnit as the test frameworks
for running our unit tests in the real container, we needed to find
a solution that matching them.
In our implementation, the fake object is a fake implementation
of the interface that the real DOC will realize. It will have the
whole specifications of the same DOC, such as being Stateless EJB,
Same transaction attributes and so on.
For monkey tests of UI, we need to Deploy these fake objects. As
we want to separate these classes from our production code we
decided to put them in test package. For deploying, my colleague
wrote a test runner that deploys these fake objects and waits for
UI to interact with these objects. Finally, calling a servlet will
un-deploy these modules and terminates the test.
