Quality Assurance According 2 Einstein
is a talk which introduces several different ideas about how we need to think
and approach software testing. It touches on subjects like mutation testing,
pairwise testing, automatic test execution, smart test non-execution, using tests
as monitoring tools and team/process organization.
Because testing is more thinking than writing I have chosen a different format
for this presentation. It contains only slides with famous quotes from one
of the greatest thinkers of our time – Albert Einstein!
This blog post includes the accompanying links and references only! It is my first
iteration on the topic so expect it to be unclear and incomplete, use your imagination!
I will continue
working and presenting on the same topic in the next few months
so you can expect updates from time to time. In the mean time I am happy to discuss
with you down in the comments.
- Hello World bug challenge
- Testing a Sudoku
THE FASTER YOU GO,
- Using Statistics to Predict Which Tests to Run
- The framework that knows its bugs
- Testing Red Hat Enterprise Linux the Microsoft way
- Automatic dependency testing with Strazar
- Automatic cargo update, test and pull request
IF THE FACTS
CHANGE THE FACTS.
- Coverage is Not Strongly Correlated with Test Suite Effectiveness
- Code Coverage is a Strong Predictor of Test Suite Effectiveness
- Mutation testing vs. coverage, Pt. 1
- Mutation testing vs. coverage, Pt. 2
- There are 101 coverage metrics according to Cem Kaner.
Which ones are you measuring and what conclusions are you making out of these metrics?
THE WHOLE OF
IS NOTHING MORE
THAN A REFINEMENT
- How to find 1000 bugs in 30 minutes
- How we found a million style and grammar errors in the English Wikipedia
- Simple Testing Can Prevent Most Critical Failures
- Need it robust, make it fragile
- btw its me who asks the first question at the end 🙂
DOING THE SAME THING
OVER AND OVER
This principle can be applied to any team/process within the organization.
The above link is reference to a nice book which was recommended to me but the
gist of it is that we always need to analyze, ask questions and change is we want
to achieve great results. A practicle example of what is possible if you follow
this principle is this talk
Accelerate Automation Tests From 3 Hours to 3 Minutes.
REASON FOR TIME
IS SO THAT
HAPPEN AT ONCE.
The topic here is “using tests as a monitoring tool”.
This is something I started a while back ago, helping a prominent startup with their
production testing but my involvement ended very soon after the framework was
deployed live so I don’t have lots of insight.
As the first few days this technique identified some unexpected behaviors,
for example a 3rd party service was updating very often. Once even they were
broken for a few hours – something nobody had information about.
Since then I’ve heard about 2 more companies using similar techniques to continuously
validate that production software continues to work without having a physical
person to verify it. In the event of failures there are alerts which are
delath with accordingly.
CAN BE SOLVED FROM
THE SAME LEVEL
THAT CREATED IT.
That much must be obvious to us quality engineers. What about the future however?
I don’t have anything more concrete here. Just looking towards what is coming next!
DO NOT WORRY ABOUT
I CAN ASSURE
YOU MINE ARE STILL
Thanks for reading and happy testing!
Source From: fedoraplanet.org.
Original article title: Alexander Todorov: Quality Assurance According 2 Einstein.
This full article can be read at: Alexander Todorov: Quality Assurance According 2 Einstein.