Rapid Testing Intensive - Day 3

Today was day 3 of Rapid Testing Intensive.  Unlike the other two days, today the "onsighters" and "onliners" split up.  Us onliners had the pleasure of working with Scott Barber, another master tester.  He gave a very interesting talk on Performance Testing.  We talked about TCO's (Test Coverage Outline), tools, and risks.

Here is the eBay performance TCO:



In the afternoon, James talked to us about Test Reporting and the use of Testing Tools. 

TEST REPORTING:

• Heart of testing.
• Structure which helps to manage testing.
• A test report is any description, explanation, or justification of the status of a test project.
• A comprehensive test report is all of those things together.
• A professional test report is one thoughtfully designed to serve the clients of testing in context.
• A test report isn't just the facts, it's a story about the facts. Learn to tell the testing story!
• Make choice of which facts matter.
• Practice test reporting. Even if your management doesn't want it, practice anyway.  It's one of the hearts of Rapid Testing.

TOOLS:

• Tools (and automation) are important for many aspects of testing the product, but they do not take the place of what humans can find in testing.
• Do not get so dependent on a tool/automation that you miss things.
• If you are spending so much time keeping the tool/automation running, you may not be thinking about what needs to be tested today.
• Are you spending a lot of effort/time/money on tools/automation that keeps breaking?
• Use free tools!
• If your company buys an expensive tool, then you have to use it because the money was spent, but not if you're using a free tool.
• They can work if your product is very easy to test and it doesn't change much.  Does that describe your product?

A nice mix in teams that James like:

• Not all testers should be programmers.
• Have at least one person on the team that loves tools/programming.
• Someone that absolutely hates tools.
• Someone good in math.
• A good writer.
• Liberal art majors (yay for us liberal arts majors!).
• Musician.
• The toolsmith should be directed by testers.

Rapid Testing Intensive - Day 2

Today was the second day of the Rapid Testing Intensive.  Another interesting day with the Bach brothers (James and Jon).

Today we talked about Test Strategy, Risk Analysis, and Test Oracles.  We also got a lecture on Test Oracles from James, and a live pair Exploratory Testing exercise from Jon.  All in all, another good and packed day!

Today's pearls of wisdom by James:

• Our job is not documentation - it is to get the testing done. Learn how to takes notes w/o interfering with your testing.
• To train yourself to take notes, try setting a timer every 15minutes for a day. It will build in a practice of taking notes.
• What should session notes include? Tell the story of your testing - briefly. For a 90min session, three paragraphs are enough or 10-15 bullet points. Tell enough so that someone can visualize what you did.
• Oracles: A medium by which we connect our feelings about the product with the opinion of someone that really matters.  If they're not satisfied, you have a big problem.
• Oracles: A natural tester just looks for inconsistencies. You have to be able to describe your testing. Oracles are heuristics; there are no oracles that are perfect. Oracles require ongoing learning.
• Oracles: You're a tourist if you test with no oracles - you're just observing - that's lack of an oracle. You're touring a product. You're not testing.
• James is nervous when not testing with other people because he's afraid he'll miss something.
• Rapid Testing is rapid learning.  Learning is a normal thing that happens all the time.
• "If you're confused, you're right on track"

Rapid Testing Intensive - Day 1

Today was the first day of the 5 day "Rapid Testing Intensive" by James and Jon Bach.  What an awesome experience to be able to learn from these two master Testers.  There are several folks onsite with them in Seattle and over 100 of us online from all over the world. 

They did not mess around.  We immediately jumped into things.  We started off with each of us working on usability testing and product familiarization on the Motors site on eBay.  It was a great way to get familiar with the product pretty quickly.  By all of doing this, 88 bugs were written!

James then took us on a high-level overview of Rapid Software Testing.   Here are some of his pearls of wisdom:

• Rapid Software Testing is a way of thinking, a mindset.
• There are no best practices.
• Learn to test!!!  After 25 years of testing, James still says that to himself.
• Testing is about discovering the invisible.
• Don't trust when your brain says something is simple.
• Test automation can't simulate what humans can find in unanticipated problems.
• In order to control a system, you must have/make a model of that system.
• A model is any idea, activity, object that represents another idea, activity, object to help you manipulate the real thing.
•  The tester is expected to relate what they are doing to the business and explain why they are doing it. 
• Alternate reading the spec and play with product: Read, play, read, play, read, play, read, play.....
• Don't mind being confused.

We then did an exercise with TCO's (Test Coverage Outline), again on Motors in eBay.  We each had the choice of working on the following:

• Interface element
• Data element
• Platform and variants
• Business logic & flows
• Oracles

I chose to make a mindmap of the cars data:


All in all a great, full day and I'm looking forward to tomorrow.