Testing Resources

I recently gave a presentation to my local testing Meetup, Boulder QA Meetup.  This presentation came out of several conversations I've had recently with different Testers that weren't aware of a lot of the different resources available to Testers.  So, I decided to do my part and pass along some of the things I've been fortunate to find out and learn about.

I believe that when you learn something, give it away.  It can only make others (and you) better.

Here is my presentation on different resources. It's a way to help anyone to start towards a list of their own set of resources.  It's by no means everything.  How could it be?  There's so many sites, trainings, people, blogs, articles, tools, books…it could go on and on.  But one has to start somewhere.

I hope some of these things can help you as much as they've helped and continue to help and enrich me. I know my list will continue to grow!


I've also put the information (plus more) from my slides into a mind map below. You can go directly to Mindmeister to get a closer view.

More Thoughts on My Rapid Software Testing Class with James Bach

I've recently returned from a great experience.  Last week I attended Rapid Software Testing (RST) with James Bach.  For those of you that don't know, I would say RST is a three day boot camp for Testers!  "RST is a Testing methodology to give you the skills of testing any software, any time, under any conditions".  Before I even left for the trip, I made the decision to be open and fearless. I wanted to experience everything James had to offer. I wanted to embrace it all.

The three days were jam-packed with teachings by James, testing exercises, the famous dice game, great food, a beautiful place on the island (Orcas Island), a wonderful group of eager Testers, and magic tricks!  Honestly, we learned so much, I wouldn't know how to write it all down.  At the bottom of this post I'm attaching two of the documents so that you can see for yourself the amount of information we got.

So, instead of telling you in detail all that we learned, I would rather pass along a few things that I hope you will find helpful:

1. RST:
   -RST is like being a survivalist! Use any resources & tools available.
   -Rapid Testing = Rapid Learning
   -Pillars of RST: Diversification, Costs vs. Value, Skill, Heuristics
   -To "adopt" RST is to adopt a discipline for studying testing.
   -Indulge your curiosity.

2. Questioning:
   -Before you start testing something, ask questions, lots of questions.
   -Example of some questions you could ask to the customer, product owner, developer, etc: How does the product work? How does the system work? What is the data used? What are the risks? What is the product used for?
   -It's okay to jump in and learn about the product before or after asking questions.

3. Models:
   -Make a model of what you're going to test.
   -All testing is related to models. Test the product against models.
   -A model can be diagrams, spreadsheets, a list, a demonstration, a  program.
   -Learn the thing you have to test. Drink it up
   -And get good at mental models as well.

4. Heuristics:
   -Heuristics are one of the keys to thinking like a Tester (along with seeing what others cannot see, modeling what you're testing and test against model, and asking LOTS of questions.
   -Heuristics can be anything and everything that helps you test!
   -All oracles are heuristics.
   -Heuristics depend on the context.
   -Heuristics say, "How may I be of service"?
   *I think I'm finally starting to grasp more of what heuristics are, but I'm going to keep studying and learning to add to my expertise as a Tester.

5. Exploratory Testing:
   -Is not a testing technique, it's an approach.  It can work with any testing technique such as 'Exploratory' Performance testing, 'Exploratory' Stress testing, etc.
   -Using exploratory testing and scripted testing together is good testing.
   -Variety gives you power.

6. I don't have to be an expert at everything, but I need to be committed to learning, growing, and teaching.

7. Risks: Where is the fire burning and what do I do to put out the fire.

8. When you run out of ideas or are frustrated, defocus. When you're confused, focus.

9. Okay to be confused as long as you're on the way to be un-confused

I don't consider myself an expert on all of these things just yet, but here's the best part...I don't have to be "yet".  It's an on-going journey, isn't it? As a lot of us Testers know, we need to be continually learning. It never stops.

RST had a profound effect on me. I felt it on the last day. It's hard to describe, but I was changed. Something shifted in me. My confidence as a Tester AND a person had grown. James is a great teacher, but more importantly, a great person that cares about people learning, growing, and believing in themselves. He saw things in me that have made a difference in me that I can't thank him enough for.

As I said in my first post about RST, if you ever have a chance to take an RST class, just do it!

RST slides
RST appendices
Pictures from RST!
RST overview video by James Bach (taped at our RST class, June 2013 on Orcas Island)
James Bach = Steve McQueen!

Me and James!

Rapid Software Testing Experience-First Thoughts

I just got back home yesterday after an amazing week with James Bach and Rapid Software Testing (RST) at Orcas Island, Washington.  My head is so full of things I learned and the experiences I had, I'm not even sure where to start.  I will take my time to get my thoughts down to share, but for now just know that if you ever have the chance to take RST from James, Michael Bolton, or Paul Holland, just do it.  Don't hesitate, just do it.  You won't regret it.  You owe it to yourself as a Software Tester to challenge yourself with this intense and information-packed class. 

I was challenged several times by James, but through those challenges a wonderful thing came out of it.  I realized I'm now believing in myself more.  I made the decision before stepping foot on the island that I was going to be fully open and present.  That I was going to put myself out there and take chances.  It wasn't always easy or comfortable, but at the end of the last day I knew something had changed in me.  Something had shifted.  I honestly don't know what all of it is just yet, but that's part of the next chapter of RST for me.  Figuring out what those changes are, what all have I learned and what more do I want to learn, who I want to be, and where am I going.  Pretty heady stuff from a three day class, but I believe that every experience in life is an opportunity to change your world if you so choose.  And I'm very much looking forward to seeing where MY world is going to go and how it's going to change!

Why I Am Going To RST

In about a week, I will be on my way to the Rapid Software Testing (RST) course. It is a three day hands-on intensive course.  And this particular course will be taught by none other than James Bach. 

The main reason I'm going is exactly because of what it says on the RST website, "The ideal student is anyone who feels driven to be an excellent software tester".  That's me. 

I've been a Software Tester for over 10 years, but here's a true confession. During a lot of those years I was a naive tester. A tester in the dark.  Like a lot of us, I didn't have a lot of formal Testing training. I had no idea for almost all of those years that there were ways and people to learn testing from. I never worked anywhere that taught or trained Testers. I knew nothing of people like James Bach, Michael Bolton, Pradeep Soundararajan, Elisabeth Hendrickson, and many others.  I didn't know there were books or websites or trainings.  I didn't know there were test groups like Association for Software Testers (AST), Ministry of Testing, Satisfice, and others. Knowing what I know now, this all sounds crazy to even admit all of this. But it's true.  Sometimes you don't know what you don't know.

I just did the best I could.  I picked up tips from other Testers along the way or I made it up as I went along.  Maybe because I'm anal by nature, I would find lots of bugs.  The things I loved about testing kept me doing it, like finding things that were wrong with the software before our customers did.  But I was miserable in a lot of ways.

So, about a year and a half ago I sat myself down and had good long talk with myself. I thought that if I'm going to stay being a Tester, I needed to really LEARN how to be a Tester, what it meant to be a Tester.  Not just do the 'job'. No one was going to do it for me. No one.  My big break was that I somehow found James Bach's book "Secrets of a Buccaneer-Scholar".  I have to say, this book is a gem and a life-changer.  It woke me up in a very big way about my own responsibility of self-learning.  I can't recommend it enough.  And my journey at that point started in a big way.

I started googling anything to do with Software Testing.  I found people like Anne-Marie Charett who were generous in coaching me about different things to do with testing. I started a blog. I discovered this great test community on Twitter. I started reading about testing from blogs, articles, and books.  I learned about Context Driven Testing.  I found out about and became active in Weekend Testing.  I took the BBST Foundations course. I even took some HTML and CSS courses to help me in web testing. I stumbled onto Ajay's Balamurugadas' first online training and never missed a day, as well as discovering a wonderful test community happening in India.  There was a whole world out there that I never knew existed!  And now I'm constantly doing something every week and most days to keep growing and learning.

And that brings me to the RST course. It's time to take this journey of mine to a whole new level. To accelerate my learning. I want to finally grasp heuristics, learn how to tackle any product instantly, and to grow in my confidence as a Tester. I want to continue on my journey of becoming an expert Tester, but more importantly, FEEL and KNOW I'm an expert Tester. And I have this strong feeling that I will someday look at this time as the before and after of my journey and skills as a Tester. The way I look at it, if you want to be a better Tester, you can't just wish for it.  You have to work on it everyday. RST is a major step in this process. 

Wish me well!

Teri Charles
http://bouldertester.blogspot.com/
@booksrg8

A Software Tester Pays it Forward

Recently I've been thinking about the fact that I run in to so many Software Testers that have never heard of people like James Bach, Michael Bolton, and Pradeep Soundararajan, as well as so many other Test gurus.  Some of these same Testers have also never heard of groups like AST (Association of Software Testers) and the Software Testing Club.  They've become such a integral part of my professional life that I just assumed other Testers will know of them as well.  And sadly, that is not the case, not by a long shot.

I'll be honest, I hadn't heard of most of these people and groups myself until I made a decision a little over a year ago to immerse myself in Testing.  I was at a crossroads in my professional life and realized that I needed to challenge myself and dig deeper in this profession of mine, Software Testing.  I started with baby steps by Googling about Software Testing, which led me to one article, then to another, and then to many others.  It was while reading one of these articles that I stumbled upon a Testing coach, Anne-Marie Charrett.  It was Anne-Marie that really opened my eyes to a whole world out there that I had no idea existed.  I started to learn about heuristics, oracles, different training courses, and other Testers.  It was Anne-Marie that suggested I start my own blog as well as take the AST’s BBST Foundations training course (which I did and passed!).  I also started following people on Twitter, then more on Twitter, and even more.  Twitter changed everything.  I started realizing that Testers communicate a lot through Twitter.  I would learn about different Testing groups, blogs, articles, books to read, trainings, and other great Testers to follow.  I hooked up with other Testers all over the world.  I took Ajay Balamurugadas’ first online training course he taught and got a new friend in Ajay in the process.  I learned about and joined the Software Testing Club, as well as other groups.  And then recently I became one of the co-organizers of Boulder Colorado's first Software Testing Meetup groups.  And most exciting, I will be attending my very first formal Test training this summer, RST (Rapid Software Testing), with none other than James Bach. And my journey continues to grow every single day. 

But my journey wouldn’t have begun unless I took that first step to explore what was out in this Tester’s world of ours.  So, instead of wondering why other Testers haven't heard about some of our great Test leaders and mentors, trainings, groups, and articles, I'm going to do my part and do what others have done for me this past year.  I’m going to make sure that I'm passing along the good stuff!  Pass along those nuggets of wisdom that come my way like others have done and continue to do for me.  This blog post is my first step.

   

Whether you’re a new or experienced Tester, do yourself a favor and start following different people or groups on Twitter, then start taking a look at their websites, then start reading their blogs, articles, and trainings they've written.  And if they've written books, add them to your library and start reading them. If you can get to a conference they'll be speaking at, go.  And if they have a training you can get to, go.

So here you are, fellow Testers.  Follow these folks, learn from them.  There's much to learn and it never, ever ends.  And it will keep you busy for the rest of your life!

The following are just a few people and groups to start with.  There are many, many more, so my apologies in advance for leaving a long list of people and groups out, but this is just to get you started on your own journey of discovery.  There are a lot of great people out there doing a lot of great things.

James Bach:  http://www.satisfice.com - @jamesmarcusbach

Michael Bolton:  http://www.developsense.com - @michaelbolton

Pradeep Soundararajan:  http://testertested.blogspot.com - @testertested

Anne-Marie Charrett:  http://mavericktester.com - @charrett

Keith Klain:  qualityremarks.com - @KeithKlain

Paul Holland: testingthoughts.com - @PaulHolland_TWN

Elizabeth Hendrickson: http://www.testobsessed.com - @testobsessed

Michael Larsen:  http://mkltesthead.com/ - @mkltesthead

Lisa Crispin:  http://lisacrispin.com - @lisacrispin
Colin Cherry: http://itesting.com.au/ - @cherryAKAtism

Jerry Weinberg: http://www.geraldmweinberg.com - @JerryWeinberg
Ajay Balamurugadas: http://www.enjoytesting.blogspot.in - @ajay184f

AST: http://www.associationforsoftwaretesting.org - @AST_News

Software Testing Club:  http://www.softwaretestingclub.com - @testingclub

Weekend Testers Americas: http://weekendtesting.com/ - @WTAmericas

Weekend Testers:  http://www.weekendtesting.com - @weekendtesting

Europe Weekend Testers: http://weekendtesting.com/ - @europetesters

Satisfice: http://www.satisfice.com - @satisfice_inc

TeaTime with Testers: http://www.teatimewithtesters.com - @TtimewidTesters

The Testing Planet: http://www.thetestingplanet.com/

Explore It! by Elisabeth Hendrickson

I just finished reading one of the best software testing books out there.  It's a brand new book on exploratory testing by Elisabeth Hendrickson, called Explore It!.  During my reading of this book, I found myself wanting to tell every Tester I  know to go out and buy this book....and I've been doing just that!  I think what I liked the most about Explore It! was how easy it was to read.  And by easy, I don't mean dumbed down.  There's some complex stuff in there, but Elisabeth wrote it in such a way that no matter if you're a new Tester or an experienced Tester, you're going to get something out of it.  Like an explorer, she walks you through the many journeys of testing software, to think as an explorer just as you would if you were Lewis and Clark.  That's a pretty cool thing to keep in mind. 

My goal is to reuse the things I've learned in this book over and over again.  For example, I've not really done a lot with charters, but I'm inspired to use them more from what I've just learned.  It's clear things like this take practice.  But isn't that what will make us the great Testers we want to be?  I also really liked how she explained helping your exploratory testing using diagramming state models, variations with variables, exploring with requirements, following the data, and her list of heuristics at the back of the book.  I've known and used a lot of these things in the past, but it's a great resource to have it in one place and again, Elisabeth explains it in such away that makes it easy to grab for a quick inspiration!

Seven Ways to Find Software Defects Before They Hit Production by Matthew Heusser

I've recently read Matthew Heusser's great blog post "Seven Ways to Find Software Defects Before They Hit Production".  I always like a good article that gives some quick tips and teaches you something.  Matthew did both of those things.  As I was reading it this time I started thinking about the best way to remember and use some of his advice.  And then it came to me.  Mind map!  So here's my attempt at breaking down Matthew's highlights and tips.  Also, I used MindMeister (free online mapping tool) for the first time on my blog, so I hope it works okay (check out the zooming at the bottom of the mind map).


Create your own mind maps at MindMeister

2012 Journey and Highlights

Yep, it's been way too long since I've written a new post.  I guess that can happen when you get a new job!  It was quite a year.  I did a lot and learned a lot. 

My year started with being laid off.  I'm not one to look at lay offs as a negative thing.  If one is prepared, they can also be a great opportunity.  I decided to take a bunch of time off (having to have two shoulder surgeries back to back certainly helped in that decision!).  But it also gave me time to think, reflect, and learn.  And boy, did I learn a lot and have some great adventures!

Here are some of the highlights:

• The biggest decision I made had to do with what I called "Teri school" by diving into a journey of self-learning.  James Bach's book "Secrets of a Buccaneer-Scholar" was a great motivator.  It's a great read.
• Worked with a great Testing coach, Anne-Marie Charrett, who challenged me and helped me to push myself.
• Took a month long online testing course with Ajay Balamurugadas.  Ajay has become a great teacher and leader in the Testing world and I continually learn new things from him.
• Started my Twitter account to grow in my relationships and learn from Testers throughout the world.
• Worked with a mentor to learn HTML and CSS to increase my knowledge in web testing.
• Refreshed my SQL skills
• Read numerous books, white papers, testing magazines/newsletters, and blogs on testing.
• Completed the 5 day Rapid Testing Intensive class from James and Jon Bach in July 2012.
• Participated in Weekend Testing events.
• Created this blog on Testing.
• Joined several Testing organizations.
• Took free online classes from Coursera and Udacity.
• Successfully completed the BBST Foundation course (Association for Software Testing)!  Not for the faint of heart!

And then I started a new job about 4 months ago at a company called VisionLink.  What attracted me to VisionLink was their mission for making affordable software for non-profit organizations.  That really spoke to my heart.

So, now it's a new year.  I'm not one for making New Year resolutions because I believe in constantly learning, growing, and experiencing things every chance I get.  And I can't wait to see what 2013 brings!

Bugs Spread Disease (a post by Elisabeth Hendrickson)

I just read a great blog post by Elisabeth Hendrickson called "Bugs Spread Disease".  I thought it was so good and an important read for Testers, I wanted to share it with you.  Do yourself a favor and take a few minutes to read it. 

http://testobsessed.com/2012/08/bugs-spread-disease/

Rapid Testing Intensive - Day 5

Today was the final day of the Rapid Testing Intensive from James and Jon Bach.

This has been a great experience.  It really makes me want to take their full Rapid Software Testing class the next time it comes up.  My mind is full!  I am looking forward to using many of the things I learned this week.  And it's been great getting to hang out with a lot of Testers from all over the world for 5 days of learning, talking, and challenging ourselves about testing.  And one of the best parts throughout the whole thing was the humor and interaction between the Bach brothers.  It made the experience even sweeter and I feel very fortunate that I was able to participate in the first Rapid Testing Intensive. 

Rapid Testing Intensive - Day 4

Day 4 of Rapid Testing Intensive with James and Jon Bach was another fun-filled (and full) day.

Here are some of the things we discussed today:

We started the with them showing us a graph of all of the bugs we have found in the last several days testing eBay Motors.  One of the important things for something like a bug graph in Rapid Software Testing is to ask yourself if the graph is truthful (filter out duplicates, etc.). If not, it's just noise.  Something like a bug graph may not always seem like something that provides value to your testing, but it may provide value to stake holders, so don't have it be misleading by not cleaning it up first.  This seems obvious, but it's easy to be misleading sometimes without meaning to.

Another thing James talked about was using documentation, don't look for some magical document that has the secrets to the universe.  Behind every document there needs to be someone you take seriously.  This also a very good thing to remember when you put together your own documentation.

Modeling:
I've always been a visual person, so drawing out how a product works when I'm new to it has come pretty natural to me. But I've really learned from James how important modeling and modeling the product's interactions is. For instance:

• If you're doing code-based testing, model around elements of the code.
• If you're doing state-based testing, model around elements of the states.
• Learn enough about the product to identify interacting variables.
• Before creating your models, survey the prodcut by reading, interviewing people, reviewing field data, doing exploratory testing, scenario testing, user testing, or otherwise keeping your eyes open during any other kind of testing.
• All testing is based on models. Without models, you can't test systematically. And you cannot systematically test anything important unless you first know that it is THERE.
• Turn options off/turn options on, play/test, see which ones actually interact.

State-based testing:

• Create a state-based model.
• Check transitions.
• All the ways to get into states and out of states (input and output).
• Any variables that get modified.
• Look for any sub-states.
• Combination testing.
• Tours.
• Flows of states.
• Understand how other variable effect state model.
• This is all part of deep coverage testing instead of just looking at product and wondering "what does this button do".

Combination-based testing:

• What if you have a lot of variables and want to test - this is combination-based testing.
• First step in combination-based testing: identify the interacting variables. Find variables that "might" interact and any "potential" risks. There are a LOT of variables in your product.

After having a representative from uTest (Testers, check it out), we ended the day doing some combination-based testing on eBay Motors.  Here is the combinations I came up with along with the results.
     

SECTION	COMBO	RESULTS
Motor -> Cars/Trucks	No fields (unclick New & Used)	43,699 matches found
Motor -> Cars/Trucks	New only	5,232 matches found
Motor -> Cars/Trucks	Used only	38,470 matches found
Motor -> Cars/Trucks	New & Used only	43,701 matches found
Motor -> Cars/Trucks	New & BMW only	11 matches found
Motor -> Cars/Trucks	Used & BMW only	2,064 matches found
Motor -> Cars/Trucks	New & Used BMW only	2,075 matches found
Motor -> Cars/Trucks	New, BMW, 3 series only	4 matches found
Motor -> Cars/Trucks	Used, BMW, 3 series only	600 matches found
Motor -> Cars/Trucks	New, Used, BMW, 3 series only	604 matches found
Motor -> Cars/Trucks	New, Used, BMW, Zip (80301), 3 series only	604 matches found

Rapid Testing Intensive - Day 3

Today was day 3 of Rapid Testing Intensive.  Unlike the other two days, today the "onsighters" and "onliners" split up.  Us onliners had the pleasure of working with Scott Barber, another master tester.  He gave a very interesting talk on Performance Testing.  We talked about TCO's (Test Coverage Outline), tools, and risks.

Here is the eBay performance TCO:



In the afternoon, James talked to us about Test Reporting and the use of Testing Tools. 

TEST REPORTING:

• Heart of testing.
• Structure which helps to manage testing.
• A test report is any description, explanation, or justification of the status of a test project.
• A comprehensive test report is all of those things together.
• A professional test report is one thoughtfully designed to serve the clients of testing in context.
• A test report isn't just the facts, it's a story about the facts. Learn to tell the testing story!
• Make choice of which facts matter.
• Practice test reporting. Even if your management doesn't want it, practice anyway.  It's one of the hearts of Rapid Testing.

TOOLS:

• Tools (and automation) are important for many aspects of testing the product, but they do not take the place of what humans can find in testing.
• Do not get so dependent on a tool/automation that you miss things.
• If you are spending so much time keeping the tool/automation running, you may not be thinking about what needs to be tested today.
• Are you spending a lot of effort/time/money on tools/automation that keeps breaking?
• Use free tools!
• If your company buys an expensive tool, then you have to use it because the money was spent, but not if you're using a free tool.
• They can work if your product is very easy to test and it doesn't change much.  Does that describe your product?

A nice mix in teams that James like:

• Not all testers should be programmers.
• Have at least one person on the team that loves tools/programming.
• Someone that absolutely hates tools.
• Someone good in math.
• A good writer.
• Liberal art majors (yay for us liberal arts majors!).
• Musician.
• The toolsmith should be directed by testers.