Saturday, June 23, 2012

DDoS - Issue Tracking Style

Sometimes following process bites you in the ass.  Or maybe i mean that sometimes following process just bites.  Doesn't mean it's not still the right thing to do, it's just sometimes hard.  By example...

We use Jira as our issue tracking system at work.  As with most issue tracking systems, if you're not incredibly (arguably impossibly) diligent at upkeep you end up getting a number of cases that languish forevermore in an uncertain state.   It's just the way of things. there's bugs that you're not going to fix...bugs that didn't make any sense...bugs that got fixed through some other issue but never got cross related...bugs that simply aren't applicable any more because that entire system has been deprecated...and...the list goes on.  If you can follow strong process and make sure that every single bug that you get in is cross-referenced properly and categorized well then it minimizes it but i simply don't believe that anyone out there has no issue with growing languished bug lists.  My opinion is that you try hard to have a good categorization scheme, strong workflow built into your process and your bug tracking system and make your best effort.  And then every couple of years you devote some time to clean-up.  

So fast forward to week before last at my company.  One of the Dev managers decided that before we start some fairly major re-design efforts that the backlog of languishing bugs should be sorted and dealt with so that when we start the design we're not only taking into consideration the requirements and issues that currently rest in the forefront of everyone's brains but that we also make sure we're building upon lessons learned in the past.  A very good and honourable intent.  I wasn't aware that she was making this least until i started to see it's results. 

So, as QA manager i have it set up that any bug that gets updated, entered, closed etc in Jira sends me a notification.  Daily i read all of these updates, generally less than 50 on your average day and it only takes a few seconds per case unless there are issues.  It keeps me in the loop on a lot of things and when i spot issues then the time spent has been invaluable.  It's a good process to follow for me and as i'm still pretty new, an important one.  Jira has a few annoyances along this line in that standard process for closing bugs for some of my team gets me 3 notifications back to back.  But i've learned to recognize these in the first case and just mark the next 2 read and move on.  

Well one day a couple of weeks ago i started noticing quite a lump of cases coming from the Dev manager.  As i was processing them i noticed that they were all old (some of them 4 years or older) and that they fell into three basic categories;

a) 'no longer an issue, closing the bug,'
b) 'probably not an issue, please retest and confirm,'

c) 'still an issue,' additional categorization added. 

So i started to process these cases myself.  And by process i mean read.  A couple, and only a couple of the close ones i grabbed and asked for re-test instead because i wasn't sure and most of the retest ones i assigned to a particular resource.  It was all good.  Fast forward again, this time until i've processed over fifty of the things and i go to find out what's up.  She tells me what she's doing and i think it's a good idea and i resign myself to some extra work for the next few days.  When it hit a 100 or so i realized that although i was seeing about a 25% rate of ones marked for closing i was never actually seeing any bugs being closed.  

So i went and asked the Dev Manager if she was closing any, to which her response was that she wasn't because Dev's don't close bugs, QA should.  Which i think is a great response and really how it should be.  I objected to the use of the term in the bug 'closing the bug,' and she agreed.  We discussed and since i review all bugs, even those that have been closed, and since she had some 750 of them to process for this short period she was going to close them as well and rather than my having to read the email notification and then open the bug and update it she would just do that part.   

This sounded like a good, safe solution to me.  Until she emailed me 15 minutes later saying that she didn't have the rights to close bugs.  

Here's where proper process bit me in the ass.  It's pretty standard to have this process as well.  It's common to only allow QA's to close issues so that you don't have the devs making decisions without knowing the full story.  It's good process.  I don't argue this fact at all.  However, when i went home that friday i still had a queue of about 100 to go through, when i came back in monday, i had 385, the dev manager worked until 8:30 PM on a friday night going through the backlog.  

Here's where the DDoS (denial of service) title of the blog entry comes in.  All last week i tried to find time to process this stack but all i had was this ever-increasing list of notifications to process because she was finding more time that i could.  When you have 400 notification emails most of them with the same sender name beside them, it becomes difficult to see the new updates from other team members.   Each scrum that week i mocked the dev manager for her DDoS attack, and each time she apologized.  I indicated it was ok but that i was still going to mock her for the attack.   It was my method of releasing the annoyance of the task.  

Because, everything else said, it's still an important task.  Two people processing all the cases seems from some standpoints to be silly but in the end, clearing out the backlog is great.  It's going to provide excellent information for the new design work and i'm learning a ton about the system as we go.  And i've caught a dozen or so that i wasn't happy with the action taken and altered it to be a little safer. 

Sometimes process hurts. 
If it's good process though, it always helps. 

When i left this friday i still had 100 in the queue.  But i had started sorting and ensuring i was going through the new ones first every day. And she's through all 750 now, so 100 is all i should have left. 

Thursday, June 14, 2012

Hiring hiring hiring

Hiring is on my mind lately, not the least because i have a couple of spots that i'm looking to fill.  This morning the twitterverse handed me a couple of articles that made me want to talk about hiring a little bit.

The articles in question are "Why you should hire like a rock band."  and "The case for hiring 'under-qualified' employees,"  You can read them as you will but essentially the first talks about hiring for culture and the second talks about hiring for hunger.  Both of these articles struck a similar chord to me, both have the same goal in the end.  You need to hire people that will fit your organization more than you need to hire the skills that your headcount requires.

Rating fit above skill is the hard part for a lot of people to swallow.  This might be in part because it's a lot more nebulous and therefore is more difficult to measure.  It's not even all that easy to adequately assess the technical skills of a candidate but at least you're capable of measuring something and having some idea about their qualifications.  Anyone who objects to this statement hasn't really tried to write a test or set of questions to give a high level of confidence of the actual practical appliable skills of a candidate and then matched them to the results down the road.

It's kind of the holy grail.  A test that shows you that they'll be able to do the work that you put in front of them at the level that you desire.  Either they test well and implement poorly or test poorly but might implement well, but you'll never know cause you didn't hire them based upon the test.

On the other hand, if you start by attempting to match for culture and personality fit with skills secondary (but not unimportant) then you stand a much better chance of getting where you're going.  If you can figure out that a person is someone who wants to achieve, will work hard, will mesh with your team pretty well and is capable of achieving shared vision with your organization then you should be able to train them to whatever level you need them to occupy.  Granted, the closer you can get to the skills you want/need will shorten this amount of time.

The next question might be to ask how you assess this fit.   It's not easy.  When i go into an interview i have a few questions written down, about 20, that i like to get ask.  Not really necessarily because i like to know that they have the right answer to the question but rather how they tackle the question and the process they use to achieve an answer and then what they come up with.  One of my favourites was discussed in an earlier post about usability.   The two-parter asking the candidate both, 'who in their experience owns usability' and 'who should own usability.'  There isn't a correct answer to this question, there's barely even a wrong answer if you can give me reasoning that makes me believe that you can think coherently about things.  But even in the way that you deal with left field questions will help me to figure out if you're going to fit in.  do you have a good attitude about it, tackle it positively, don't fill your answer with too much bullshit etc.  I know there's a lot of science behind interview process but in the end the assessor, or interviewer, has to figure out the fit thing on their own personal level.

Here's one of the good things about bringing in someone to interview who's not really qualified for a job; they don't hold back.  Generally they have nothing to lose, so they really put themselves out there.  there will be some that are too nervous to make that work but that's the thing with someone who's not qualified, if they can't shine in the interview you really don't want them.  In a lot of ways the decision becomes easier because you only care if they really shine.  In tech, sometimes you have to consider people who are fully qualified but have quietness issues or smell weird or whatever because you're hiring for an odd skill set and your price range is fixed.  That doesn't work with the under-qualified.  If they don't impress the pants off you in the interview you know that they're not going to have the go-gettum-ness to learn the skills to do the job.

I've had a lot of success in hiring for fit and ability to learn for Jr positions.  Especially in QA.  QA's a tough one for finding qualified jr's though.  Generally they only stay a Jr for a couple of years and they won't really be looking during their first job for something else until it's time to move up.  So you have a choice, you can hire fresh out of training, and frankly i've never ever been particularly impressed with the candidates i've seen coming out trained for QA or you can hire someone with an obvious aptitude that you can train.  a person who's shown that they want and need to learn things can not only learn the skills they can learn to adapt to your culture faster, easier, better.

And here's the clincher; I'm not just talking about social/work culture.  I'm talking about culture of quality.  You get to take a receptive, clean (so to speak) mind and imbue them with your quality goals and concepts.  it's a very powerful way to help your team grow in quality the way that fits your quality vision.  Top down to the bottom and then bottom back up.  Having people who are ready, willing and able to achieve shared vision with your rapidly is very powerful and can't be discounted.

In summary, no matter your skill needs, always try to hire for fit but don't discount the under-qualified because longer-term they might be the best choice for the job.

Saturday, June 9, 2012

Hiding in the Corner

So i'm working a new job now, with a new team.  I've been here just over 3 months at this point and while i have a good handle on the culture here, i'm still learning some of the ins and outs. 

This week something happened that demonstrated to me that there are things about the culture that are impacting our ability to be successful and upon noticing these i found myself giving my first lecture to my team.  

Everyone knows that tech folks tend to be a little more socially awkward, on average, than your standard human being.  But that said they do cross a spectrum of personalities and trends for interpersonal communication.  But as is true with any group these personalities come together, combine with the traditions of the organization and form your culture.  The culture of the technology folks at this place trends towards quiet and insular.  My team gets along with each other pretty well on a quiet respectful basis and interacts with the other eng teams similarly.  The other eng teams kind of fit the same description.  It's not that we don't laugh when we're together but it's more that laughing only happens when we're together and not with others at the company. 

This description does not fit the other groups at the company.  As you might expect sales, marketing and client services are a lot more social, interrelate, participate and etc.  That's pretty natural. 

Well fast forward to this past Wednesday where the office admin sent out a couple of emails asking the employees to come in on Thursday wearing red and white or wearing something that says Canada on it.  The CEO became a Canadian Citizen on Wednesday and they wanted to have a little informal surprise celebration in front of his office.  

So the next day i come in, wearing a bright red t-shirt and a white over-shirt.  I enter through the back way so that if the boss is there he'll not see me and start to have suspicions.  I notice that not much of engineering is wearing red or white but i naively assume that at least a couple of my guys would have done something.   I see the first guy and i mock him a bit.  I feel that mockery is a very strong tool for behaviour modification, especially when it's an informal change that you're looking for.   Not too long after that i'm in a meeting room having my morning scrum with my team and not a single member of my team in that room is wearing red or white.  More like, their clothes seem to have been chosen for the opposite because there's not even any random bits of white on anyone's clothes. 

I think about doing a little good natured chiding but then i decide that this simply isn't good enough, that there's a really important point here that they are missing.  So we laugh a bit and then i go into lecture mode. 

There are two reasons why not conforming to work events like this are bad.  We've already talked about how engineering is a little insular in our company.  This is causing some problems.   At least due in part to the fact that the teams in engineering haven't really been building any relationships with the other teams at the company.  This has made it easier for other teams to play the finger pointing blame game rather than the 'let's be a team and figure this out' game.  Relationships are one of the key things that help reduce the blame game.  It's way harder to point finger's when your brain is going, 'Frank's a good guy, he wouldn't do anything on purpose to cause this problem.'   By standing back and not participating in these events it's encouraging the 'us' and 'them' mentality.  There's the people who are cool and interesting and care enough to be involved and the 'weirdos in the corner' who don't. 

The second reason is a little more self serving.  The CEO's office is pretty much in between our two groups.  So i asked the team what they thought he would think when he steps out of his office and looks to the right and sees a sea of white and blue and then looks to the left and sees an ocean of not even remotely red and white?  And better yet, what if he looked to the left and saw a little island of red and white in the blue and grey group and knew that that island was QA.  I'm not saying he'd rain cash down on us or anything but it's really important to stay both in the mind of of your CEO and while in that mind to be thought of positively.

Then my boss showed up not wearing a lick of the right colours...sigh. 
You don't have to play politics to be a good manager but you have to understand relationships, what they do and what they mean.  

I think the team got it. 

Wednesday, June 6, 2012

How Quality Tools Present Value

So during my lunches i'm studying the Certified Manager of Quality/Organization Excellence Handbook in order to hopefully take my test for my CMQ/OE certification in the future.  I'm still in the pretty early stages and it's going to take a while since i essentially dedicate 1/2 hour per day to it.  Today i was reading about Change Management in the Leadership section and I came to a conclusion that, while it can't be generalized over all Quality Tools, i think holds for a large number of them. 

The generalization is that these tools maximize their value mainly for the participants in the creating of the forms and not so much for after-the-fact consumers of them.   I came to this conclusion while looking at an Interrelationship Digraph.   The interrelationship digraph is a method of representing multiple elements of a process and capturing how they interrelate.  Below is an sample of one that i randomly found via google.

sample interrelationship digraph (random)

The thing is, a picture like this probably has some value to everyone who understands it but the value is going to be on different scales.  When i look at many of the tools that make up quality, like the DFMEA, or a Pareto diagram or this diagram the way to maximize the quality of your graph by far is to be one of the participants that are in the room while you're working together to produce the content.  When you sit in a chair and actively contribute to the analysis that produces the diagram you're not only part of the discussion that draws each of the shapes on the diagram you're likely also coming up with some of the ideas yourself.   There's simply no better way to learn and understand than to be involved in the creation of the idea. 

As with any model, it's the reduction of a large pool of contextual and factual information into a simpler form.  But for a person who reviews the document later, if they were in that room, their brain still retains remnants of that context and can bring it back up.  To a person who wasn't in the room, even if they are highly involved in the subject matter, a diagram like the one viewed above loses a dimension.  I think that's a good way of looking at it, it really does flatten out to the 2 dimension you can capture on the page but for a person with a personal involvement to the creation, there's enough collateral information floating around in their thought processes that it really lifts up off the page to become multi-dimensional. 

In the example i've been using, the interrelationship digraph, in the text of my book, they showed how it was utillyzed to come to a number of conclusions about the best process to use in implementing a suggestion system.  As i read through the conclusions i realized that if you stepped away and tried to use the digraph itself to build your argument for the conclusion that it really didn't support it very well.  It pointed in the right direction, for sure, but it was only skeletal support at best.  However, by taking into account the user stories mentioned in the creation of the digraph, it really supported the conclusions well.  The book provided the context to bring the digraph into the extra dimension for the reader.  

So what am i trying to tell you?  Am i trying to tell you that you shouldn't review the outputs of Quality Tool investigations?  Certainly not.  What i'm trying to instill in people, managers especially, is that the higher the participation you put in utilizing these tools does not only result in a higher value for the output of the tool but it also results in a much higher personal value.   You will never get the value as a consumer of this output as a non-participant as you would for being there and contributing.   Don't sluff off those meetings because they're boring and your team can handle being a member, not only do you bring value to the process but you bring value to yourself. 

Monday, June 4, 2012

Vanilla and Creme - the worldwide conspiracy

How's that for a eye-catching title?

I've been looking for a way in my brain to tie this post to Quality directly so that it fits my blog theme centrally but i'm having trouble finding that connection.  It may be that by the time i have finished writing it that i have found a way to tie the two concepts tightly together but then again i may not be able to.   

However i definitely do have a connection to one of my sub-themes. One of the future articles that i'm going to write is around my thoughts to the question, 'what makes a good software tester.'  This article that you're now  reading is really an exemplary example of the concept of  'down the rabbit hole.'   One of the core attributes to a good tester is being observant enough to spot something while testing, whether functional, integration, regression or whatever, that is just off enough to tweak something in the brain that says, 'that's not quite right.'  And then to be curious enough to spend effort and time digging into whatever is off until you have a documented reproducible bug.   It's a combination of both of these things observant and curious.  Without the two you don't end up with the all important reproducible documented bug.  

What can all of this possibly have to do with Vanilla, Creme and worldwide conspiracy you might add?  Well, as with all rabbit hole excursions, it's a little bit convoluted. 

It kind of starts yesterday when the wife and i went car shopping.  Not that the process of arriving at car shopping was even remotely simple but not really important here.  We had settled on a few things and ended up being a 2011 Kia Soul in the colour Vanilla Shake.  Vanilla shake is a creamy beige colour.  It was a fairly stressful day, giving up my wife's first grown-up car as a trade in and spending a heap of money on a new one.  As a result of this i came up with a little trick to maybe help my wife relate to the new car a little faster.  We have friends that name their cars, we could name ours.   A brilliant idea.  It would help her identify with the car faster as well as bringing the whole car buying concept into the realm of a little approachable at a cognizable level (could that really be a word?).   

[aside - Ok, we've just hit our second blog sub-theme here, management and what makes a good manager.  Knowing people, how they relate, accept, process and deal with change makes a good manager.  A good manager generally has some pretty strong knowledge of change management and some of the tricks that help people transform with less stress.  Which probably also highlights the quality concept of quality wholeness (my just made up term). Or, to really embrace Quality and make it successful in your work life, you need to make it part of your whole life.  The same is not untrue of your other skills. /aside]

So my next step was to start her off with some suggestions. Nothing obviously right came to mind but i thought maybe i could look up some words for vanilla or cream in other languages and one of them might sound cool.   i started off by going into google translator, throwing vanilla and cream in there and picking other languages to look at them.  As i hit the languages i already sort of knew myself like french or Spanish i wasn't so surprised that the words were essentially the same with a little bit different pronunciation or accent.  i started to go wider afoot and learned a limitation of google translate (in this case probably a targeted feature).  If i ask for english to be translated into Russian, the response i get back is in the Cyrillic alphabet.  This held true for pretty much ever non-latin character language.  It held the same for babel fish and microsoft translate. 

But by this point i had already looked at over a dozen languages and for each and every one vanilla and cream were the same.  Things only started to get weird when i asked one of my employees from the middle east what the words were in Farsi - um...same thing.    By this time translating vanilla and cream had taken on a life of its own and the naming thing didn't matter at all (see rabbit hole).  A side question had come up and some very fast research showed me that vanilla had been brought back to europe from Mexico and was named by the Spanish.  This helped resolve the vanilla part of the puzzle.  The Spanish named it from a spanish root and they were the route into the rest of the world for the product.  A standard name for it make sense, but cream?  Cream is used by everyone that milks cows.  How could the word be the same?  What kind of weird conspiracy could be at work here?

I wandered a little farther afield, my Russian employee - same thing.  Things got really weird when i went to the Ethiopian guy working in our CS group.  Same freakin' words.  there were few more options but i sat down and started writing out an email with this entire story to my wife.  i was nearing a tizzy with how completely absurd things were.  It simply made no sense. 
As i was writing my email, however, everything settled out to a much more standard form of normalcy. 

My Russian employee came back to me and said that she had made a mistake.  Cream, as in the product that was the fat off the top of the milk is actually known as splipky (my own spelling) and that creme is their word for whipped cream.  That makes much more sense for the randomized fashion that language transforms in our world.  Apparently whipped cream wasn't much of a product in Russia until French desserts came into the picture, bringing their word with it.  That was good enough for me and the mystery, although i am sure there is more digging to be done.  One of the important skills as you become more senior in testing is to learn the law of diminishing returns with respect to rabbit holes.  This one had been interesting and exciting but work has to be done. 

[aside - i did come up with a few relationships between this story and my core blog themes so that's cool.  not the least of which are the thoughts flowing around through my head of the different paths for quality that these words have taken through our language.  Vanilla has managed to travel through our languages essentially intact but cream is a completely different story.  Language will never ever be a reasonable example of Quality, but maybe what it is an reasonable example of why Quality isn't always the only end goal. Cause language is a cool mystery to play with all in its own.  There's usually a reason for it's insanity, you just have to find it and that is the fun part. /aside]

To end my little story, my email about the rabbit hole had the desired effect and we are talking names and some calming down is occurring.  Ironically enough, while we haven't chosen a name yet, on her own, my wife threw out the concept that while she wasn't sure why, bunny seemed like an oddly apt name for the car.  i'm not, however, driving down it down a rabbit hole.