Sunday, September 23, 2007

Charting Uncontrolled Territories

Charting Uncontrolled Territories

Arunabha


The Statistician looks at the expressionless face of the High Priestess of Process and Quality and sighs. If only he had heeded the well meaning words of warning of the Junior Priest and the hordes of PLs, he would have been enjoying his day, reading a couple of eccentric emails that was part and parcel of the Software world and getting ahead with his own research. New generation network technology allowed you to get connected to your work from anywhere. So what if this company had engaged him as a week-long consultant to review their ‘High Maturity’ initiatives. He could have stamped his ‘Reviewed OK’ on all the endeavors and lived happily ever after. Instead he had tried to be honest, and hence was here, looking at the austere expression of the fanatic female who sat in front of her.

“You mean to say our approach has some loopholes?” Priestess asks.
The Statistician sighs. If the general opinion of the company is anything to go by, this promised to take a long time.
“Well, I’d say the initiatives have been innovative to say the least …,” he begins.
“Yes,” the Priestess is pleased. “We are a level 5 organization. And Organization Innovation and Deployment is a Process Area that we satisfy.”
The Statistician steals a glance at the Junior Priest seating beside him, with a guarded expression which suggests, “I told you so.”
“Well, let me elaborate on that. I’d say the innovative use of statistics in your measurement program stretches the subject a bit … well, sometimes alters the shape of the subject altogether.”
“I am not sure I understand you,” the High Priestess looks at him with a challenging expression. Unmindfully she straightens a framed certificate on her desk, which proclaims her to be one of the company’s internal Six Sigma Black Belts. Statistician or not, she knows to deal with data too.
The two young men sitting in front of her wince on seeing the certificate, for entirely different reasons. The Statistician has the foreboding of strenuous argument against the ominous signs of little knowledge. The Junior Priest remembers the sleepless nights he had spent in office while preparing for the High Priestess’ online Black Belt examination.
“What about the Statistical Process Controls on defects that we have in place? I heard you have some reservations about them …,” The Priestess has championed the Process Improvement Graphically group - and she is the Leader of the PIG Heads in the organization.

The Statistician steels himself for a battle. This is one area about which he feels strongly and is not about to give up without a fight.
“Well, madam, let us take your System Test Defects. You want to plot the number of System Test Defects generated per day in a c-chart.”
“Yes,” the Priestess is confident that she will be able to generate a c-chart at the click of a button using MiniTab.
“Well, madam, Let us consider what c-chart is all about. It deals with the number of defects found in a fixed number of units of samples – where several such samples are observed. There are two constraints. One, the sample size of each sample is fixed and two, each unit of the samples is similarly produced.
“Now, tell me something. How doe we define sample size here? We are testing the same application… and hence the sample size for all intents and purposes is 1 – which in itself is an anomaly. The sample does not vary from day to day, so tabulating defects for multiple days makes no sense … all the defects belong to the same sample”
“You are making things complicated,” the Priestess says.
“Okay, let us suppose that these are different units of development and independent, tested separately. The first unit is tested on the first day, the second on the second day and so on…Firstly, this is not how business takes place
The samples themselves are different, and does not match the basic assumption of the c-chart that these generated samples are identical. So comparing their defects on the same c-chart again makes no sense.”
The Priestess looks long and hard at the Statistician. She has lost the train of argument long back, and what simmers inside her is boiling anger that the Junior Priest has not done his job in warning her about what was coming. Suddenly, drawing up from inspiration imbibed in the Bullet Proof Manager Training, shifts her delegation gaze towards the Junior Priest, who winces.
“Please explain it to him,” she says in a freezing tone.
Junior Priest swallows and looks apologetically at the Statistician.
“Actually, when we were asked to generate control charts by hook or … I mean … well, for generating control charts, we thought of the c-chart. And the sample size in this case is the number of test cases a testing team executes per day.”
The Statistician thinks hard.
“So, how big is the testing team in general?”
“Well, six to eight members in some projects. More in some, less in some. In the same project it may vary from day to day too, because the testing team handles multiple projects per day,” the Junior Priest says.
“So, you mean to say that all these variable number of testers come up with the same number of total tested test cases each day? To keep the sample size constant? Is that possible? And what do you compare it with the next day? Where do you get the next data point? From a different application?” the Statistician is bordering on the sarcastic now.
“Well, no …,” the Priestess interjects. “We expect the code to be debugged and then tested again. We plot the defects of the test cases tested on the debugged code…”
“What?”
The Priestess shrugs. “The point is to make the control chart show improvement. So, in a debugged code, the number of defects go down, hence showing improvement.”
The Statistician holds on to the table for support.
“So, you test the application and plot the number of defects as the first point in the control chart. And then the application is debugged and the number of defects from this application is plotted as the next point.”
“That’s right,” the Priestess sighs in relief. At last she has been able to make this academic understand the business. She looks unfazed. Unless one has practical experience, academic knowledge amounts to nothing. After all it is she who is planning on submitting a paper to ASQ on the Realistic Applications for Process Enhancement using Statistics (RAPE Statistics in short).
“In that case you are tampering with the basic assumption that the populations are the same. Forget the peculiar sample size. The populations themselves are different. You are changing the population from which samples are drawn on the second day. How on earth can you plot it in the same control chart?”
The Junior Priest avoids the eyes of the High Priestess this time. He makes a show of speaking on his cell phone. The Priestess fumes. The argument is against all she has worked hard for, and what infuriates her is that the whipper-snapper spewing Statistics in front of her seems to utter many things which she has never heard of, yet things which he implies are axiomatic. No doubt some upstart out to demonstrate his statistical skills.
She decides to end the tirade on Control Charts.
“Now, what about the other measurement initiatives?”

The Junior Priest sighs. "That's another blog," he thinks.

No comments: