Who’s in charge? Anyone?

Posted on April 5, 2009 by John

Many things about software development are unique, and attempts to apply techniques or thinking from other endeavors lead down the wrong path.  But there are quite a few things that are pretty universal, and I want to comment on a really big one.

Whenever I see a situation that is turning out to be a mess, I always ask “Who’s in charge?  Anyone?”  In a sizable majority of cases, the answer is “Nobody in particular.”  Most often this is because there is an issue, or a root cause of an issue, than spans organizational boundaries.  In that case, there are many people and organizations who are all involved – and perhaps all doing a good job – but nobody is reponsible for an overall positive outcome.  Each involved person would tell you that they did their part and accomplished their defined mission, so what else did you expect?

A specific example of this regards customer satisfaction.  Sales sold the product to the customer, development built the product and provided fixes as necessary, support organization handled calls from the customer – and they all did their respective jobs in a reasonable manner – yet the customer is very unhappy.   How could that be?  Very likely because the customer had to deal with each of those organizations individually, and probably many individuals within each.  From the customer’s view, nobody was in overall charge.

Whenever I want to make absolutely sure that something turns out right, I always make sure that a specific individual is in charge of and responsible for the outcome.  Provided that person is competent, a positive outcome is very likely.

I remember very well a situation many years ago when I was running the development organization at a small software company.  We had recently landed our biggest ever customer and we needed to get them live on our product.  The head of our company (my boss) assembled his staff of department heads, told us how important this mission was, then sent us on our way, each to do our respective parts.  Pretty soon it became apparent that things were not working out well.  My boss then called us back together to chew us out for poor execution.  Each department and department head had done their defined jobs, but none of us were personally responsible for the outcome.  This was a clear leadership failure on the part of my boss, and it was a lesson that has always stuck with me.  If he had appointed one person to ensure a positive outcome (which implies that he also had publicly empowered that person to carry out that mission), it would all have been different.  Of course, any of us department heads could (and perhaps should) have taken it upon ourselves to do this without any further direction, but among a group of busy peers that is not very likely to happen for many reason.  How easy it would have been for my boss to have just appointed someone.

So now I always ask (privately or publicly, as appropriate): Who’s in charge?

Posted in Leadership, People, Why Things Go Wrong | Leave a comment

Have You Failed Lately?

Posted on March 29, 2009 by John

Since we all want to be successful, failure must be bad.  At the very least, if you fail at something, you are obviously not being successful, and maybe it will make others feel that you are incompetent, not talented, or a loser.  Or maybe you want to avoid failure at all cost because it might cause serious financial or professional consequences.

Thus avoidance of failure tends to be a major objective for most people.  This has unfortunate consequences, because success is not the avoidance of failure.  Success is the achievement of a great accomplishment, and great accomplishment often is accompanied by risk of some degree.  And risk by its very nature entails the possibility of failure, even for the most talented.

Preventing failure should not be our highest objective, because being excessively risk-averse will cause us to always take the safest path, which also will reduce our chances of big successes.

Note that I am not advocating excessively risky behavior.  Indeed, best practices (such as agile development) emphasize risk reduction – but not its outright elimination.  You want to get the odds in your favor, but if you are afraid to act unless you are virtually guaranteed to avoid failure, you won’t ever accomplish much.

And there is another positive side to occasional non-fatal failure:  It is one of life’s best teachers.  When things go right we can all be happy but it hard to say what one learns from it.  But when things go wrong, one usually learns a lot.  I know I certainly have – every failure that I have been a party to is etched into my mind, along with what I will do differently from now on.  It has also convinced me that I can survive and carry on to try again.  Many self-made entrepreneurs have gone from rags to riches – and back to rags – and back to riches again. They are not afraid to try and thus risk failure, because if they do fail, they will just try again, usually with increasing chances of breakthrough success.

One of my favorite questions to ask job candidates during an interview is to tell me about some of their failures.  If you haven’t failed, you haven’t tried anything significant.  We’d be a better and stronger organization if we all operated with more boldness and less risk-aversion.

Years ago, a friend of mine had a big failure.  Equipped with a Ph.D in EE, he designed a digital electronic device that was very novel at the time.  He started up a business to build and sell the devices, and he was so confident of success that he ordered enough parts to make several thousand units.  Unfortunately, his business was a failure because he was only able to sell a few units and he was left with the expense of all the unused expensive parts.  Being desperate to cut his losses, he took out an ad in a hobbyist magazine to see if he could sell some of the leftover parts.  Much to his surprise, he sold all of the parts very quickly.  So – he ordered more parts – and they also sold out.  Today, many years later, he owns a business that grosses several hundred million dollars annually.  That would never have happened if he hadn’t failed at first.  I wouldn’t mind failing like that.

Posted in Execution, Leadership | Leave a comment

Why this blog?

Posted on March 29, 2009 by John

I started my career as a programmer at IBM and spent the subsequent 3 decades developing software and managing its development at 4 different companies in 3 different countries.

During that time I have learned a lot and have had several major revelations.  At some point I will reflect on most of them here.

But the one that led to this blog was an incident 6 years ago when one of the managers who reported to me came to me with a problem that he was having.  I started working through it with him, starting at the beginning and exploring it as a new problem.  It suddenly occurred to me that I had faced (and handled) a very similar problem earlier in my career, and did really need to start from scratch in this case. Now we all rely subconsciously on our accumulated life’s experiences to guide us in our personal and professional lives, but usually do so in a very informal, ad-hoc manner.

It occurred to me then – maybe for the first time as a coherent, conscious thought – that during the course of my many years in software development that I had indeed accumulated quite a body of knowledge on the subject, but just how much I really didn’t know.  At that point I decided to start writing down everything I knew, as a catalogue and perhaps even as a test.

A few subjects and pages came very quickly, but what I hadn’t expected was the snowball effect: one subject led to another, and soon I was nearing 100 pages.  Doing this was a fascinating experience that I would recommend to anyone, in any field, at any point in their career.  I started simply, just creating a Word document that I did not publish.  However, I did bring copies along to the interview in which I got my current job, and while I don’t know for sure, it might have been a “tipping point” factor that led to a job offer.

Over the past few years I have been so focused on my job and other life activities that my paper on software development has just been quietly spinning around on my hard drive untouched and unread.  A recent re-read was interesting, showing just how much I had learned in intervening period – life being an endless quest for and accumulation of knowledge and experience.  So instead of going back and rewriting the paper, the second edition is now appearing in the form of this set of online articles.

My motivation for doing this?  In any life situation, newer and younger people are coming along all the time.  Like young(er) people everywhere, they lack the wisdom that comes from long experience.  Some don’t think that they need it, some think that they don’t but don’t know where to find it (classes and books are great but have their limitations), and some don’t even know that they lack the knowledge.  In every case, every new generation of leaders tend to make the same mistakes over and over again until they, too, accumulate “the wisdom” (if they are lucky).  So my motivation is to pass along what I have learned and concluded during my time in the business, in hopes that it helps even just one person.

Meanwhile, I will continue my life’s journey of continuing to learn.  By the time I am 800 I will probably know most of what I would like to know – or perhaps my horizons will just keep expanding.

Posted in Personal | Leave a comment

Catalyst for Change

Posted on March 9, 2009 by John

There’s turmoil in the financial markets and lots of uncertainty in the world right now.  The world economy is in a major recession. The media is full of dire news.  As a result, our customers are spending less.

I don’t know where this is all going or where things will end up, but I do know at least one thing for certain:  We are not in a “business as usual” situation.

So you can react in any of several ways:

A. Ignore what is going on and hope that things will just stay the same, because there’s nothing you can do about it anyway.

B. Panic.  Brace for the worst.  Be depressed.

C. See the changing circumstances as a catalyst for change, and as an opportunity.

Which category are you in?

I suspect most people are in category A, while some are in B.   I wonder if anyone is in C?  I am.

I love old sayings because they are a distillation of hundreds of years of human experience – the “wisdom of the ages,” so to speak.  So here is one that might make you think that C is the right place to be:  “Necessity is the mother of invention.”

If things are going along well enough, there is little motivation for change.  Change is always difficult.  It’s much easier to come into work again today and do what you have always done.  But sometimes it takes external force to cause change.  If gas prices remain relatively cheap, there will never be enough motivation to ever get away from the internal combustion engine.  None of us like to pay more for gas, but unless circumstances cause the price to go up, nothing will change.

And so it is with the economic circumstances.  It’s less obvious what needs to change, but we should look upon the current situation in the world as a catalyst for change.  Here some:

  • Find more efficient ways to do what we still need to do. Adopting Agile development methodology comes immediately to mind.
  • Do things in completely new ways or with a different approach. One example: for our product documentation, instead of having technical writers creating every word, they become organizers and editors of what others produce.  Both employees and customers know a lot about our products.  Another is to start offering more of our products as a service, with a monthly subscription fee instead of a high up-front capital cost.
  • Stop doing stuff that really isn’t needed. Do we really need to put ALL those little enhancements into our existing products?  Oh yes, customers have asked us to do this stuff, but how much of what we are doing will result in any new sales or will be the make-or-break factor in retaining existing customers?   Do you really think that someday you will be working on Release 27.0 of the same product you are working on now?
  • Do what’s REALLY important. Instead of doing a lot of little things, we should concentrate our resources on a few projects or activities that truly WILL make a difference.  Like something new that we can sell, something that dramatically increases the synergy between our products, something that many customers will REALLY notice.

The current economic circumstances are painful but will cause change.  Those changes can be either good or bad.  The most successful people and organizations will not just react to them, but will find ways to turn them to their advantage.

Posted in Agile, Execution, Leadership | Leave a comment

Theme for the New Year

Posted on January 2, 2009 by John

We’re starting a new calendar year, and with it come big new challenges.

Much could be said about this. Over the holidays I came across a brief article from Time entitled “Myth of the Fearless Entrepreneur” that makes a great starting point.  Consider this:

In a survey of people who had founded growth companies, only 12% attributed their success to an “unusual or extraordinary idea,” but 88% reported that their success was due mainly to “exceptional execution of an ordinary idea.”

This doesn’t diminish the value of creativity to our business success – I think we could use a lot more creativity – but “exceptional execution” would be a great theme for the coming year.

Often times the winning – or losing – margin in a competitive event is razor thin, and thus just “a little more” is enough to make all the difference. The same is true in how we design and build our products, and how we treat our customers. If our products could be just a little faster, a little easier to use, a little easier to install and configure – it might be just enough to cause that big prospective customer to make a “yes” decision for our company instead of for a competitor.

A lot of products and services today seem mediocre or indifferent, or at least some aspect of them. For example, that MP3 player (not iPod) I got recently seemed well made, except – the user interface is maddeningly illogical or unusable in places, and the instruction manual was written by someone whose mission did not extend beyond providing a superficial and cursory set of instructions. Doesn’t make me very happy, or think very well of the company that made it.

Think of the products and/or services you provide. Are they as good as they could be? If you were secretly to watch a customer use what you have built or deal with your organization (or even you) – would you smile or would you cringe? Think of things that went wrong in 2008 and what caused the problems. Was it just bad luck, or could those problems have been prevented by just a little more planning, a little more double checking, or a little more effort in the right place? Here’s a little example:

At one point last year I attended a Live Meeting where there were over 100 attendees. The presenter logged in to Live Meeting at the same time as everyone else, then wasted 15 minutes of everyone’s time while he fumbled around trying to get everything to work right. In a culture of exceptional execution, that presenter would have joined the meeting early to make sure that the audience had a smooth experience.

Great products and great services can only result from exceptional execution.

Let’s have an exceptional 2009!

Posted in Execution | Leave a comment

Second Best, Tomorrow

Posted on January 2, 2008 by John

That certainly doesn’t sound very appealing, does it?  But it might depend on the context, and it actually might be a pretty good idea in software development.

During WW II, when the British were under attack by German bombers, they desperately needed a way to detect incoming hostile aircraft as early as possible.  They knew that it was possible to bounce radio waves off of objects like airplanes and measure the distance somehow, but there were a lot of details to be worked out to make this work well.  The British scientist who directed the R&D work on this project, Robert Watson-Watt, became frustrated with the slow pace of development work because he knew that time was of the essence – if they didn’t get something working fast, even if it wasn’t perfect, it soon wouldn’t matter at all.  So he told his team to aim for “second best, tomorrow.”  The goal was not to create something second-rate, but to build something that worked even if not perfectly, and have it done tomorrow, and not take months or years to come up with the perfect solution.  Once a basic system was working, it could be used right away (time to market), and could be enhanced over time if necessary.

The product was radar, and I think Watson-Watt’s thinking is relevant to what we do here today.  While we are not building software in response to an emergency wartime threat, we are working in a highly competitive world where speed to market is of the utmost importance to increasing our revenues and overcoming our competitors.

In my opinion, the best way to create software rapidly and economically is to use agile development.  Agile is a topic that other writers have already mentioned many times in this forum, including Jeff’s recent post.  There are many aspects and methods of agile development, but the overall methodology that I like the best is simply iterative development with cross-functional teams.

The key concept of iterative development (sometimes called “iterative and incremental development”) is the use of a series of fixed time-boxes, each one of which results in something that works and is incrementally better than its predecessor.  Once you get something working, you can demonstrate it, get feedback, refine it … and even ship it, if good enough.  We don’t have to worry about the approach of enemy airplanes, but I’m pretty sure our competitors are hard at work right now on their own good ideas, so the sooner we can get our products to market, the better.

Watson-Watt actually used time boxes for his radar development team.  He gave them fixed amounts of time to reach incrementally better versions of a working product, and then put each version into production.  This might sound like a reckless disregard for quality, but it’s not.  It just has to be good enough to work and do its intended job, without any frills or extras.  Watson-Watt’s crude term for that was “second best” – we might chose different terminology – but the concept is the same.

Posted in Agile, Execution, Process | Leave a comment

Cross-Functional Teams (CFTs)

Posted on April 20, 2007 by John

We’ve had a lot of great discussion here recently on the very important topic of iterative development and I suspect we are far from exhausting the topic, but let’s shift gears for a moment.

One of the topics I mentioned in my article on Development Effectiveness was that of cross-functional teams (CFTs), also called “project teams.”

Definition: A CFT is a group of people who work together from start to finish to produce a piece of software, and whose make-up consists of one or more people from each of the key disciplines required in the development process.  A typical CFT consists of:

  • a business analyst (or equivalent title)
  • one or more engineers
  • one or more QA people
  • a systems analyst (for organizations that have that role)

It could also include full or part-time participation from a variety of other disciplines such as UI design, database design, tech writing, and others.

I think that CFTs are a natural and essential part of doing iterative development.  Why?  Iterative development requires short cycles, so you can’t afford to have work done sequentially where each functional group throws things over the wall to the next functional group in line.  Because a CFT is a cohesive group that takes a piece of development work from start to finish, it is ideal for doing work quickly with good intra-group communication – just what is needed to make iterative development work.

Further CFT characteristics:

  • Each CFT has a leader.  This could be anyone on the team who has good leadership abilities.  In my experience, is it usually the business analyst but sometimes one of the engineers.  This person is essentially the project manager for the work being done by the CFT, and takes overall direction from the project manager of the whole project.
  • The CFT should be the right size – not too small or too big.  If you look at my earlier post on What Makes a Great Team I argued that 5 people was the best team size.  A team twice that size starts to lose cohesion and might be better separated into sub-teams.  Thus a large project likely has multiple CFTs.
  • The entire group works together from start to finish.  The QA people don’t join late and have to catch up, and others don’t go on to start new work leaving QA to clean up a mess.
  • The team works closely together.  Everyone on the team needs to understand the requirements, the solution, and how the solution will be verified and validated.
  • The CFT is semi-durable. A good CFT needs a lifetime long enough that the team can form some cohesion.  I think a good CFT will last for at least the duration of one release cycle, and possibly longer.  If it lasts longer, it would be good to allow some members to move to other teams and some new people join, while preserving a core.
  • CFTs are flexible.  A CFT might be created for the express purpose of building a large feature, and/or it might be given a set of features to work on over a series of iterations.  When it is no longer needed, it can be dissolved and the resources reassigned to other CFTs.

I’ll make one other assertion, perhaps controversial: When you are on a CFT, it is the group to which you owe your highest loyalty.  You are no longer part of the engineering team, or the QA team, or the BA team.  You are a member of a team of people whose goal is to produce some specific software, and any other mission or instructions are secondary.

I don’t know to what extent CFTs are used in other places, but I’ve used them before at other companies with great results.  Within the my development organization we have been using CFTs to some extent already, and are adopting them more completely on some new projects.  We’re off to a good start so far.

Posted in Agile, Organization, Process | Leave a comment

Can We Iterate?

Posted on April 11, 2007 by John

In my last article I asserted that iterative development is one of the key practices that will improve development effectiveness.   I speak from personal experience on this, and there is abundant literature that supports this claim.  So why aren’t we all jumping on this methodology?

At one company I worked for it was the practice to capitalize most major new development projects.  That is good because it allows us to postpone the expense of doing a significant amount of our development work until the results of that work are being offered for sale, thus making development more affordable.  But it is also bad because it requires us to prove that we are developing an asset for which we will be able to recover the development costs in the future.  Part of the burden of proof is called “technical feasibility,” and appears to be a significant impediment to doing iterative development.

Here’s a layman’s explanation:  Capitalization is really a “deferred payment plan” for the cost of software development. Without capitalization, we would have to incur all of the cost of developing a piece of software up front, but we would not start recovering the cost of that development until after development has completed and we start selling the product.  We are out of pocket for the cost of development until we have sold enough of the product to cover our cost, which would reduce our ability to do more development.  When we capitalize development costs, we are deferring the expense for a period of time, usually 3 years, during which time we expect to cover our costs by selling the product.  This is a bit like buying something on credit.  If you pay cash, you incur all of your cost up front but your benefit happens over a period of time thereafter, while if you buy on credit you are paying for the product at the same time as you are benefitting from it.

But there is a price to be paid.  To capitalize a project, we have to be very sure that the project will pay for itself – a bit like a credit check.  Part of this requires a strong business case that makes us confident that the project will create enough revenue to more than cover the development cost, and the other part is proof of technical feasibility from development, which means that we can actually build everything needed to support the business case for the estimated cost.  To capitalize, we are required to show that we expect to cover the deferred cost.

This is a problem for iterative development because in true iterative development, not everything is known up front. Typically the requirements are discovered and validated over a series of iterations as a release emerges.  If you are doing iterative development, many things are not known precisely so how can we ever declare technical feasibility?

I believe this can be done, although doing so might violate (slightly) some of the principles of “pure” iterative development.  Here’s how:

We need to have a starting point, so we use a base set of requirements as defined by Product Management to define a release. Subdivide the release cycle into a number of iterations, perhaps 1 per month, so a 1 year release cycle would have 12 iterations. The goal of the first several (3?) iterations would be to prove technical feasibility of the given requirements and thus would be non-capitalizable.  Those iterations would be used for fleshing out the requirements, estimating, prototyping, and even coding. At a certain point, all of the base requirements should have good estimates and technical feasibility is declared. All subsequent iterations are then capitalized.  This violated a tenet of true iterative development, but I think preserves the spirit.

I would combine this with one more practice, perhaps controversial. The committed plan for the release (i.e. the set of requirements that are covered by the estimates that are used for technical feasibility) should not consume 100% of the development capacity.  This is reasonable because any responsible plan should include some contingency.  If the committed plan is conservative because it includes contingency, it not only assures that the committed content can definitely be delivered on budget, it should also provide some “extra” capacity that can be used for new things that arise during the iterative process, whether new requirements or whatever. and this would provide a lot of the flexibility that iterative can take advantage of.

So I think that we can do iterative development within an environment that requires capitalization.

Posted in Agile, Process | Leave a comment

You Can’t Test Quality Into the Code

Posted on January 24, 2007 by John

… yet it is surprising how much we still rely on testing as our primary defence against defects.

To produce quality products consistently, the process must also have the objective of removing all defects before test entry. The objective of testing then becomes to verify and validate the product – not to fix its defects.

This is not just a theory or an idealistic view of what might be possible in the distant future – this is something that actually can happen.

At one point in my career I worked on a very large project at another software company.  There were almost 1000 people on the project, and it resulted in a product that has been widely recognized as being one of the most solid and reliable pieces of software in the industry. Yet the QA group within that was less than 1/10 of the total staffing. How could that be? Most projects we are familiar with typically have a QA group that is at least 1/2 the size of the programming group if not larger.

There were many contributing factors, but a key one was this: Every programmer was expected to produce essentially defect-free code, leaving only a few tasks to QA such as system test, load test, stress test, and any testing that required complex data, scenarios, or environments. As a programmer I remember very clearly having every one of my technical designs critically reviewed by my peers and all the resulting comments and corrections, the time well spent on unit test, the glee expressed by some of my programmer “friends” whenever they would find a defect in my code during a code review, and the mortification of receiving a defect report from QA once. While this might sound like a shame-based culture, it really didn’t feel that way at all. It would be better described as one of competitive excellence in which everyone felt a keen pride in their work and felt a strong desire to produce really good products.

I see no reason why we can’t make our culture more like that. And if pride in one’s work isn’t enough, one additional thing we have going for us is that our software really matters – most of it is used in hospitals by medical people who are working on matters of human health, sometimes life or death – so what better motivation could there be to produce quality software than that?

So you can’t put the quality in by testing – it has to go in from the beginning, starting with good designs and code that programmers are confident in and proud of as theirs.

Posted in Execution, Quality | Leave a comment

What Makes a Great Team

Posted on September 25, 2006 by John

Years ago, when the PC first came out and for some while thereafter, it was actually possible for a single programmer to create a meaningful software product. But those days are long gone now. I don’t know of any non-trivial piece of software today that was produced by a single programmer. Modern software products require thousands of man-hours or more to produce, and one person doesn’t have enough time to do this. Thus good software today is always produced by more than one person, and the only effective way to do this is through those people working together as a team.

But you can’t just put a bunch of people together on a project, call them a team, and automatically expect teamwork. So what is needed to really make a team work?  This is probably one of those questions that you either have never really thought about, or you just assume that you know the answer.

So if you think you know the answer, write it here: _____________________________

If not, or even if you think you did, read on.

If you were to research this topic, I’m sure you could find a lot of different answers. I haven’t done this, but I did come across one definition that I really liked.  It lists 3 key elements that are needed for a great team:

  • Clear definition of roles.  In an effective team, each member needs to know exactly what his role is so that he knows how to interact with the rest of the team. If everyone on the teams does their own thing, the team is not necessary.  Furthermore, it is essential that each team have an undisputed leader.  An interesting socialogical study with children showed that if kids were put into “teams” and told to solve a problem, the usual result was that one kid would dominate and the others would act disengaged – not an effective team.  But the study also found that if roles were defined when the team was established, the kids then understood what they needed to do and worked to reinforce the given structure.  Interesting.
  • Small team size.  There have been various studies on this, but many sources suggest that 4 – 5 is the optimal size for a team. If your team is much larger, it starts to lack cohesion.  It’s probably no accident that basketball and hockey teams are about this size. While baseball and football use a larger number of players, it is debateable whether or not 9 or 11 players are really working together with complete awareness and support for all their fellow players.  If you must have a large team, it probably needs to be sub-divided into more manageable groups of … 4 – 5 people.
  • Presence of an outside threat.  While it isn’t always the case that teams can only exist under competitive situations (for example, people might form a team to do charitable work, or they might work for the government), there is a lot of evidence that it is competition that really makes a team come alive. (There are other types of outside threats that could motivate a team, such as fighting for survival, but let’s hope that that is not one of the conditions that face us in software development!)  In our case, the outside threat clearly must be our external competitors, and thus a key motivation for our teams should be beating our competitors. Without being aware of our competitors, do we really know how well we are doing?  Consider this: If you went out for a run, do you think you would run faster if you were in a group of competitors or all by yourself?

The above thoughts pertain not only to our technical teams, but I think they apply even more so to our management teams.  Think about the team(s) that you are a part of and whether or not they have these characteristics.

I’m sure that great teams might have a few other characteristics; anyone have any thoughts on what some of them might be?

Inspiration and some data for the preceding came from an article entitled “How to Build a Great Team” by Jerry Useem.  The original article is no longer available, but you can read a copy at: [link TBD]

Posted in Organization | Leave a comment