About Me

My photo
Related, I keep most of my code here https://github.com/rdammkoehler

Saturday, May 3, 2008

Granularity Optimization

[I've been working on this entry for quite a while, I've just finally decided to publish it.]

When planning a project it is critically important to find an optimal task size. There are several reasons behind this. First and foremost, any estimate is destined too be inaccurate. Additionally, it is impossible to effectively assign more than one resource to a task. Lastly, if tasks are defined to specifically they become brittle and must be remade more frequently. 

With respect to inaccuracy, an estimate made early in a project, when the details are not well understood, or made under pressure to simply publish a schedule is commonly incorrect by as much as 50%, sometimes more. If the project team iteratively re-estimates tasks the accuracy of the estimate might progressively improve as the start-date for the task approaches. However, the nature of estimating tells us that we are simply making the best educated guess we can about the effort required to achieve some goal. None of us are able to predict the future with consistent accuracy and precision. So, if by using an iterative re-estimation technique we can manage a accuracy of +/-10% on our estimate we're doing pretty well. But, if a task is estimated at 30 days (240 hours) and we are wrong by 10% we have at least three things to consider. 

First, if we finish 10% early (27 days) we may create idle time for the person(s) working the task. Idle time can lead to one or more dangerous effects; gold-plating being the most dangerous. As they say, Idle Hands are the Devils Tools. There are also resource utilization considerations, but they are less relevant and, at least in my opinion, irrelevant. 

Second, if we finish 10% late (33 days) we may slip a delivery or otherwise impact future tasks which has a cascading effect on the overall project. This is a more pronounced problem to be sure. I think the consequences should be obvious to the reader.

Third, in either case we over pay for the task in terms of money; for early delivery we pay for idle time (or the Devil's work), for late delivery we pay for the additional work.

In order to make effective assignments of resources ideally we have one task for one worker. This is true in most kinds of work. Those who choose to employ paired programming, for all intents and purposes are using two people in the place of one. Often times large tasks are created and assigned to a team member; e.g. Build sub-system X. But as time passes and sub-system X is not complete, more people may be assigned to do the work; to help. This is not an effective way to bring a task back in line with the schedule. If one team member is in the midst of developing a solution for a task, and suddenly more team members are added to the task, everyone's productivity slows. The original team member must slow down in order to communicate the work he has done, the work that remains, and all the considerations that he may have made to this point. The new team members can't make any progress until all this information has been communicated. Finally, all of the assignees must now communicate with each other about what has been and will be done. 

Worse still, if too many people are assigned to a task you can exceed the maximum partition-ability of the task. Think two workers, one shovel. If an when you exceed the maximum partition-ability of a task, the result is that the 'extra workers' must be idle in order to prevent conflicts. Furthermore, communication about the task must also increase which degrades all worker performance. The next result is that the task takes even longer to complete than originally scheduled.

The final concern about task granularity is over analysis and the creation of too many too detailed tasks. This becomes a problem simply because changes in the design and architecture specifics of a project will cause these tasks to become invalid. They must then be discarded or replaced. If this occurs too frequently the overhead of managing the task list outweighs the advantages of the details. This will impact team performance creating 'start delays' of tasks and also potentially increasing the amount of waste within a project; especially if some of these detailed tasks are taken from a backlog to fill gaps in the utilization of team members. 

Finding the balance between too much detail and not enough is a significant challenge for a project planner; it is one of the distinguishing characteristics of a great planner. Keeping in mind the impact of inaccuracy, the maximum partition-ability of a task,  and the risk of over analysis while planning can help smooth out a task plan while still providing a solid time/effort management mechanism.

Partitionability

Partitionability describes the degree to which something can be functionally decomposed. The partitionability of a process, algorithm, or function is determined by the refinement of a design. The partitionability directly correlates to the number of distinct atomic tasks required to complete the implementation of the process, algorithm, or function.

Partitionability is used when estimating the man-time requirements for implementation. For example, some functionality X requires 22 man-hours to complete. X can be decomposed into 5 distinct component parts (its partitionability is 5). Optimally, X can be completed in 22 / 5 = 4.4 hours by a team of 5 developers. Stated formulaically;


To = Tm / P


Where To = Optimal Time, Tm = Man-Time, and P = Partitionability



Good, Fast, and Cheap. Pick Two

The designer's Holy Triangle or, as I call it, the Magic Triangle. There is another set of considerations beyond the Magic Triangle. Call it a corollary triangle, I call it 'the technology balance'. The idea is that within a technology project there are three considerations; The use or creation of new and innovative technology, getting the job done (or doing business), and correctness (in the academic sense). I'm aware of other considerations but I feel these three are the the most contentious considerations. 

When technology projects start there is often a lot of conversation about what technologies should be applied. Often, teams choose to use newer or more cutting edge technologies in favor of older more stable technologies. Part of this is facination with the new, part of it is a hedge against those things we don't like about the old. All of these are valid and reasonable determining factors for technology selection. 

Another observation is that technology teams, given enough time, will argue endlessly about the correctness of any given solution. There is always debate about what is 'the right thing' in tool selection, pattern application, hardware configuration, design and planning approaches and just about any conceivable facet of the project. 

Always in a project there is a push to deliver the product on-time and within the budget. We've all heard the complaints about schedule driven decisions, right-to-left scheduling, scope creep and defect management; contributors to concern about delivering on-time.

We go round and round these three topics in every project. Depending upon the focus and dicipline of the team and it's leaders we may not complete a project because of one or all of these considerations. By being aware of these contentions and managing our activities in accordance with the balance that must be maintained we can be more successful with our projects.