Showing posts with label agile. Show all posts
Showing posts with label agile. Show all posts

Tuesday, October 13, 2009

The feature -> task conversion

The inspiration for this post came from a question to my previous post Kanban vs. JIRA, regarding the problem of how to use JIRA to model Kanbans pull based approach to tasks handling. I will try to describe my view on how to address this problem, which isn't a so much a issue with JIRA, but a fundamental task management challenge.

The root of the problem lies in the conversion of the customer/product owners input requirements/features, into a operational set of tasks which can efficiently be processed by the team, and lead to a system with the functionality needed by the customer. This problem manifests itself very different in a Scrum and Kanban oriented project.

In a Scrum project the activity of processing the features contained in the backlog, into processable tasks isn't really addressed. There is a lot of focus on defining and prioritizing features, which is a very important activity, but this doesn't in itself lead to a effective task pipeline (push oriented) or task pool (pull oriented). This isn't necessarily a problem as this conversion activity can just be added to the development process. It can on the other hand turn into a root cause for a lot of problems such as breaking sprints, inablity to finish tasks satisfatory, inferior quality etc, if ignored. This is exactly the case for most of the Scrum projects (and development projects in general) I have encountered, that is, the tasks used to manage the sprints doesn't efficiently reflect all the software development aspects needed to deliver a finished system.

In Kanban the input to the process is a backlog of directly processable tasks, and Kanban therefore requires some kind of mechanism for 'feeding' this task pool. The major benefit of Kanbans approach to task processing is that the requirement of only working with well defined, operational tasks become glaring obvious, and a failure to achieve this will quickly break the Kanban process, because of the teams inablitity to finish tasks. The problem of ensuring that the operational tasks used in the daily work reflects the full set of work need to complete the required system remains, and in must be addressed through activities outside of the Kanban process.

A very good tool in helping with the development and management of the task backlog/pool is defining a number of attributes and task relations which can be use to qualify tasks before thay are added to the 'processable' task pool. These task attributes can also be used by the team 'pulling' the tasks from the backlog to determine which tasks would be the best to work on at a given point in time. The problem is that the task model can become quite complexed as the number of attributes and tasks grows and a good issuetracker, like JIRA, is therefore very useful in managing the model, ensuring consistency and allowing focused views on the aspects of the task model need in a given situation.

Note that I'm not talking about a simple project management tool here, as the tool needs to  include all aspects of the feature -> task lifecycle. Because of this, I will referrer to an issue pool which is the full set of features, bugs, actions, tasks, etc. The task pool, or backlog is the 'processable' task view we can use when accessing the issue model from a task management role. This is relevant, when we work in a Scrum, Kanban or other task oriented context.

Here is my list (very verbose) list of issue attributes I find helpful in nurturing a good issue model together with the relevant JIRA field used for storing the information
  • Summery: This is the 'human readable' key to the issue. A precise and relatively distinct summery is essential in being able to generate efficient issue and task overviews as found in the Scrum and Kanban oriented cardviews, or different tool filters. The summery should be updated as the scope and content of an issue becomes clearer or changes.
  • Detailed description(Description): The full description of the issue. Will properly start out empty and grow as the content of the issue becomes clearer. The ability to include rich content in the issue description is impotent to avoid the need to distribute the issue information over more than one tool.
  • Importance(Priority): Here I prefer a very course grain scale, like the 5 levels found in JIRA. This is partly to allow the team performing the work to have some freedom in choosing which task to work on next (pull task), partly because other issue attributes will contribute other dimensions to the issue model.
  • Iteration(Fixed version): All projects have some kind of time partitioning, like version, increments, sprints, releases, deployment etc. I prefer to use these iteration containers (together with the importance attribute) for modeling issue priority or severity, instead of a very detailed relative prioritization between all the backlog tasks, as described in Scrum. I think that customers (product owners) find this more coarse grained priority system more natural, both because the detailed priority between 2 different issues isn't necessarily a concern to the customer, but also because the iteration containers directly reflects the very tangible deployment and release milestones, which are the real concerns of the customer. It should be possible to placed issues in multiple versions with different granularity, eg. sprint/minor version/major version.
  • Estimated (remaining) time to implement (original and remaining estimate): Always a nice thing to have on a issue when considering when to start work.
  • Issue relations(Links): A major concern not addressed in an one-dimensional backlog, as found in Scrum, is the dependency of tasks on one another. The definition and maintenance of this issue model aspect is a critical input to the team choosing which task to work on. Without a good issue dependency model, it will be difficult to finish tasks, and critical paths will not be addressed.
  • Software domain(Component): An indication of which part of the system under development the task belongs to should be available (if applicable),  eg. database, GUI, business layer, test, analysis, etc.. This can be used to pull tasks according to the competences of the available team resources. 
  • Business domain(Component): Issues should be qualified according to which business domain aspects they reflect (if applicable). This can be view as a high level categorization of user stories/features/use cases.
  • References(Hyperlinks): As issues have relations to other parts of a projects information, eg. requirement, test, stakeholders, etc, this references should also be added to the issue. A very generic way of achieving this is by hyper links in the description.
  • A 2 level breakdown of the issues(Issue-sub task): This reflects the two faced nature of issues. The first (and top level) part is the initial feature oriented aspect of the issue, which is also the part interesting to the project stakeholders, like the sponsor, product owner, users, executives, etc. The second part is the task breakdown of the issue, which is the operational tasks the team can work on. Other project management systems operate with a more complicated 3 or 4 level hierarchical task model, but here I again prefer the multi dimensional model, which much are better at modeling the many aspects of the dynamics in a project.
If the issues are broken down and enriched with all the aspects listed above in a efficient issue tracker, a good foundation for a operational task pool should be available. Of course the road to a correct breakdown and growing of tasks isn't trivial, and reflect the teams increasing insight into the nature of the project. But as I mentioned in the beginning, a efficient and rich model is crucial in the development of a efficient task management system.

So how do we pull task from this multi dimensional issue model. Well, a very pragmatic way of doing this is to define a number of views/filters, which can be used as prioritized mini task pools, from which to pull tasks. An example of such a filter setup could be:
  1. First all Critical issues should be handled.
  2. Secondly all analysis task from the next sprint.
  3. All analysis task from the next version.
  4. All test specifications/acceptance criterias for the next sprint
  5. All task contained in this sprint
  6. All major issues.
  7. .........
This could be flavored with with prioritizing implementing full features, selecting task based on the competences of the free resources, task dependencies, etc.

So in conclusion there is no simple answer on how to produce a efficient task queue where there is a trivial answer about which task to work on next. Gaining insight into the multi-dimentional task model is on the other hand a crucial activity, which can lead to the success or failure of a project, depending on the projects ability to implement a working feature -> task mechanism. Here the rich modeling and filtering capabilities of a efficient issue tracker is critical, especially for more complexed projects. Still, this is secondary to the availability of a competent team, who can do the actual analysis and management work needed to do the actual feature -> task processing.

Sunday, May 11, 2008

Death by architecture

One of the less mentioned dangers of a waterfall or engineer type approache to software development is the focus on the initial activity of defining and implementation the architecture of the system.

Usually the critic of the 'old' plan-drive methologies runs along the lines of inflexibility and inability to cope with the unknowns of real world software development. This issue is often adressed in 'mature' organizations by assembling a group of experts in the discipline of IT system construction (IT architects), and tasking them with defining the high level technical aspect of the system.

One of first task addressed by the architects is usually to define the non-functional requirements, like performance, availability, scalability, maintainability and so on. These input are used to define the architecture of the system. this includes selecting the technologies to use for building the system. The architecture will often include such things as layer models, security models, persistence models, domain model and a lot of other models for crosscutting concerns.

If the company defining the architecture also provides the platforms for deploying the system the architect will also be tempted to include a lot of enabling technologies in the architecture. This will include a potent cocktail of enterprise, high-availability, cluster, SOA...... middleware, platforms and frameworks which would (in theory) solve most of the non-functional challenges.

This all sounds very clever, but this approach can in the real world be devastation to a project progress. The problem is that the initial focus of the development project will be centered on creating the system architecture, instead of producing visible progress.

If you are lucky you will have access to people understanding the technologies making up the architecture. But more often you will a team with a more generic competence profile, and all the middleware and enterprise platform will remain badly understood black boxes, continually impeding the development of the system, while the promise result gained by using ready-made enterprise software never materializes.

During this phase the trust between the customer and the software development team will deteriorate, because the development team is unable to demonstrate any progress and and understand of the problem domain will not increase significantly. The moral of the development team will also go down, because the members are also unable to see, where the project is going.

Many project have never really gotten through this phase, and have died struggling to define and implement the 'right' architect. If you are able to fight yourself out of the architecture desert, the scars on the customer relationship and the team moral will quite deep and make the work ahead more difficult.

So, how should one avoid this?

As always, the solution is to use a more agile approach. This include following the agile best-practices of:
  • Keep it simple: Do not introduce complications needed because of non-functional requirement, until you can't postpone them any more.
  • Make it work, make it right, make it perform: Avoid focusing on concerns, which wouldn't have an effect until later in the project. You should off course not totally ignore later stage challeges, but the development should always be driven by the shortterm gains.
  • Evolutionary development: Grow the system from a simple seed, nursing and growing the application on a continuous basis.

Wednesday, January 30, 2008

SCRUM vs. JIRA

In my role as JIRA evangelist, I have been struggling with making JIRA function efficiently in our departments SCRUM centric environment. The challenge here is that the central SCRUM concept of a sprint is very difficult to model in JIRA.

JIRA's way of partitioning the project into versions might at first glance appear as the same thing as sprints, but a deeper difference between SCRUM's and JIRA's view on software development lurks below. Where JIRA's concept of versions focus on the progress and roadmap of the application, the sprints in SCRUM acts as containers for sets of tasks, which the development team can work on. This indicates a fundamental focus in the two approaches to software projects:
  • JIRA models an application focused approach to software development, with a task management aspect.
  • SCRUM models a task based approach to software development, which hopefully reflects some kind of application development.
In an ideal world this would be two sides of the same thing, and is actually one of the premises SCRUM is build on. But in the real world cracks start to appear in the illusion, that SCRUM is a good way of managing application development.

The difficulty of JIRA to model SCRUM is just one of a number of symptoms that SCRUM is a less than perfect tool for application development. Even though pretty much all of our development teams have at this point succumbed to the euphoria of SCRUMs blessings, no one has really been able to consistently finish 'good' sprint. Usually the sprints ends with a lot of lose ends, and a very unclear picture of what was achieved compared to what was planned.

This is most commonly contributed to a number of more or less unjust factors, like external intervention, unclear goals, imprecise estimates etc. In my opinion all these annoying 'factors' are more an indicator of how fragile the SCRUM sprint is , because of all the implicit requirements the sprint has to its environment. These sprint prerequisites include:
  • Stable team: The central task processing engine in SCRUM is the team. If the team isen't 100% dedicated towards completing the sprint, the sprint breaks. Causes could be the allocation of people for external crisis handling or changes to the partial allocation of team members.
  • Stable sprint backlog: The task set to be processed in a sprint is defined by the sprint backlog. If this changes the sprint breaks. Causes could be the introduction of emergency tasks or just general reprioritizing of tasks by higher powers.
  • No external dependencies: The SCRUM name implies that the team gangs up for completing the sprint at hand and without assistance from others. If the team isn't able to complete the task defined in the sprint alone they have to draw on external resources. If the external resources do not act fast the sprint breaks. This is a very likely result of having external dependecies, which will be outside the control of the SCRUM team and probably have other priorities.
  • Clear goal: One of the cornerstones of the sprint is the team being able to share a common goal and everybody works hard at reaching this goal. If the sprint goal is unclear, much of the 'uniqueness' of the sprint concept disappears, and the sprint 'degenerates' into normal iterations or increments. In my experience the set of tasks very seldom sums up to a common goal, most of the time the sprint backlog is defined by the priorities in the product backlog, ad-hoc bugfixes, available resources and other unrelated tasks.
  • Small, well estimated tasks: Even if the list of tasks in the sprint backlog remains stable, the ratio between the task estimates and the actual work used for completing the tasks has to be pretty stable to be able to predict the velocity of the team. If the velocity of the team is unpredictable the sprint breaks. A lot of the tasks needing to be solved in software development are very hard to estimate individualy. Some examples are bugfixes, analysis work, integration tasks, etc.
  • Visual results: One of the valuable outcomes of good sprints should be clear and visible results of the sprint. If this isn't possible the value of the sprint diminishes significantly. But a lot of the work needing to be done in a software project are enablers for the development of the application, and as such doesn't produce any direct result. These types of tasks include: Framework creation, design and architectural work, documentation and specification work, education, etc
All of the points mentioned above are aspects of software development one should strive to improve. They all contribute to better and faster application development. But in the world I live in, very few of the points are stable enough to act as the foundation for a development method. SCRUMS needs every single one to work perfectly to succeed.

The result is that everybody is currently walking around crying SCRUM as the answer to every project management challenge, but nobody is really seeing it work correctly in their project. Any critical approach to improving this situation has been paralyzed by the blind followers of the SCRUM movement, who have hijacked agile development the last couple of years.

Thursday, November 1, 2007

Tales from the history of documentation, chapter one

One of the main battles I have been fighting during the years, is the fight against inefficient documentation. Inspired by these challenges I would like to tell a tale of the epic documentation struggle through the ages.

Once upon a time important information was stored by carving runes it into huge stones. This way of maintaining information was thought to work very well by the elders. It had worked for many generations, the documents would last for many, many years, the rune stones never moved, so the information should be easy to find and it was ensured that only the wise elders was allowed to contribute to the documentation.

But for some reason the writing on the stones weren't used that much by the common people, they mostly preferred to talk to other people with the relevant knowledge or just write notes in the dirt. When the common people were asked why they didn't use the stones, they answered, that it took too long to go to the stones and read the information and the information written there was out-of-date and wasn't really relevant for the common people.

The elders response to this was that the common people needed more teaching on how to use the rune stones and more rules regarding rune usage should be developed (and of course written on other rune stones). Another improvement would be the manufacture of even more stones to close the holes in the existing information and more effort should be put into updating the rune stones.

People were sent on rune stone courses, massive amounts of rune stones were produced, but the usage of the rune stones didn't improve much.

Then one day a young enthusiastic fellow approached the community with a new media for maintaining documentation. It just consisted of thin sheets of wood (he called it paper) and you could just write on it with small pieces of charcoal. He said that the paper was a major improvement to the rune stones, the paper was much 'easier' to used than the rune stones, everybody could contribute to the documentation, you could easily update the writing on the paper and you could bring the relevant documentation with you to use it where you needed it.

The elders listen to the young man and told him this paper thing looked very nifty and he was very welcome to use it, but it could of course not be used for the more important information. After all, it wasn't as safe as rune stone, it could burn, rot, bleach or blow away. Anybody could write on paper, which would be dangerous, who would control the changes. It would also be impossible to maintain the archives the rune stones provided, when everybody could just pick up a document and take it with him.

The young man was pretty disillusioned by this lack of vision from the elders, but agreed to follow the elders decision to only use the paper for things considered less important by the elders.

The years passed, and surprisingly enough paper gradually started to replace the rune stones. Apparently the intangible concept of 'easy to use' and the possibility of everybody to contribute was found to be 'better' by the common people. Soon all documentation was stored on paper and the rune stones retired to museums and parks.

End of chapter one