Kanban is an alternative to sprints, for contexts in which the team is persistent across projects and it makes sense to let them manage their work apart from the projects that they contribute to---e.g., an ops team that supports the organization via a help desk and ticketing system.
You could certainly think of the sprint board as a kanban board with a finite lifespan, but I don't think there's any benefit to doing so.
There would be a benefit to thinking that way. Instead of doing a ridiculous sprint planning, instead just keep pulling from the backlog. Rather than predicting what you will get done (and invariably failing to predict well, or using developers own predictions as their accountability and a way to pressure overtime), perhaps just see what you get done instead. Then instead of planning, actual work could get done. Anyone needing estimates can look back historically and compute number of tasks getting done and extrapolate against the backlog.
By keeping the backlog prioritized at all times, and pulling from the backlog instead of a 'sprint backlog' - it avoids priority inversions. Planning what will be your highest priority, that possibly changes mid-sprint, is a priority inversion.
Kanban works well in all situations where sprints work well. Instead of trying to predict in a planning, you just do the work instead. Any predictions are still the same based on looking at historical data and comparing to the backlog, just less waste involved and less subterfuge to try and get developers to work overtime (I'm not aware of any sprint planning that really takes into account error bars, so the prediction/planning is BS & management planning porn, nothing more)
You can't really do this without having some idea of how long each item in the backlog will take. If feature A is twice as valuable is feature B, should A or B come first in the backlog? Depends if A takes the same time as B, or four times as long.
(ignoring various other considerations about how you prioritise, and just focusing on delivering value here)
This is why rough estimation is part of the planning process in XP. The developers estimate stories so that the product owner can prioritise them. The estimates don't need to be precise, just roughly right relative to each other, to act as a denominator.
Since the developers are only pulling from the top of the backlog, the product owner doesn't need to prioritize the whole thing, just enough to keep the team busy. That means the developers only need to estimate stories which the product owner is considering for prioritization.
You could try to do this continuously - a developer needs a story to work on, so they ask the product owner, who picks a few candidates, then asks the team for an estimate on them, then selects one to be next. But that would involve everyone interrupting everyone else quite often. So typically, you get together once a week and do this until you have enough stories to keep the developers busy until the next get-together.
You can tell, roughly, if you have enough stories for a week by counting how many size units of stories the developers get through in the previous week - that is, your velocity.
Sure you can keep your backlog prioritized at all times. Prioritization can be net benefit, or can be net ROI, the choice is yours! If you find out that item A will take half the time, perhaps that will boost its ROI and change its position in the queue. If you instead want your prioritization to be strictly based on expected return, then it does not matter how long it takes.
Nothing in this way of defining priority is really any different between scrum & kanban. Perhaps the only key difference is when work is allocated to be done, scrum often focuses on 'how long will it take' to avoid "assigning too much to one sprint". Hence, a low ROI item could certainly be fit into a sprint simply because of time constraints, even though it might be really low on the overall ROI list (thus a priority inversion where the wrong thing is worked on simply because it fits an arbitrary time period).
> You could try to do this continuously - a developer needs a story to work on, so they ask the product owner, who picks a few candidates, then asks the team for an estimate on them, then selects one to be next.
If you have your backlog prioritized (or at least deep enough you can pull work for a while), then doing this continously is cake. The developer pulls from the top of the backlog. When a new task or a story is introduced, it is stack ranked. When and how often you introduce new stories in kanban is really up to you. So it need not be a constantly disruptive process. Typically new stories are looked at on a daily basis, which allows for a lot of responsiveness. At the same time, if knowledge changes about a story, it's priority rank is changed right then. It does not take very long to do stack ranking.
> You can tell, roughly, if you have enough stories for a week by counting how many size units of stories the developers get through in the previous week - that is, your velocity.
Tl;dr, velocity makes some interesting assumptions and uses bad math
Velocity does not take into account error bars. This is bad math! Each story has an error bar of give or take 30% (assuming it is well estimated with Fibonnacci). EG: An '8' could be a 6.5, or could be a 10.5. Yet, somehow by adding up the expected size of stories together the overall accuracy increases.. This is an oddity and bad math. Velocity should always be reported with a plus or minus, yet, it doesn't.. Somehow all of math behind propogation of error is assumed to cancel out. Beyond bad math, velocity only says how well you did last sprint, and "past performance does not guarantee future results." In short, velocity assumes that people can estimate. The only tasks that will have reliable estimates are ones that are very similar to something you have already done many times (eg: how long would it take you to brush your teeth with this new electric toothbrush? You know how long it takes to brush teeth with a convention toothbrush, so your estimate would be pretty good). But let's say, how long will it take you to learn Electromagnetism? Cool, you just learned that, how long will it take you to learn Calculus III? If you've taken Calc I and II, you would perhaps have something of an idea, but you're still to be really way off on estimating that.
> Since the only purpose of velocity is to know roughly how many stories to prioritize in the backlog, none of this matters in the slightest.
Given error bars are not taken into account, the proposition is that velocity is incredibly inaccurate. Presumably, most of a backlog is prioritized (otherwise a high priority story could be at the bottom, leading to priority inversion). I thought the purpose of velocity was to know commitment size for a given sprint (how many stories to bring into a sprint). Given that the inaccuracy of velocity is not taken into account, nor the compounding of the uncertainty, knowing how much to commit based on an inaccurate value leads to a lot of "failed sprints".
Re: prioritization, how you evaluate priority is completely up to a team. Anecdotally, Maesk ran agile and did not take into "cost of not completing a story". By that measure, there were three stories that were worth over 100x all other stories combined. Yet, they delayed deployment and efforts of these stories in order to "fit" in other stories that had low effort. Logically, with that large of a benefit, they should have stopped literally everything else and gone all in on those 3 stories. If you're really interested, I can try to dig up the youtube talk that went into this anecdote. The speaker in that video also noted that the data they looked at suggested the benefit of stories usually was magnitudes more than the level of effort. Hence, at times it almost does not matter the level of effort since the return is a better proxy for overall value.
Prioritisation can be whatever you want, scrum does not say what 'priority' means. That can be as silly, or as complex as you want. There are many inputs to priority. In practice there seems to be an over-emphasis (near-obsession) over 'cost' rather than value provided. A good example of this are items that have deadlines to them. At some point, the "cost" of implementation is not the biggest factor, yet it often gets the most attention and time allocated (vs say, dollar cost of not implementing per day)
My point with velocity is that it is inaccurate and using it will be just wild guessing. It's like saying, it's 65 degrees out - "cool, I'll wear a t-shirt". But, it's 65 degrees out plus or minus 30 degrees. As an engineer/scientist, we should already know that any number should always have error bars reported, otherwise it is meaningless. Similar to the temperature, you can look outside and infer the temperature through other means. It is likely then that "velocity" is noise compared to the true means we use to know if we have enough in the near backlog or not. Worse yet, velocity is often used as a cudgel to force overtime and declare a sprint "failed," even though velocity is so uncertain (yet is not treated as an uncertain value).
>There would be a benefit to thinking that way. Instead of doing a ridiculous sprint planning, instead just keep pulling from the backlog.
The purpose of sprints is not to divide up the work---this is a common misunderstanding. The purpose of the sprint is to set incremental goals based on incremental learning. So instead of setting a goal to create the complete software system from the beginning, you set a goal that can be accomplished within a fixed timeframe that people can make reasonable estimations in and that will test some of your key assumptions about the software.
Simply throwing everything into a massive backlog and grinding through it indefinitely is a recipe for failure, since there is no opportunity to reflect on the larger picture: the team is too busy focusing on the assembly line and not paying attention to the product. And of course you can still have "priority inversions" in kanban, but unlike the finite sprint, you may go months before you realize what you've done wrong.
As I said, there are appropriate times to do kanban, but the situation you describe is not one of them.
> So instead of setting a goal to create the complete software system from the beginning, you set a goal that can be accomplished within a fixed timeframe that people can make reasonable estimations in and that will test some of your key assumptions about the software.
In Kanban you can do all of that, except faster. Instead of waiting for the arbitrary sprint boundary, you can immediately evaluate an item that has landed in production and immediately re-prioritize the backlog based on any learnings. That would be compared to: we will deploy this in sprint A, then in B we will see the results, and then in sprint C we will work on our updated items. The latter is a many week delay compared to assigning tasks in strict sequence of "ship/measure/adjust".
In the worst case, you ship at the beginning of a sprint and then instead of working on the next priority thing (that you learn from having shipped), you instead keep working on the other sprint items. At that point, it could be waterfall, working on items that you now know are pointless, but you still have to anyway because it was planned some time prior to do so.
Kanban doesn't forbid regular retrospectives. You can still pause frequently, reflect, and update your process accordingly.
The purpose of sprints is to try to estimate the completion of a release -- and as progress is made the drift from that original prediction. Sprints in theory let a manager estimate a team velocity. Together with estimates for the task in a release, that, in theory, gives one the ability to predict when a set of features (a release) will be complete. I have yet to see this actually work, but it could just be reflective of where I have worked.
>The purpose of sprints is to try to estimate the completion of a release -- and as progress is made the drift from that original prediction
This is not the purpose of sprints. The sprint is a fixed increment of time intended to encourage incremental development of complete software systems. The idea is that while people don't have an intuitive idea about how much work will have to go into developing a complex application, they do have an intuitive feel for how much work they can accomplish in a couple of weeks. So instead of asking, how much time will it take us to implement a fixed set of features, the team will be asking how many features can we implement this sprint?
>Sprints in theory let a manager estimate a team velocity.
A manager is not a consumer or producer of team velocity. The team itself is the only legitimate consumer and producer of team velocity. Agile is about team self-management.
This idea of asking how much can be implemented in a given sprint is an important clarification. Though, this can be impractical and/or sub-optimal for many contexts. The one-size-fits-all approach of scrum, where so often it is a company or a new person joining and just declaring that "we'll do scrum" now is where things can get de-railed because it is not the universal way to do all development.
So, to paraphrase the distinction, instead of death marching for a large set of features without any feedback, the team decides "we won't do more than 2 weeks of work at a time before we ship & deploy some feature."
(I really wish I could find the link) A recent hacker news article looked at this situation and noted how this does not really allow for extensive R&D where there is no deliverable after the end of two weeks. An example was Google BigTable which had no deliverable at all for 8 months. Yet, the investment into BigTable lead to a very huge pay-off, and the point made was that there was no good way to break up the R&D required into two week deliverables, and the ROI of that work was far more than most any 2 week increment.
Overall the issue for scrum is context, it has its places where it works well - but it is not the one-size fit all solution that so many places try to make it out to be. For example, two weeks does not always allow for a lot of rework, experimentation, architecture, and R&D. A team can choose to make a deliverable be "the results of R&D", is this still fungible thinking and is not actually delivering a release.
Personally I would be a lot happier whenever a scrum master and/or product owner/manager that run a scrum team could clarify exactly when scrum is a bad fit (this feels like the expert-beginner problem). If a person knows when scrum is a bad fit, they can at least try to mitigate those problems and/or lean on other development methodologies. It is almost like silo'ing into one programming language and using only that language despite it not being the best tool in some situations (and also of detriment, various alternative techniques are not learned; eg, sometimes a bash script is the right tool to use, and not branching out and trying to solve some problems with a full blown programming language leads to a lot slower of a solution. To illustrate this, Amazon once had a team spend time to find all phone numbers on its website. This took a long time and the program was complex and never worked well, a very competent shell scripter got wind of this and solved it in 15 minutes with a bash one-liner. This became an interview question at Amazon at one point even, in part to check if a candidate will realize that their language of choice might be a bad fit for the problem at hand)
Breaking a story down into smaller stories is not the same as incrementally developing a complete system. This is the difference between reductionism (breaking a large pre-defined project into smaller pieces for execution) and holism (always delivering a complete system, but incrementally extending the functionality as the team learns more). Sprints provide the time increment for the latter. Note that a sprint increment may differ from project to project. But most teams find 2-3 weeks to be a good increment.
Even in a reductionist approach, though, sprints provide a regular opportunity to review the appropriateness of the breakdown or the validity of the parent story.
You make a really good point on the importance of having a complete iteration at the end of the sprint.
In my experience, the fixed time increments have always felt too constraining.
I think a team can accomplish a similar goal of having regular incremental progress in the form of complete systems by relaxing the fixed-length sprints requirement.
The alternative that I favor is Kanban with defined releases. A release, as a collection of features, is a complete system, but it doesn't necessarily have to be finished in the same time that the previous release was. A larger release may take six weeks instead of the smaller previous release that only took one.
You make a lot of good points, thank you for sharing your thoughts.
General reply to those who say, "but you can have planning and retrospective in kanban!":
Sure, you can. But it requires discipline to actually do it. It's very easy to get wrapped up in a task that runs over, and when you multiply that tendency by everyone on the team, the idea that it will actually get done regularly in practice is optimistic, to say the least. And if you are doing it regularly, together as a team, is that really so different from a sprint?
Conversely, the notion that you can't change course mid-sprint is simply wrong---you absolutely can. The concept of sprint commitment is an acknowledgment that there is value in completing something workable, even if it's the wrong thing, as opposed to staying constantly in a state of WIP. But if it's obvious you're doing something no longer needed or technically feasible, nothing in scrum says you have to complete the sprint as planned.
That incremental goal is not mutually exclusive to kanban. The difference is whether you are stating at the beginning what goal you will hit, vs just working towards that goal. It's the same thing, one creates pressure to hit inaccurate estimates, the other just marches towards the goal.
> Simply throwing everything into a massive backlog and grinding through it indefinitely is a recipe for failure
Why is that? Scrum has the same backlog.
> since there is no opportunity to reflect on the larger picture
Why is this the case? Kanban often says that you talk about retrospective items any time they come up. You then even can re-prioritize such items right when needed, meaning you solve todays problem today, not tomorrow. Retrospectives in scrum can often come too late (and personally I've never really seen them work well, too little, too late, and often compete against 'business' priorities that always win out).
> And of course you can still have "priority inversions" in kanban, but unlike the finite sprint, you may go months before you realize what you've done wrong.
Kanban is less susceptible to priority inversion. You keep the highest priority at top, it is evaluated more often, and you pull from the top. Why does this mean that months can go by? Why is that any more the case than say scrum where you are encouraged to fix priority for weeks at a time?
> As I said, there are appropriate times to do kanban, but the situation you describe is not one of them.
I think the burden of proof is on you to state places where scrum uniquely bests kanban. The statement that kanban is best for places where ad-hoc work comes in constantly is a case where scrum spectacularly fails. For regular project work, kanban can be a great fit (I'd say universally better than scrum, since planning is just complete BS. Kanban is often little more than scrum without the planning sessions and with far more frequent prioritization, and you get to keep all of the same retrospective & estimation tools)
I answered your general point in a separate reply, since others brought up the idea of continuously planning and retrospecting in kanban.
But I'm confused by this statement:
>planning is just complete BS
To me, planning is where design meets reality. Planning involves several activities, but probably the most important is prioritization. I'm curious why you think that prioritization happens more frequently in kanban, where the convention is to simply take the next task off the stack. Are you saying that every time someone takes a task off the kanban backlog, they independently re-prioritize the entire stack? Or do you have regular re-prioritization sessions with the team? If it's the former, that seems like a lot of overhead and opportunity for conflict and misunderstanding. If it's the latter, how is that really different from regular sprint planning? Because surely, as part of the prioritization, you would also consider things like estimation of difficulty and risk.
... for contexts in which the team is persistent across projects
and it makes sense to let them manage their work apart from the
projects that they contribute
Here I disagree only in applicability. Kanban can be successfully used in a single development project if the team maintains prioritized tasks based on risk and/or dependencies. To wit, a development team with sufficient maturity can identify and plan their work, address same in the requisite order, and have confidence that the probability of success will increase as each task is completed.
Prioritization is an output, not an input, of the scrum process. That's the whole point, to engage the user early and often so that information about the desired functionality can be derived from experience and not just speculation, and then fed quickly back to the team for iteration.
So sure, if you:
- release usable software
- collect feedback from stakeholders based on the release, and
- re-design and re-prioritize the perceived work remaining
on a regular basis as a team, then you can use Kanban for a single project. But it will look an awful lot like sprints, because it's hard to do all that on a continuous basis, it would be a lot of overhead and context-switching to do it daily. On a project of any size, you'd have to do it all periodically. Say, every couple of weeks.
You could certainly think of the sprint board as a kanban board with a finite lifespan, but I don't think there's any benefit to doing so.