Agile Metamorphosis Anti-patterns

Agile Metamorphosis Anti-patterns

Over the years I have had the opportunity to observe the progression of Agile teams as they form, mature and eventually disband (yes, I have seen that happen too) due to natural evolution in technology and markets. Having revisited and mulled every impactful change over varying state of minds, seeking different answers at each visit has broadened my perspective on the changes a development leader needs to address as she / he attempts build and grow an Agile team – a goal certainly worthy of the effort.

Setup of an Agile software development team is an endeavour unlike other domains. Certainly, the dynamics and challenges involved in software development work – no matter how mundane the problem space, are underappreciated (if at all) by most organizational leadership. This could be out of ignorance or a matter of convenient ignorance.

The goal of this write up is to simply put in one place content which could help leaders (note the conspicuous absence of any people management title here) make the navigation through this journey a smooth and rewarding experience. Of course, everyone’s comments are encouraged - nay solicited; all shared relevant learning will assuredly go into improving this article.

The negative approach in the article is studied. It is easy to see why if you give it some additional thought. A lot of teams today are in the process of ‘transforming’ to Agile – I would rather advocate use of the term ‘metamorphosis’. The difference in the terms appears nuanced but critical for 2 reasons - 

1. A metamorphosis is irreversible.
2. A metamorphosis takes a certain process, the right conditions and an unmitigated amount of time.
3. A transformation implies simply a change in process, or an approach change in the way we do things, but a metamorphosis implies that the outcome is a fundamental change in team / org working that takes into consideration people and behavioural aspects.

There is a rudimentary understanding of the ‘whys’ around the processes and ceremonies of Agile, but I did not find discussions or opinions around the intermediate team ‘states’ during an agile metamorphosis and the challenges therein.

Most people would go looking for Agile related changes and challenges only when they are in the middle of it – perfectly late to start looking, but critically important to find the right answer. All my opinions will be peppered with examples – can you identify someone you know? :)

At first some groundwork -

Understanding that Agile changes are ‘hard’ on the team and harder on the work flow is an important first step towards setting up agile. Another important factors to consider, study and identify are :

1. People dynamics in development team –

• close friendships and loyalties.
• varying personality types - who is a leader and who is a follower.
• Who is the most competent developer, who is the ‘glue/maven’ developer who works well with others and is well liked and respected.
• Developer who is individually brilliant but anti social – this will also be the developer who is most resistant to change.
• Be very mindful of the developer who does not want to share knowledge – these silos are created for cynical reasons and are generally well thought out. I will write more on working with differing personality types in subsequent posts.
• Dev team members who are uninitiated to the Agile principles in practice. These are the leader’s challenge and potentially their best allies in the journey – pointer to the leader: remember that trust is a two-way street.

2. stakeholders in the wider audience who need education on Agile principles:

§  Do you have senior managers whose only take on Agile is ‘faster delivery’?

§  Do you have stakeholders who think Agile is limited to a ‘new way’ the development team works?

§  Do you have a senior stakeholders who keeps asking for dates instead of what a feature does? Is not concerned with dependencies that are yet to be identified or features yet to be fully defined?

§  Do you have a stakeholder who believes that the production server is the actual assets created? ( yes – this did happen with me). If the reader is unable to fathom why this is a concern, know this :- you can’t see the problem, because you are the problem.

3. Stakeholders whose buy in is critical to the Agile team’s success – 

§  These are senior managers, customers who will not only fund your work, but also be the primary users and /or advocates of the features/ changes / products that the team has put out for use.

§  These are the people who have placed their credibility and prospects on the line based on their trust in team’s ability to deliver.

§  They want change to be delivered and have the vision to share your team’s enthusiasm.

§  These stakeholders must always be kept in the loop and managing expectations (around time, cost, quality, function) is central to everyone’s success.

4. Identified colleagues and stakeholders who have an antipathy to Agile –

The antipathy will largely be veiled. It is visible through persistent negative feedback. Although there should be an independent post for locating the antipathy – here are a few quick examples. If any of these matches your situation - congratulations – the team has identified the ‘antipath(s)’ in their midst:

• Do you have any members who come up with very logical reasons for how an idea / process change / POC will not work, but are lacking new suggestions on what would work instead?
• Are there developers who do not inform of delays, impediments or problems unless asked explicitly?
• Are there members who come up with detailed post-mortems of misses in the goals at the retrospective? Or worse – during a presentation to stakeholders?
• Does your immediate manager know of deliverable problems (no matter how trivial or temporary) even before you go and inform them?
• Then here are a few of my all-time favourites:
• A PM won’t sign off on beginning of work since we don’t have a fully fleshed and signed ‘final version’ of a feature from the BA.
• A BA who will not UAT validate partially built feature or will wait till completion to point out what was not part of the requirement.
• A development team member, who warns that they are “not responsible for delivery timelines” as the requirement specs are not signed off, even when the specific sprint story assigned, is sufficiently fleshed out and all impediments cleared.
• Non-development team members giving everyone the run around when gathering requirements / learning current system behaviour.

5. C-suite stakeholders:

• It is important to understand that there could simply be features that the team needs to work on priority simply because these features / changes are what one needs to earn the confidence of the right people. At any time during the Agile metamorphosis, the leader will need strong advocates to the changes she / he makes. As non-intuitive as it may sound, it is useful to build some additional tech debt if required to push this through. It may also not be the best course of action in terms of fixing / improving the technical debt, product performance or even end user traction – it is simply a means to build confidence, support and continue to allay concerns around visibility, control and monitoring – all too common in the C-suite when process changes are suggest. It helps to remember that they have a macro, financial cause and effect view to consider, even if they themselves are not myopic in their understanding of the long-term advantages.
• Know the most important senior members – help them look good. Meet their commitments to outsiders at first. If the expectation is unrealistic or overly ambitious be courageous in informing them with data to validate your claim. Risk alert – expectation management is important. Small initial wins may lead to the stakeholders carrying higher confidence in the team’s ability to deliver than necessary. They may set difficult / impossible goals based on unfounded projections – the best way to mitigate this risk is to talk about the easy and difficult achievements upfront and honestly. (I advocate this approach although you could run the risk of personal cost – in the bigger picture you will certainly have dodged a bullet)
• Understand the inner cliques / clubs and steer clear of antagonizing any of the ‘connected’ members, they may or may not help but certainly possess ability to do harm – most such people are unencumbered by concerns of common organizational betterment. We unfortunately must live and work with them. I would recommend ‘the secret handshake’ as a good quick read to equip yourself better.

Conway’s law - Agile team’s process & product architecture

In terms of Agile anti patterns, it is tempting to lay blame on the internal politics that restricts a truly elegant solution from coming to fruition. It needs to be understood that not only does the software solution not have to reflect the organizational structures towards ownership if that is not conducive for flexibility to change and reuse; there is infarct an opportunity to lay out products that would gradually build momentum for org structures to be reconsidered. 

A systems architecture is a good place to begin the engineering goals for an Agile journey. It not only ensures organizational buy in, it also guarantees that goals are shared after being given proper thought. The inertia around change in work structures is less palpable if the change proposed is at a systemic level. Any future team structures, or process changes are tied back to the architectural goals whose advantages are already espoused and agreed upon.

Here are some examples but note that I am not implying these as best practices. These are merely examples to put the interplay of architecture and Agile practices in in better perspective:

• Being able to move towards micro-services from layered legacy code. This would mean that the monolith or shared services need to be deprecated. This would certainly mean working across multiple development teams in charge of layers. This is a sure-fire way of getting into turf wars with development teams. PS: be aware of hidden issues, if some teams are willing to hand over without an argument.

• Being able to write a shared function that spans departments. This would imply that the ‘ownership’ of the feature will need to move out of individual department silos, as the shared function comprises of process that touches upon multiple systems (e.g.: finance, inventory, loyalty programs, external payment portals, etc) which lie within different department domains. This is also a potential for riling up senior employees over control and identity. Deeply entrenched roles and identities are hard to break. The best way around it is to simply involve each of the layers as stakeholders as interested parties in the deliverables who will also provide validation sign off. - of course, more on challenges around ‘definition of done’ in future posts.

• A better product architecture to support organizational layout for the future. One may be able to take advantage of a start-up lean structures by building ideal Agile teams – teams that are self-organized, coherently multidisciplinary; thus, self-sufficient in delivering increments of usable products. A dose of pragmatism here: all changes must be measured against a scale of impact, cost, need (all together and independently on each as well) and not against the scale of an idealistic system – an easy pitfall for engineers.

• Some handy examples that the reader may find relevant to their situation:
• There is no point in trying to break down an existing system into layers / micro services if the software in question is already in maintenance mode. E.g.: a layer on top of a systems that runs on a mainframe.
• There is hardly any point in trying to modernize a critical system that has incomplete documentation, lacking / retiring ‘engineering experts’, missing source code, or incomplete regression testing in place (all examples culled from personal experience). It is cheaper to rebuild the old system/solution over time and look to sunset at the earliest.
• The business impact compared to the volume of work in upgrading the system is minimal or non-blocking. (think WSJF in backlog grooming)
• There is too much resistance and politics around proposed changes. The knowledge is silo-ed with a few individuals who see withholding knowledge as the only means to protect their jobs. 
• At this juncture the leader should pick their battles based on -
• How long do we have? – if there is even a timeline. Can I revisit this over time?
• Do we have all impediments identified and cleared?
• Do I have committed resources to effect the changes I want to see?
• Do I have alignment from ‘powers that be’ for prioritizing the work?
• Criticality of the system, and an assessment of the steps required to make changes
• A brutally honest assessment of their position in organization.

Know thy problem domain

Particularly true about any Agile methodology, is that it is malleable to specific project needs, so far as the core values and principles are not compromised. Each project has a distinct character and related set of requirements. It is essential to understand the problem domain that the project attempts to work with even before we begin to grapple with what successful Agile implementation looks like. In my late teens when I worked in manufacturing for a few years, I walked into the storeroom of a large manufacturing organization. Looking at their paint and copper part inventory - both items need tight inventory control – one for cost other for bulkiness and ability to be a bottleneck – I ask one of the most intelligently dumb question - aren’t you ISO compliant and following Kaizen; then why is this inventory so large and not procured based on JIT principle? As all good teaching moments are wont to be, this was embarrassing-

1. the inventory was actually JIT based, the 3.5 tonnes of paint was what they needed for only one month worth of paint jobs.
2. The cost of the paint was the least important factor in consideration - out goes ABC costing analysis that I learnt so well. What was important was that the paint job was among the last steps before delivery but an important factor towards brand recognition.
3. The paint supply company was notoriously well known for delayed shipments and transport was usually delayed as well due to poor road connectivity – I was aware of this.

Sighting success

Extrapolating the example to being able to identify individual project problem space requirements, the leader advocate for Agile should be able to define what success criteria look like in their individual projects given constraints and requirements. Here are a few broad examples that might help build better context:

• A web portal project has more lateral changes than deep business logic changes. The changes here can be smaller but the test suites need also be wider and encompass all scenarios (mostly scenarios are learnt over time – don't try to build an exhaustive list off the bat, as that does not work). If it is a mature monolith that needs to break into service layers as well as a separate UI layer (very common these days) work can still be broken beginning a clear separation of state-full v/s state-less behaviours. This lends itself to small, short iterations where progress can be visibly tracked and frequently monitored.

• An API service Dev portal will need request chaining tests along with proper contracts in place. Think about detailed documentation (e.g.: swagger for java), deep regression testing, backward compatibility (HAETOS – in case of REST JSON services) and automation frameworks that support API contract objects maintenance. These projects need good groundwork for better maintainability and a full regression suite to ensure congruence in behaviour over a timeline (as the consumer impact is hidden but critical for user relationship). This type of project too lends too small visible changes but needs large amounts of regression and performance testing. Deep changes can be made transparently. Test and deploy cycles (after initial framework setup) can be quick as can be development.

• A Machine Learning model deployment and change needs a larger development cycle and higher number of validation rounds with a limited set of tests that must be strictly adhered. Such development cycles are larger duration and the changes cannot be made in small increments. In fact, the improvements after successful deployments of new ML model would be drastic even for relatively smaller gains in accuracy. Another reason for a longer cycle is to ensure that a ML model does not over fit only a small section of training data over a set time window but can work generally over different data sets. Detailed documentation and a number of manual tests and observations may be initially required.

• A data procurement project can have defined set of standards for quality but can only be truly proven on data sets over a period of time. It is also important to note that data sets being large, processing cycles are longer duration. Test scenarios will constantly need tweaks until we get to a stable point, where most ‘transformation’ is accurate and expected ‘noise’ in data is covered by filter being placed. Regression test suite will consist of a continuously upgraded data set that is run through the data pipeline. Development cycles may be short, but deployment cycles will be over a longer period until behaviour is observed over time periods.

Buckling the Agile development process (the “shortcut” agile anti-pattern)

We spoke of an example that mentioned JIT and Kaizen in manufacturing. It is an important example that means to serve a dual function. First was to help define success in an Agile setup- the fact that it is contextual and heavily dependent upon the end goal. The other is to help the leader make a clear distinction on temporary process compromise v/s what would be an agile anti-pattern that forces compromise on quality and delivery, leading to untenable buggy products.

In context of CI/ CD and incremental metamorphosis over sprints - It builds the case for a need for the leader to have at least rudimentary grasp of manufacturing principles that inform Agile today – Kaizen and JIT. The fact that software is not a tangible asset is both a blessing and a curse. Since we are not building a physical asset, it is easily changeable, modifiable; just the same waste associated with software development is not tangible either – thus not easily quantified or even noticed. That the end result is a moving target with scope creep and scope changes also does not help matters. Every time we make changes to the software product that we do not use should be legitimately seen as waste.

With the view of software as an actual physical asset, the Agile advocate leader is better poised to understand the gravity of pinning down CI/CD (and resultant need for regression testing) as a need to ensure JIT in software development process. We only build what is exactly needed and only when needed, thus deliver it to production at the earliest. The regression serves as the benchmark to ensure that behaviourally our ‘new’ product version meets all current and new requirements in the digital dimension – like a physical product would in the real world.

The idea of Kaizen is also therefore also not a stretch, where incremental changes to the process to gain small but significant changes are advocate instead of one big bang (like in waterfall system where all requirements are expected to be met exactly). This is what Agile advocates when it mentions – responding to change over following a plan – small validated changes that improve the state of the product responsive to stakeholder demanded.

With these 2 similes in hindsight, we can now see examples of Agile anti-patterns in process with a newfound appreciation – maybe you know someone who matches these behaviours, and can tell exactly why that is detrimental:

• Does not understand why we need to ‘waste’ money on testing harnesses or tools? Don’t we have QA and UAT done for 3 whole hours by our PM, BA and the whole stakeholder teams?
• Does not understand why unit test cases are needed? The developer wrote the code, should he not test it before check-in? If they did, what will rerunning them get us? Why do we need unit tests written on old code?
• Can’t we just get some automated cloud-based tools to test the UI? Why do we need unit test cases here?
• We don't need to delay a release for setting up test suites in the CI/CD platform. You can do it when it is ready? - then it is the same guys who come around to ask why something failed in production and why testing did not capture it!!
• What is the business value of the testing automation engineer? They are not building any features. - getting money and time for building a test suite and integrating it has previously been a losing battle – less so now. They best option is to build the costs into each feature – essentially incrementally automate testing of any code area that we touch, and hope that we will have strong coverage over a period.

Lessons from the new normal of Continuous Deployment

Team built the test suite, which runs against the CI/CD platform and we deploy successfully. Next few iterations some test from the suite fail during deploy. Here is an interesting set of questions I received that you may too -

Why do you say that some tests were ‘fixed’? Did they not pass the last time? Did we release last time with broken testing?

What does the end to end test case help? If we have this covered, why ask business team to test? Why do you need sign-off from them?

A sign-off of course is always needed as a final set of sanity tests as well as confirmation of new features in place is essentials. Getting a sign-off closes the loop and also signal to the business that driving product behaviour is their calling- not that of development team.

What do you mean by test data has changed in the system? - this is hard to explain.

The best shot you have is to create benign test data and have that injected into each environment, so that your test cases do not rely on changed underlying transactional data. Remember, this is easier said than done. You may have to fight for the funding again.

How does fixing one test case break another? How did they both pass earlier?

Then one explains the changed business scenarios and how they impact existing behavioural assumptions of a test case. What usually breaks are edge cases and the best option is to build cases around artificial injected data for the test alone.

Why do we need a full-time automation testing team? The manual testing can be outsourced and is cheaper?

This truly makes one smile - not one to use clichés but nothing screams  ‘penny wise pound foolish’ better than this question.

We will follow up on this in future to discuss upon what could be a leader’s first step when starting the Agile journey. What size of work chunks make sense for the team based on specific scenarios. Of course, the attempt will be to pepper the content with examples that the reader can relate to in their teams. To that end- do send me your examples that can be included here. Every addition makes it a better read for all of us.