Graebel Agile SOA

Note: This is part of a new and continuing series which is collected under the Agile SOAcategory.

Here at Graebel, we believe that an Agile methodology provides great benefits to our development teams and our customers. Similarly, the decision to implement a service-oriented architecture has the potential to provide numerous benefits to the enterprise and our users. The question then becomes: Can an Agile methodology and SOA complement one another?   

In many ways there are strong synergies between the two disciplines as well as potential conflicts. First, we’ll examine areas where SOA and Agile are complimentary. Secondly, we’ll discuss specific areas where potential conflicts may arise.

SOA and Agile are complimentary

SOA is architecture. SOA stresses that businesses must be able to respond to the market and that by building services, we can get rid of the duplication and get closer to the elusive goal of reuse. By building services instead of applications, teams can leverage work by others from within the enterprise as well as externally.

Agile is a methodology. Agile stresses that everything changes and that software development teams must be able to recognize and respond to change frequently. By introducing both technical and non-technical practices, our development teams are able to help Graebel become an agile enterprise.

Architecture and methodology can be used together. They are complementary by nature. Moreover, SOA and Agile share the same broad goals. They both recognize that change is inevitable and that organizations need to effectively cope with that change. So we would expect that Agile is by default the methodology of choice when building SOAs and vice versa – right?

SOA and Agile may conflict

You would think that with shared goals the overlap of practices and architecture implied by these two techniques would be in agreement and not in conflict. At this point in time there is little in the SOA community that is agile and vice versa. Why is this?

One of the main reasons is that the SOA community approaches the agile methodology discussion with its own perception. Agile is historically grass-roots and small-project based, although throughout the past years the community has gained experience and learned to adapt the principles of the Agile Manifesto to large projects. SOA is a newer initiative and is top-down in nature, taking an incremental approach to the development of loosely-coupled services, processes and messaging components.

Two distinct areas where the SOA and Agile approaches are at odds:

• SOA encourages that architecture be upfront while Agile has a derogative term for this approach – coined BDUF (Big Design Up Front).
• SOA encourages teams to split along functional lines while Agile encourages cross-functional teams. 

Graebel’s Approach

To enable the best aspects of SOA and Agile, Graebel uses a leave-and-layer strategy in which the application platforms are incrementally transformed from point-to-point solutions to service-enabled interfaces. This transformation provides reliable access to the right data and processes at the right time, all while continuing to run the business in a transformational manner. This unique approach includes the best aspects of an Agile methodology to plan and manage the transformation and a Separation of Concerns process to build each layer of the SOA stack for each application in the enterprise. 

Graebel’s SOA Leave-and-Layer Methodology

This methodology, which we refer to as Incremental / Leave-and-Layer provides a wise alternative to the BDUF approach. Additionally, we are able to accommodate a cross-functional teaming strategy meaning that our SOA Program team is able to join existing application development teams for 2-week release cycles as each application is added to the integration architecture.

Graebel's SOA Program

The Third Day – Expanding Agile @ Graebel

Note: This is part of a new and continuing series which is collected under the Agile Journey category.

Prelude

In the last post The Second Day – Establishing Agile @ Graebel I described how we embarked upon a pilot project where we followed a scrum mindset. I also briefly discussed several aspects and discoveries that we experienced along the way and compared them to what we do today. The pilot project was an overall success and it laid the groundwork for an expansion of agile methods to the software development efforts under my direction at Graebel. This expansion is where we begin this post.

Introduction

Our pilot project was a perfect opportunity to try agile as it was green field development and thus wasn’t encumbered by legacy code, processes or architectures. We were able to dedicate developers who could stay focused and learn the scrum cadence and disciplines. We were also fortunate in that the business did not fully understand their own needs which enabled us to design, build & try all within a short iteration. Yet the pilot wasn’t representative of the other half of my department’s mission – which included support, maintenance, enhancements and brownfield development on custom developed applications as well as packaged applications.

Team Structure

One of the luxuries we had on the pilot was that we were able to dedicate team members to the project. However my team wasn’t large enough to have dedicated teams focused on each product or internal customer we supported. So instead we organized teams around a backlog that was a “shared” backlog – a combination of backlogs from several groups of products and customers. These teams handled support items, maintenance requests as well as enhancement development efforts.

In most situations this team structure worked fine. However one of the teams had a very demanding customer who began to dominate this “shared” backlog and its prioritization at the expense of the other products & customers served by that team. We tried numerous ways to control this situation but in the end this noisy customer got their way by going through various political channels. So our ultimate solution to this problem was to create a dedicated team of two developers who solely focus on this customer’s backlog.

If you are an IT department that is resource challenged my advice would be to try the “shared” backlog approach. However you should be brutally honest about the customers you serve. If most of the customers are well intentioned and can play nice with the other supported customers, then a shared structure should work fine. Yet if you have a customer like we did that is too demanding, then my advice would be to at least create a small, separate scrum team (even as small as two) to be dedicated to that customer. This solution follows the concept of the agile pattern of ‘sacrifice one’ but applies it more broadly.

All the experts say that the ideal scrum team size is 7 +/- 2 and that the team should be singularly focused. That works great when you have ample resources. We were resource challenged at the time and thus we had to be creative in finding a solution. We have been quite successful with teams as small as two so I encourage others to try the approach if the situation calls for it.

Note: Today our team is larger than when we began our agile journey. We no longer organize our teams in a ‘shared’ backlog structure. Instead we organize our teams by program.

Unanticipated Emergencies

In sprint planning, we would estimate the available hours across all of the team’s resources and then pull in backlog items until we reached capacity. As was always the case we would start the sprint and pretty much the next day an emergency or urgent request would present itself. We tried following the ‘sacrifice one’ pattern but many times that wasn’t sufficient. We entertained the thought of creating a dedicated support team but quickly dismissed the idea since we were already resource challenged. We even tried telling the customer that the sprint was “locked in” and the request would have to wait until the next sprint. (As you can imagine, this last approach didn’t go over too well and was quickly removed as an option).

Instead we built a buffer within our sprint plan by only planning at 80% capacity of the available hours in any given sprint. We reserved the balance to an “emergency fund” where urgent or emergency requests could proceed through a ‘quick checkout lane’. This approach worked and enabled the team to stay focused on the goal of the sprint while also providing flexibility and agility to the business.

Note: We still follow this approach today. However if we have a green field development effort, the team plans at 100% capacity until feedback begins to come in from UAT efforts.

Conclusion

We encountered many challenges as we worked to apply agile to traditional IT efforts and projects. We discussed two challenges today (team structure & unanticipated emergencies). Yet one of the more defining challenges was how to approach our customers & their needs from their perspective and not from a technical or development mindset. We will pick up this development vs. business process challenge in our next post.