The Fourth Day – Establishing the BA Role

Note: This is part of a new and continuing series which is collected under the Agile Journey category.

Prelude
In the last post The Third Day – Expanding Agile I described how we encountered many challenges as we worked to apply agile to traditional IT efforts and projects. We discussed two challenges in particular (team structure & unanticipated emergencies). We pick up in this post our challenges in approaching our customers & their needs from their perspective and not from a technical or development mindset.

Establishing the BA Role
Initially my team was solely focused on coding software. I had a team of .NET developers who supported, enhanced and built business applications for our various customers. We did not have QA resources nor did we have business analysts. When customers had new requests, they would come directly to me. I would serve as the liaison between the business and developers facilitating requirements gathering and ensuring what was ultimately delivered met the business’s needs. Yet serving in this role became unfeasible as my team’s responsibilities and applications under management expanded. There were just too many moving parts for one person to coordinate. As a result, I established the business analyst (BA) role within the IT process.

Establishing the BA role was tough as many business users didn’t understand what a business analyst was and thus they wondered why we needed the role. Instead they argued that we should be hiring developers as it was more code written that was needed. Another challenge we faced was that the business viewed the BA as a project manager. Thus many of the initiatives we were working on became an ‘IT’ project and not owned by the business. We were unsuccessful, for a time, in changing this mindset and as a result, our BAs were not able to focus their attention on legitimate BA activities (enterprise analysis, requirements planning, requirements gathering & etc.)

Recently, with the help of the business, we have been able to evolve the BA role. The BAs now fulfill the role of product owner on the scrum team. The BAs are now officially titled as either application analysts or product managers (depending upon focus). We have gotten away from the term ‘Business Analyst’ as our initial experiences necessitated a change (superficial or not).

We are now focused on further refinement of the ‘BA’ role within the development team. One vital step in this effort was to acquire a highly talented and experienced product manager. We got fortunate at the end of 2009 and were able to acquire this talent and now this individual is leading the charge in transitioning the BAs into the product owner role on the scrum teams.

In addition to hiring an experienced leader, we have also developed a BA competency model that outlines a skills inventory for the role as well as proficiency definitions. This competency model enables us to establish targets, assess proficiency, measure and prioritize gaps, leverage team strengths, create individual development plans, and help standardize tools and processes.

Conclusion
Establishing the BA role has definitely been a challenge. Many organizations have this role defined and integrated within the business (via a PMO) as well as within IT. We did not. Thus we created the role within the IT development group out of sheer desperation to better address the demands our customers were placing upon us. We have a long way to go to truly execute the role appropriately and as outlined in our competency model. However as we learned with Agile, change isn’t a short process and must be looked at in terms of months and years.

Graebel Agile SOA

Note: This is part of a new and continuing series which is collected under the Agile SOAcategory.

Here at Graebel, we believe that an Agile methodology provides great benefits to our development teams and our customers. Similarly, the decision to implement a service-oriented architecture has the potential to provide numerous benefits to the enterprise and our users. The question then becomes: Can an Agile methodology and SOA complement one another?   

In many ways there are strong synergies between the two disciplines as well as potential conflicts. First, we’ll examine areas where SOA and Agile are complimentary. Secondly, we’ll discuss specific areas where potential conflicts may arise.

SOA and Agile are complimentary

SOA is architecture. SOA stresses that businesses must be able to respond to the market and that by building services, we can get rid of the duplication and get closer to the elusive goal of reuse. By building services instead of applications, teams can leverage work by others from within the enterprise as well as externally.

Agile is a methodology. Agile stresses that everything changes and that software development teams must be able to recognize and respond to change frequently. By introducing both technical and non-technical practices, our development teams are able to help Graebel become an agile enterprise.

Architecture and methodology can be used together. They are complementary by nature. Moreover, SOA and Agile share the same broad goals. They both recognize that change is inevitable and that organizations need to effectively cope with that change. So we would expect that Agile is by default the methodology of choice when building SOAs and vice versa – right?

SOA and Agile may conflict

You would think that with shared goals the overlap of practices and architecture implied by these two techniques would be in agreement and not in conflict. At this point in time there is little in the SOA community that is agile and vice versa. Why is this?

One of the main reasons is that the SOA community approaches the agile methodology discussion with its own perception. Agile is historically grass-roots and small-project based, although throughout the past years the community has gained experience and learned to adapt the principles of the Agile Manifesto to large projects. SOA is a newer initiative and is top-down in nature, taking an incremental approach to the development of loosely-coupled services, processes and messaging components.

Two distinct areas where the SOA and Agile approaches are at odds:

• SOA encourages that architecture be upfront while Agile has a derogative term for this approach – coined BDUF (Big Design Up Front).
• SOA encourages teams to split along functional lines while Agile encourages cross-functional teams. 

Graebel’s Approach

To enable the best aspects of SOA and Agile, Graebel uses a leave-and-layer strategy in which the application platforms are incrementally transformed from point-to-point solutions to service-enabled interfaces. This transformation provides reliable access to the right data and processes at the right time, all while continuing to run the business in a transformational manner. This unique approach includes the best aspects of an Agile methodology to plan and manage the transformation and a Separation of Concerns process to build each layer of the SOA stack for each application in the enterprise. 

Graebel’s SOA Leave-and-Layer Methodology

This methodology, which we refer to as Incremental / Leave-and-Layer provides a wise alternative to the BDUF approach. Additionally, we are able to accommodate a cross-functional teaming strategy meaning that our SOA Program team is able to join existing application development teams for 2-week release cycles as each application is added to the integration architecture.

Graebel's SOA Program

The Third Day – Expanding Agile @ Graebel

Note: This is part of a new and continuing series which is collected under the Agile Journey category.

Prelude

In the last post The Second Day – Establishing Agile @ Graebel I described how we embarked upon a pilot project where we followed a scrum mindset. I also briefly discussed several aspects and discoveries that we experienced along the way and compared them to what we do today. The pilot project was an overall success and it laid the groundwork for an expansion of agile methods to the software development efforts under my direction at Graebel. This expansion is where we begin this post.

Introduction

Our pilot project was a perfect opportunity to try agile as it was green field development and thus wasn’t encumbered by legacy code, processes or architectures. We were able to dedicate developers who could stay focused and learn the scrum cadence and disciplines. We were also fortunate in that the business did not fully understand their own needs which enabled us to design, build & try all within a short iteration. Yet the pilot wasn’t representative of the other half of my department’s mission – which included support, maintenance, enhancements and brownfield development on custom developed applications as well as packaged applications.

Team Structure

One of the luxuries we had on the pilot was that we were able to dedicate team members to the project. However my team wasn’t large enough to have dedicated teams focused on each product or internal customer we supported. So instead we organized teams around a backlog that was a “shared” backlog – a combination of backlogs from several groups of products and customers. These teams handled support items, maintenance requests as well as enhancement development efforts.

In most situations this team structure worked fine. However one of the teams had a very demanding customer who began to dominate this “shared” backlog and its prioritization at the expense of the other products & customers served by that team. We tried numerous ways to control this situation but in the end this noisy customer got their way by going through various political channels. So our ultimate solution to this problem was to create a dedicated team of two developers who solely focus on this customer’s backlog.

If you are an IT department that is resource challenged my advice would be to try the “shared” backlog approach. However you should be brutally honest about the customers you serve. If most of the customers are well intentioned and can play nice with the other supported customers, then a shared structure should work fine. Yet if you have a customer like we did that is too demanding, then my advice would be to at least create a small, separate scrum team (even as small as two) to be dedicated to that customer. This solution follows the concept of the agile pattern of ‘sacrifice one’ but applies it more broadly.

All the experts say that the ideal scrum team size is 7 +/- 2 and that the team should be singularly focused. That works great when you have ample resources. We were resource challenged at the time and thus we had to be creative in finding a solution. We have been quite successful with teams as small as two so I encourage others to try the approach if the situation calls for it.

Note: Today our team is larger than when we began our agile journey. We no longer organize our teams in a ‘shared’ backlog structure. Instead we organize our teams by program.

Unanticipated Emergencies

In sprint planning, we would estimate the available hours across all of the team’s resources and then pull in backlog items until we reached capacity. As was always the case we would start the sprint and pretty much the next day an emergency or urgent request would present itself. We tried following the ‘sacrifice one’ pattern but many times that wasn’t sufficient. We entertained the thought of creating a dedicated support team but quickly dismissed the idea since we were already resource challenged. We even tried telling the customer that the sprint was “locked in” and the request would have to wait until the next sprint. (As you can imagine, this last approach didn’t go over too well and was quickly removed as an option).

Instead we built a buffer within our sprint plan by only planning at 80% capacity of the available hours in any given sprint. We reserved the balance to an “emergency fund” where urgent or emergency requests could proceed through a ‘quick checkout lane’. This approach worked and enabled the team to stay focused on the goal of the sprint while also providing flexibility and agility to the business.

Note: We still follow this approach today. However if we have a green field development effort, the team plans at 100% capacity until feedback begins to come in from UAT efforts.

Conclusion

We encountered many challenges as we worked to apply agile to traditional IT efforts and projects. We discussed two challenges today (team structure & unanticipated emergencies). Yet one of the more defining challenges was how to approach our customers & their needs from their perspective and not from a technical or development mindset. We will pick up this development vs. business process challenge in our next post.

My first taste of agile.

This is a first in a series of posts about the processes, disciplines, tools and vision of Graebel’s agile software development methodology. 

Lets start this off with some info about me and my background.  My name is Nate Kresse I started my career in the software industry about ten years ago developing web applications using cold fusion.  Over the years I gravitated towards the .Net world eventually building web applications with ASP.NET to now running the software engineering at Graebel.  I have degree in Computer Science and really enjoy everything about software: from design to software methodologies (such as agile.) 

Sprint Backlog Item Sticky Note

When I started as a developer at Graebel agile was completely unfamiliar to me.  I was quickly immersed into a world of custom sticky notes (see image to the right) posted all over the work area.  Right off the bat I learned about scrum teams and how self managing teams were the wave of the future.  Some of things I noted was that Graebel really was adhering to the Agile Manifesto and adopting people and processes over tools and documentation.  By using paper everything was visible and and right in front of you but getting totals and tracking were a little bit of a pain. 

Early on the developers where controlling everything.  Requirements, development, testing and releases.  Developers also conducted all release and sprint planning.  The major problems I saw were all the hats a team member had to wear.  A single developer was doing 4 context switches through every back log item.  We were also a small team of about 7 developers taking on the entire backlog of the enterprise.  That means in one sprint we could be switching between 4-5 applications.  As a result, throughput was crawling, development would develop right up to deadlines with a version of the release that was under scope compared to the original vision.  Since deadlines were tight all our test driven disciplines were thrown out the window supposedly to get the release done faster (so we thought at the time.) 

We had intial success with developer run scrum teams because we were delivering software quickly compared to previous efforts.  Also the types of applications we were building were pushing the businesses ability to integrate with outside customers and vendors as well as enhance the business from an operational standpoint.  The basic functionallity was there but we lacked in areas such as usability, maintainability and overall scalability.  Our execution needed work, our disciplines needed work and our responsibilities needed to be revamped and refocused.

In my next post I will highlight the next step in Graebel’s agile evolution to combine a new business analyst team as well the development team into one single process.

The second day… Establishing Agile @ Graebel

Note: This is part of a new and continuing series which is collected under the Agile Journey category.

Prelude

In the last post Let there be light! I described how a past project failure and some of its circumstances served as fertile ground upon which the seeds of agile could be sown and nurtured. I examined how the 4 core elements of the Agile Manifesto were relevant to the circumstances experienced on that project. Yet it wasn’t until I joined Graebel in 2005 as the Director of Web Technologies that I was in a position to apply agile software development methodologies within a development context.

Note: We partnered with Pete Behrens of Trail Ridge Consulting to mentor us through our agile adoption and implementation. His expertise and efforts were invaluable throughout the fits & starts we experienced during our pilot project.

Introduction

Throughout the interview process at Graebel I positioned my engineering mindset with agile development methodologies rather than the traditional waterfall mindset. Specifically I was a fan of scrum as I believed the practice was best suited to be applied to the full workload of a corporate enterprise IT department. My long-term plan was to take the scrum approach, apply it not only to new software development but also to support, maintenance and application enhancement development activities. Yet before I could achieve that mission I needed a pilot project where scrum could be applied and to serve as the catalyst for change within my newly acquired development team. We embarked upon that pilot project at the end of 2005 and what follows is a quick description of several aspects of our initial scrum implementation.

Team space

We took down cube walls within a section of cubes to create a six person bullpen. We relocated most of the members of the team into this bullpen to better facilitate face-to-face communication and collaboration.

Note: We were unable to move everyone into this space as we had significant resistance to moving to such an open environment from several “legacy” members of the team. However today we restructure our floor plan at least quarterly according to projects and their teams.

Sprint Planning

Initially we rolled with 30 day sprints. However progress was elusive for the team and planning was extremely difficult because our backlog was very shallow. We adjusted to one week sprints and as momentum built we expanded to two week sprints.

Note: Today we still plan two week sprints. We found that one week sprints were too chaotic and sprints longer than two weeks didn’t facilitate enough agility for the business.

Stand-ups

I have struggled with the need for stand-ups from the beginning. I remember meetings where we would go around the horn asking people “What did you do? What do you plan to do? Do you have any roadblocks?” People would just look to the task board and regurgitate their tasks without adding any valuable context. After they stopped talking, we still had no clue as to their progress towards the sprint goals. The stand-up was not meant to be a status report but that is essentially what it became.

Note: Even today we struggle with stand-ups and are working to transform them to contain focused communication that enables the evaluation of progress towards the sprint’s goal. My thought is that if I need a status, I can just run a sprint report and spare everyone the meeting. Yet the meeting is a key collaboration point and discipline so instead of getting rid of it we just need to transform it.

Story Boards and Task boards

We wrote our user stories out initially on index cards and then moved to yellow stickies. These stories were either taped or placed on a whiteboard that had four sections: To Do, In Development, Testing and Done.

Task cards were the result of the user story being broken down into logical development tasks. We initially used index cards, progressed to yellow stickies and then proceeded to have our own custom stickies printed. These task cards were placed on a whiteboard with five sections: To Do, In Progress, Awaiting Verification, Verified and Complete. These cards and their progression across the whiteboards were instrumental in keeping the sprint and its progress in front of the team at all times. This same effect could not be provided via an electronic tool.

Initial Task Board Concept

Note: We eventually got rid of the manual cards in favor of leveraging online tools to track their state, progress and particulars across our process. Yet today we are in the process of bringing back a manual Kanban system to run in parallel with our online tools. Relying completely on our online tool resulted in a significant loss of visibility and awareness across the team, within management and with the customer.

Product Owner

Initially I served as the product owner within the team. I was the one who developed the technology vision and strategy for our pilot project and thus was best positioned to serve in the role. Plus at the time there was not a business analyst on the team. If there had been a business analyst, they would have been the most logical choice to serve in this role.

Note: Today we have a team of business analysts that we actually call product managers. These individuals are collocated with the team and own the release, product backlog and product roadmap.

Looking Ahead

This posting is the starting point for a new and continuing series which will be collected under the Development Process category. Nate Kresse, Graebel’s Manager of Software Engineering, will lead this series and examine the processes, disciplines, tools and vision for Graebel Agile.

Let there be light!

Back in the mid ‘90’s I had the opportunity to establish and lead a small software development company named Strategic Innovations. Initially Strategic Innovations focused on application development via Visual Basic. But as the ‘90s progressed we naturally expanded our expertise to include Internet-based application development via Java and ASP technologies. In 2000 I sold Strategic Innovations to a Midwestern company, Meritage Technologies, who was rolling up small Microsoft-focused software consulting companies in key markets in hopes of eventually taking the company public.

One of the first projects we landed after being absorbed into Meritage Technologies was a job to develop a Java application to enable Fortune 500 companies to manage their global Six Sigma efforts. This application was truly innovative in concept but the project was a nightmare and by most measures a failure. A solution was ultimately delivered but it was over budget, under scope and late.

When I eventually had the luxury of assessing the project I identified several key circumstances that when taken in isolation did not cause the project’s failure but when combined together they created a ‘perfect storm’. Some of those circumstances were:

• The application was 100% conceptual and there was not a point of reference to draw upon within the client’s business or even the general marketplace.
• A monumental effort was undertaken by the business analyst at the beginning of the project to elicit and define the envisioned application’s full set of functional and technical requirements.
• The requirements were well documented and placed immediately under change control.
• Development was “outsourced” to a sister office in another state at the request of corporate management.
• The business analyst was the sole interface and intermediary between the client and the remote development team.
• A significant amount of requirements churn eventually stymied and all but paralyzed the development team and its progress.
• Once the requirements began to churn the project became a bureaucratic mess due to a rigid change request process and an overarching focus on the project plan.

It was not until a few years after this project that I was able to start to correlate these circumstances together and begin to comprehend their association and collective contribution to the project’s failure. It was also at this time that agile methods started to gain traction in the development community as an emerging development mindset and alternative to the typical waterfall approach. I can still remember reading the Agile Manifesto and having all kinds of bells and whistles going off in my head as I reflected upon the principles behind the manifesto. It was an epiphany and a key watershed moment in my professional career.

Working software over comprehensive documentation

Since the project was truly a ‘greenfield’ development effort the expectation that the client definitively define the application’s requirements at the outset was completely and utterly unrealistic. It isn’t that the client didn’t have a clue about their business but rather they had no point of reference to draw insight or knowledge as to exactly what needs the application was to fulfill. Thus the requirements that were initially defined and well documented essentially became worthless once development got underway. The only thing that would have enabled the client to more fully understand their needs and the application’s requirements would have been working software that they could have touched, felt and played with.

Customer collaboration over contract negotiation

The primary reason why such a huge effort was undertaken to capture all of the application’s requirements upfront was because both parties “needed” a defined scope to build a contractual agreement around. Once the contract was in place a rigid change management process was enacted to ensure the contract’s integrity as well as to facilitate mutually agreed upon scope changes to that contract. Yet as pointed out previously the client had no frame of reference to draw upon when defining the application’s requirements and thus the contract was, for the most part, based upon an ill-defined scope and a premature set of requirements.

The result was that collaboration with the customer was essentially squelched. Why this happened was because as the application became more tangible in the client’s eyes, the more clarity that was realized about the application’s true requirements. Yet that clarity necessitated endless change orders as well as addendum to the contract. Eventually whatever collaboration and trust that had existed disintegrated into an ongoing negotiation and renegotiation exercise.

Individuals and interactions over processes and tools

Outsourcing development to a sister office was a corporate mandate in an effort to help offset the economic downturn experienced by this sister office as a result of 9/11. This decision decimated any semblance of a team as well as effectively prevented meaningful collaboration and cooperation between the client and development team. This separation eventually contributed to mistrust between the business analyst and development team as delivery dates slipped and confusion surrounding requirements prevailed. Combine these circumstances with the previously discussed challenges and it can be seen that close, daily cooperation between the business and technology team is essential.

Responding to change over following a plan

Given the nature of this project as described above it can be plainly seen that change was inevitable on this project. But in every facet of the project life cycle, change was considered the antithesis to the project’s success. It was to be eliminated and avoided at all costs. The reality is that change should have been fully embraced and enabled throughout the life cycle and within the disciplines of the project. Yet both the client and development team were incented both contractually and monetarily to keep to the plan regardless of whether responding to change would have helped ensure the success of the project.

Conclusion

It was this project experience and resulting soul searching that prepared me to fully embrace agile when I took the reins of software engineering at Graebel back in 2005. Yet over the last few years we have not only embraced agile for our new development efforts but have expanded our agile mindset and disciplines to encompass all of our software engineering activities from maintenance, support, minor enhancements, major enhancements, architecture, infrastructure and large capital projects. We have also incorporated business analysis and product management disciplines with the goal of quickly expanding into program and portfolio management disciplines.

Future posts will bring insight into our current processes and disciplines with a look to the future and how we plan to continually improve, reduce waste and journey further into what we call enterprise agile.

In the beginning…

The Enterprise Agile Blog’s mission is to establish a new conversation within the agile community focused on agile within the enterprise context and specifically within the corporate IT setting. Most agile enthusiasts approach the agile methodology from an ISV or new software development perspective. This isn’t a bad thing but it certainly leaves a large void in the dialog for those people, like myself, who are attempting to establish agile within a corporate IT setting where new software development isn’t the only show in town.

So where do I see this going? Quite honestly this blog’s purpose is a bit selfish. First I am seeking a dialog where I can tap into the experiences of other agile enthusiasts so that I and my team can learn and continue to apply new disciplines, principles and processes. Second this blog will be an avenue for the team I lead to share their experiences and expertise with the agile community. Lastly I hope to provide insight (the good, the bad & the ugly) into my organization’s agile journey and our struggles and successes to implement agile within a corporate enterprise IT setting.

Now a bit of background… My name is Alan Ruth and I am the Sr. Director of Enterprise Applications at Graebel – a global relocation company headquartered in Denver, CO . I joined Graebel almost 4 years ago and have since had the unique fortune to lead a great team of software engineers and business analysts. We began our agile journey in late 2005 and have not looked back since. Yes we have had plenty of fits and frustrations over the years but we are definitely a much more nimble, effective and strategically aligned technology department due to “living agile” than we ever were before.

We did not tackle this endeavor on our own but instead engaged Pete Behrens of Trail Ridge Consulting to act as a mentor and guide. This partnership with Pete has been invaluable and I would highly recommend such an arrangement to anyone. Another key component that enabled our implementation and the success we have realized so far to date is the support and leeway provided by several key executives within Graebel. One in particular, Graebel’s CIO and my boss – Don Tarkenton, was and continues to be instrumental. From day one Don fully supported my agile vision and gave me the freedom, flexibility and encouragement to institutionalize the mindset and its “new” disciplines. He didn’t always see the “light” but he remained a steady champion of the vision and never wavered. One other vital component to our implementation that I believe must be mentioned is my team of engineers, analysts and their leaders. Without their willingness to change, take my direction and endure my constant desire for improvement we would have never gotten far.

Graebel follows a Scrum methodology with some nuances (like 2 week sprints instead of 30 days). We are primarily a .NET development shop but we also maintain/enhance custom Oracle Forms applications. We leverage Microsoft’s Team Foundation Server for our source control, builds and agile process management. The team consists of about 20 software engineers and 5 business analysts (which we call product managers). The team is geographically dispersed between Denver, CO and Wausau, WI.  

With all that behind us we can now effectively embark on our journey. Subsequent postings will focus on how we have evolved to fit agile into our corporate environment as well as share our successes, disciplines and vision for agile at Graebel.