The CGI story on agile scaling success on a large project

This intro is from the presenters:-

PAS was a joint venture development initiative by 4 major oil and gas
companies and CGI. Devon, Encana, Husky and Talisman joined with CGI to
develop a new Production and Revenue Application. Each company put 3 senior
business resources on the project. The development component of the PAS
initiative cost $35M over 5 years with up to 90 people on the team. This
presentation is on how we used Agile to achieve this mammoth undertaking.

The product is currently running in production at 5 major oil and gas
companies with great reviews. It is also in production for several mid and
smaller sized companies. CGI is currently marketing the product.

This was my first ever meet with the Calgary agile methods user group, CAMUG and it was a great experience. Off late, I have been looking for my graduate research topic on agile methods, especially around scaling agile beyond a single team and this was just a perfect session for me.

I noted down a few of their discussed topics that I would like to share with my readers:

1. The project continued in dev mode for 5 years, a $35M project.
   
2. It started with only 3 members and extended to a team of 90 following Scrum+XP.
   
3. The project had 5 sponsoring companies or clients.
4. They had 6 teams working in the same sprint cycles.
5. In each team, there were significant members from the business working full time with the team onsite and in the same open-space arrangement.
6. They had some 3000+ unit tests and also 300+ acceptance test.
7. They automated all repetitive tests.
8. The had external experts visiting them from time to time. Eric Evans once visited them and helped them getting better grasp on Domain Driven Design.
9. There had been a $800K and one month deviation from the projected cost and timeline.

They also discussed some issues related to working in a multi-team, multi-client project. I remember the following ones:-

1. They had to balance between features/bug fixes.
2. The deployment was taken care of by the respective sponsors, not the teams.
3. They had issues with sprint backlog management.

They highlighted some aspects with great emphasis:-

1. Open communication is the key to success.
2. Onsite and full time real user availability is very important for such a big project.
3. Automated tests, partial pair programming and continuous integration is a big plus.
4. Staying co-located was really helpful.

What about scaling beyond?

1. I asked them, if they felt it would still be a similar success if there had been 12 teams or even more with 200 people or so. They said, it would be challenging. But in response, they also said, following waterfall would make life much harder with such a big team, if at all it could be a success at the first place! Yes, I also believe this.
2. One of the audiences asked if they outsourced a part of the project and the response was 'No'. CGI has a big team in India with a very high CMM ranking, but the project manager and asst. project manager said, it would be much difficult for differing time zones. Also, it would be difficult to transfer the knowledge as well as replicate the value of onsite client presence. The development manager also added that, the cultural difference between an agile and CMM setup would also been an issue if they had to go for the off-shore team.

The session was much more eventful than what I could compile. So, if you are anywhere near Calgary and interested about meeting agile practitioners from the industry, lets meet at the next meet! 

 

 

Rambling stories from my days @ Code71 : Startup days

Sighs!

Back then, on Friday, May 26, 2006, I was a final year undergrad student at CSE, BUET. I was looking for a part-time software dev job opportunity and found a little one page advert for a student job at asha-technologies (lately renamed as Code71). Visited their website and thought it might be worth giving a try!

As usual, I was interviewed over phone and enjoyed a long 2 hour interview on premise. I found it really inspiring and gained more interest towards the job after this session. I started waiting for a response… but was pretty certain about a positive one!

Good things happen in quick successions in our lives. Agree?

I agree. Because I just started a soul-journey with my soul-mate, Shahana, on the March of 2006. Then got the job offer on the 26th of May! Couldn’t be happier. A job for me during that time was just beyond its compensation, it was an inspiration, opportunity to see the real world and being part of something bigger than myself.

Photo taken at Chhera dip (The Torn Island), St. Martins, Bangladesh. The last photograph taken with my first digital camera bought from my first month’s Salary!

I started my job on the 1st of June, 2006 at Asha-technologies. Asha-tech was still to find an office and Omar asked to meet him at his home office! That’s how it all started. We met a few times a week at Omar’s home until we moved to our office at the green building of green square at green road, Dhaka on the first of July.

Asha-tech signed up a client before it was even formed! So, once we moved into the office, we were building our first asha-tech product, an online loan financing gateway. We were enthusiastic about agile scrum/xp practices from the first day at office. The result was found in just less than 3 months. We rolled out our first release of the product. It was really a happy start. A fantastic start for a start-up. The client started generating revenue in less than 3 months of project inception!

The office setup was a small but adequate enough for our team. We installed long backup UPS (2 hours * 5 computers), A/C, IPS and dedicated internet connection with Wi-Fi. The work environment was full of fun. We were going for an outing to a team event every month and even more often going to delicious buffet places. We played table football and bowling… every time had a record breaking score and a new winner. We went swimming and believe me, Omar is as good a swimmer as he is a master in software technology. He would go to swimming with a flipper and flip like professional swimmers. He can even swim without using his hands/legs at all, lying side-on and in all such actions… I learned swimming keeping heads down and the easy way of breathing from him! Thanks!

Technologically we were maturing as well. Started using XPlanner for managing our project, attended daily standup meetings and started doing TDD. I won’t claim we got everything right at the first try… but, we were trying consistently, improving bit by bit… to this day.

For the most part, I was enjoying my job and became used to the pressure of a job + undergraduate studies. I said, good things happen in close succession. Another proof here! My first term final result with this job was a 4.0/4.0. I never scored 4.0 in a term before this nor had a 20+ hour/week job alongside my studies.

I strongly believe, my job taught me the attitude towards work, “plan, act and retrospect”. After over three years, I would suggest any new entrant in the software industry to start a career with passion. Its fun with passion. Its a win everyday with passion.

Lesson#2: Pace Your Sprint Rightly

In my previous post, I said about being incremental. Here come the next thing, being “ITERATIVE”. A prefer the term Sprint than Iteration.

So, once you decided to take small increments, make sure you reach the targets sprinting.

I suggest you prepare for sprinting with the right techniques and tools. A few recommendations-

1. Never miss a daily standup meeting. Spend only 2 minutes (or less) per person answering the three questions – what did you do yesterday, what’s on the plate for today and what is blocking the race?
2. Install visual clues for bringing the under the hood stuffs to daylight. Remember, being able to address the loopholes is the key. The solution usually follows automatically.

The key concept to internalize is, sprinting is a balance race. You need a good start and keep the momentum to reach the touch line on time. Its a small race and if you fall back, the touch line may seem too far to celebrate.

At Code71, ScrumPad, is helping us in sprinting. Our sprints are two week sprints in most projects. We found the team communication holds a key in meeting the deadline. Since, within a sprint, someone of the team may need an unplanned vacation or an issue may arise out of the blue, the team needs to step up and put extra efforts. Again, visual clues help the team in keeping everybody posted timely.

If a team finds it difficult to meet the deadline and find the sprint length to be too small, then what? Should they linger the sprint length? NO. The solution is to even shorten the length. To make sure, the team can plan for short spans with better accuracy. Lingering the sprint length addresses the wrong disease and hence may not solve the problem.

Lesson#1: Going Incremental is Natural!

As Scrum and Agile processes tell you, do it in small sprints, 2 weeks preferably, and go incremental. Take a small piece at a time and sprint.

I took this image from internet and credits to this site.

It’s easy to work out the Natural way than to defy it! If you are iterative and incremental, you are reaching towards the right hand side of the image! And to make the evolution faster, I suggest you try out Scrumming with ScrumPad. If you value “teaming” and “delivering” in sprints, ScrumPad will give you the oxygen to breathe while you sprint!

I suggest you follow a routine to iterate. At Code71, we follow the below-

Day 1: Plan for the next 9 days. Breakdown work items into smaller tasks, estimate at hours for each of the tasks and assign each task to an individual.

Day 8: Release in test server (shadow version of the real production environment) and collect customer feedback.

Day 9: Work on customer feedback and go live in production server. Always tag the version that is put in the production following the standard -

R<ProductReleaseVersion>.S<SprintNumber>.B<BuildNumber>.P<PatchNumber>

This gives us the rollback point when there is any issue in the latest production deploy. Of course, we make sure we do not need to touch on the tags too often!

How are you sprinting? Let me know if you have a suggestion.

Its easier to get lost! Would you?

I was talking to a friend of mine, who started a IT start up and working full time with a few others to foster the business. He was telling me-

“We have 13 projects running at this moment. We are working our heart out. But, we are not finding any time to do Test Driven Development, Continuous Integration and all the good things…”

Well, I understand how it feels! As the title says, “Its easier to get lost!”

 

Do you fancy getting stuck at this maze?

For startups, it is difficult to control the pulse. Because, you want to reach your billion dollar spot as early as possible, this week, or even earlier, today or even just in a few hours! And, as days go by, you start to feel the need for some process, to safeguard your efforts and quality, to hit the deadlines, to get in the rhythm of a sustainable pace.

But it may be really difficult to catch the ball at the first jump. We, at Code71, believe we need to be visibly competent and confident about the quality of our work. And worked hard to get the gears together in motion. Now it is cliques great. We have got the people motivated to follow the best practices, the tools to “do more with less” and the process to ensure a “sustainable pace”. I advise, you do at least the following to learn to escape the maze from tomorrow-

• DIE for planning. Plan for short spans. We plan for two week sprints.
• While planning, estimate honestly. Estimate all the hours necessary for following best practices. (producing automated test scripts may take 40% longer time, but bear with it)
• Plan on your inputs. You cannot push your inputs. So, you may need to squeeze the output to match against your capacity.
• DIE to meet the deadline according to the plan.

This is the mindset. You need to start believing to sustain. You need to plan honestly and never miss a deadline. After reaching the deadline, you retrospect. Find the loopholes and start filling those one by one. If you don’t see a loophole, find a ladder to reach higher. But do not place a ladder on a land mine!

Its not difficult if you have the preparation. To prepare, at a minimum do the following-

• Have a Continuous Integration (CI) server up and running 24x7 or DIE. (We use CruiseControl, its free and great)
• Make your CI server to send build, test and coverage report for you. Keep an eye on the coverage report and DIE to remain well above the the 80% line.
• Keep track of your activities for retrospect. We use ScrumPad. It helps you to iterate, collaborate and deliver.

The keyword is “sustainable pace”. I found Scrum to address this sustainable pace really smartly. If you haven’t tried yet and looking for the loopholes to fill or the ladder to climb up, I suggest you learn about agile and Scrum, find an expert and give it a try. It’s worth than you might have expected.

If you know a better way to keep delivering better, I will appreciate for sharing your idea to me.

Pair Programming: How am I feeling?

For an explanation of Pair Programming, please visit this wiki entry.

Good

I am pair programming on the ScrumPad project now and the feeling in general is good. I have this perception is the work quality is better than doing alone. Also, the amount of time wasted to find small issues are now much reduced. Overall, its good.

Productivity Increased

I definitely found the productivity boosted by a factor of 1.5 at least. May be this is specific to my case, as I am working on a mid scale refactoring task. We worked on two user stories so far and completed the stories in 1.5 times faster compared to the individual estimates. We are doing estimation on a regular basis and the variance between estimation and actual hours are less than 5% as we see in the last 17 sprints in this project.

Disclaimer

I am yet new to this practice and not in a position to draw a conclusion yet. However, I believe our open space set-up and strong team culture helped us a lot to start this pair programming.

Are you pairing? Please, share your feelings and suggestions.

What does a Scrum Master really do?

Hmm! I am now a CSM (Certified Scrum Master) after two days of training with CST Pette Deemer.

You want to congratulate me! Lets congratulate me first-
You: Congratulations! It's really a great news! How does it feel to be a CSM?
Me: Greaaaaaaaat! You know, its so cool that my profile gets a CSM on it!
You: Oh Sohan! I am so sure that you will contribute more at the company?
Me: Yeah, I surely will. I just need a product owner and a team with me to contribute!

(Now that I am so good with the certificate let me be your teacher on 'Scrum Master'ing)

Well, I have been following SCRUM for almost two years in my team and we had a lot of questions in our minds regarding the process. Some of the questions are answered and some are not. But to be very positive, the outcome of the training is good. It clears the philosophy behind SCRUM and also creates a clear vision on why one should follow SCRUM. I will be going on a question answer mode that closely resembles my learning at the training.

Q: Is scrum master a part of the daily scrum meeting?
A: Yes, he is involved, but stays outside the team's SCRUM cycle and takes note whenever someone points out to anything that is a hurdle before the team in meeting sprint commitment.

Q: So, is Scrum Master is a lead role?
A: No. Its rather a helper role to the team.

Q: Is it possible that Scrum Master is not co-located with the team?
A: Generally its bad. It seems unlikely that she can remove blocks from heaven (the distant is always a heaven!).

Q: What are the core responsibilities of the Scrum Master?
A:
Help team to follow SCRUM.
Do anything to remove blocks reported by the team.
Do nothing to create any block for the team.
Update the Burn Down charts, the 'Not Started, Started, Completed' board and other progress indicator 'things'.
Facilitate the team in meetings - avoid unnecessary time loss meetings, manage meeting place and things like laptop, projector, mic and sound systems... all that it takes for the team to have a potentially meaningful and quick meeting.

Q: Is Scrum Master part of the team?
A: Well, it depends. If the team is small and doesn't need/cannot afford a full time scrum master, someone from the team may take on this responsibility. But, a dedicated one is most likely to be effective.

Q: What should be the bullet points of the Scrum Master's characteristics?
A: Helping, self-motivated and feels comfortable to help people on any sort of problem.

Q: Enough is Enough. What else do you know about Scrum Master?
A: Hmm... Scrum Master is not executed if the team does not meet their commitment (unless its because of the fact that the Scrum Master wasn't able to remove blocks). Scrum Master always guards the team from product owner's pressure and change requests in the middle of sprints.

I could have a longer list of Q & A that I have actually learned at the training. But I am not going to do that partly because, I have only a few readers (namely I, me, myself and Sohan) in my blog and partly because I do not see comments on my posts too often.

So, if you have read this and really think you want to know more, energize me with your comments and questions :-). I will be really glad to share my learnings with you.

Prerequisites for IID

Up to now, I have figured out that teams trying to behave agile sometimes fall into the cave of pretending to be agile (e.g. stand up meetings and less than just enough planning and short iterations so on!). Often, when this pretending takes places for days, it creates a illusion on the minds of the people that they might be doing the "right thing". And all on a sudden, someone discovers things are breaking and there is no way out of this disaster!

So, I suggest for teams who are trying to get agile to meet the following prerequisites first-

1. Team members must have a certain level of all-round skills.

So, when we do not preserve seats for requirements gathering, planning and designing and still want a good architected software, we need to make sure we are not asking beyond the capacity of one team member. From my experiences I have found that, even a very good communicating team with enough of discussions and reviews cannot control the quality and integrity of the product when some of the fellows don't meet the all-round skill requirements.

So, an agile team is expected to meet all-round individual skills like
a. Database design concept.
b. Object Oriented design concept (must know about design patterns (e.g. from GoF) and design principles)
c. Standard Coding expertise.
d. Testing sense and writing effective test codes.
e. Understanding the business domain clearly and concisely.
f. Quickly respond to change requests and define effective way to meet system owner's demands.

2. An "agile" skilled team must practice the following

a. Effective Design and design reviews.
b. Good coding at the very first time (if possible).
c. Must write test/guard codes for automated testing.
d. Continuous and periodically re-factorization of the code base and architecture.
d. Some sort of mentoring to make sure things are in rhythm.

3. An "agile" skilled team must use appropriate tools for the "best practices"

a. Use smart collaboration tool to make sure all the stakeholders are on the same ground.
b. Use effective test suits.
c. Version management system.
d. Automated build system and continuous integrations.
e. Project management and time tracking software.

I will try to get more subjective on each of the items mentioned above in my upcoming posts.

Two Aspects of Agile Software Development Process

I have been in practice with the famous agile software development process called "Scrum" for more than a year. I have had the opportunity to see a SCRUM team developing from scratch in a completely start-up environment.

I will point to the two aspects as,
1. Management aspects.
2. Technical aspects.

1. Management aspects
I encapsulate the things like working on a story by story basis, standing up for daily status update meeting, producing deliveries in short iterations and without much of prior planning into this group. I believe this aspect heavily depends on the way the second aspect is handled.

2. Technical aspects
Technical aspects encapsulate the practice of Test Driven Development, continuous and periodic Refactoring and other hard core technical things like this. Despite the truth that, this aspect is not the much anticipated by the client, being "Agile" there is no alternative to achieve what is anticipated, "Business value", without this.

Developing Software falls into the category of craftsmanship. So, its hard to define the line and expect the players to stop on a referee's whistle. With agility, developers are free to add their creative inputs to the software design and code base. So, without attention to the technical aspects may lead to a process of "Iterative Incrementation Destruction" instead of "Iterative Incremental Develeopment".

In subsequent posts I will try to come up with actual examples in favor of first making the "techi" things right.

Visit www.scrumalliance.org for Scrum related knowledge.