Weblog for the AMIS Technology corner
Project Management
Oracle Team Productivity Center
Jul 12th
‘Oracle Team Productivity Center (TPC) is an Application Lifecycle Management (ALM) tool that enables software development teams to collaborate and work productively together when developing applications using JDeveloper.’ (OTN TPC page)
TPC provides unified access to different ALM repositories from within JDeveloper and it allows to define relations between the so-called work-items in these (separate) repositories. It consists of a central repository, a JDeveloper extension and a set of connectors to other systems. Currently, connectors to Jira, Bugzilla, Rally Software and Microsoft Project Server – task management are available. This means that we can, for example, access and work on our Jira issues from within JDeveloper. In addition it provides a task service and a Google Talk client for JDeveloper.
TPC is installed on an application server, e.g. Tomcat or Weblogic and requires a database to store its data. It also requires the connectors to be installed, although they are not really used to connect to the other systems. The connection to the repositories is made from JDeveloper, so you need the connectors and TPC extension.
The TPC architecture:
Compliments; Instant productivity improvement for software teams, with a small effort….
Apr 25th
Hello, you project manager/team leader. I expected this title to grasp your attention. Would you like to know how to improve the performance of your team members? This can be done without massive statistics or an expensive performance improvement program. This magic pill is called positive feedback. Just give your team members the credits for their work and compliment them for their achievements.
As a project manager we are aimed on the end result. In our day to day job we are focused on the things that are not yet done and the things that could go wrong. This focus on future result and possible impediments make us forget the past achievements of our team.
Automatic testing Oracle Service Bus using Hudson, maven and SoapUI
Feb 23rd
A lot of current projects are implementing some sort of service based architecture. Testing in this architecture becomes more complex. When implementing an OSB project with Scrum you test-automation is imperative. Scrum will require more frequent testing of your system. This is only feasible (in time and money) when you automate as much as possible.
Using soapUI you are able to create visually SOAP tests on your OSB implementation and running them against the defined infrastructure (develop, test, acceptance). SoapUI enables with easy tools to implements verification and validation of the responses of your OSB implementation. When running the test you are also able to set limits in SLA response times on all the calls. This way you are able to monitor depreciation of performance in older parts of your OSB implementation when adding new services.
You can record and edit your SOAP test easy with the soapUI interface and edit it later. When you maven-enable your project it is quite easy running your tests when you implement the “maven-soapui-plugin” (see my other posting http://technology.amis.nl/blog/3061/automated-soap-testing-with-maven). In the meantime version 3.0 of this plugin is released.
When implementing this with Hudson you do not have to convert the results.xml into a Surefire report. Hudson will manage this for you. Hudson will also enable you with an historical overview of all your test results.
Agile software development, the principles. Principle 11: The best architectures, requirements, and designs emerge from self-organizing teams.
Dec 14th
This is the eleventh of 12 posts about the principles of agile software development. Purpose is to go back to the start of the agile manifesto (http://agilemanifesto.org/principles.html) and discuss the implementation of the 12 principles in real life software engineering. Goals of agility are to go deliver software of higher quality, faster, with a higher acceptance to end-users and able to follow the changing business requirements to strive for competitive advantage.
The question is: is this going to work in practice or is this only based on a nice marketing and sales story.
Principle 11: The best architectures, requirements, and designs emerge from self-organizing teams.
For a long time the engineering expertise (and also software engineering) was based upon the condition that you worked with specialists. These specialists emerged from the principle of division of labour and made it possible for these specialists to focus their attention on their specialism and create the best solution within their field of expertise. The Interaction designer designed a user interface, the architect created a global systems model, developers created code and infrastructure specialists created the necessary environment to run the application on.
Everyone was specialized and delivered the best solution within their capabilities. However when all these components where put together, noting worked. It is an illusion that specialists can design and foresee everything beforehand.
Within agile projects the solution is to use a self organizing team to perform these tasks. This team may consist of specialist, but this is not a requirement. The requirement of this team is that they work together and self-organize all aspects of the systems to be delivered. This team is permitted to make errors and invent their solutions, provided that they deliver and evaluate frequently (retrospective) and learn from their successes (and errors).
Agile software development, the principles. Principle 10: Simplicity–the art of maximizing the amount of work not done–is essential
Aug 3rd
This is the tenth of 12 posts about the principles of agile software development. Purpose is to go back to the start of the agile manifesto (http://agilemanifesto.org/principles.html) and discuss the implementation of the 12 principles in real life software engineering. Goals of agility are to go deliver software of higher quality, faster, with a higher acceptance to end-users and able to follow the changing business requirements to strive for competitive advantage.
The question is: is this going to work in practice or is this only based on a nice marketing and sales story.
Principle 10: Simplicity–the art of maximizing the amount of work not done–is essential.
The KISS rule (Keep it stupid and simple) applies here. Simple things are easy to understand, and straightforward to implement. Simple things do not cost a lot of time (or money) to implement and are therefore also easy (painless) to revert.
The middle part of this principle “maximizing the amount of work not done” is harder. When implementing agility in an organization this is the cause of discussion. Maximizing the work not done implies that the agile method will skip some processes, code and steps that where necessary in the traditional way of delivering software. This is where agile is linked with Lean. Lean focuses on reducing waste and creating a lean process. Work not done (that was planned to do) with the same end-value, can be classified as waste. In a discussion about work not done, you ask the person asking the question to specify the added business value of the work relative to other tasks. When the added value is not clear this is probably waste. Here you have to discuss the added value of intensive review sessions, bureaucratic procedures, complex frameworks and other corporate rules we meet every day.
Use of critical chain projectmanagement in an AMIS project.
May 26th
Introduction:
At AMIS we have a group of Project Managers who exchange experiences and knowledge with each other. This group read James Leech his book about Critical Chain Project Management and the group agreed that we would try out the ideas described in this book in our projects. The book takes the ideas from Goldratt (The goal) and transfers them from a Production environment to a Project environment. The results described in the book (projects delivered for far less (up to 40%) than the original time and budget) made us very enthusiastic.
The key issue described in the book is to find the ‘bottleneck resources’ in a project, and make sure that these ones are always at work at full capacity. In case of a System Development Process it usually is a top developer or a functional designer. To make sure they can go on with their work, one has to arrange that there is a buffer of work in the chain before them, so they cannot run out of work. One must also make sure that these scarce resources are working on the things that only they can do, for making the team realize their goal (Project delivered in time and on or below budget), and nothing else.
A second important advice being made in the book is that planning estimates tend to be on the ‘safe side’, meaning ‘pretty high’, and often get used up all the way and then some. This happens because when team members have to make an estimate for a certain amount of work, they build in a safety buffer, because they don’t want to ‘fail’ if they don’t make it in time. Often the team manager and/or the project manager do the same thing to the overall planning, which gives another buffer. So one would expect that these estimates will be met during the building phase of the project. Unfortunately that is not often true. Why? Because when a project member has finished his part of the job in a certain amount of time, he tends to make it a little better/more beautiful programmed which consumes the rest of the time, and if it doesn’t works out as planned it can even cost more time than originally planned. He can also slow down or wait to deliver it till all time is really consumed.
What did we do.
In the spring of 2008 we did a small project (~€200.000) with a team varying from 3 people at the start of the project to 8 people at the busiest moment of the project. During the kickoff of the project I mentioned the team that I wanted to implement several advices from the book in the project, and find out what the results are. The team had some difficulties with the assumption that they ‘wasted time’ if they finished early with a piece of work. They felt they were looked at as ‘cheaters’. I convinced them that the advices of the book are not about checking up on them all the time. Even more, I told them that according to the book, in a ccpm steered project the project members are expected to work at a normal speed, without stress, including coffee breaks and small chats with colleagues etc. etc., that sounds even better than a normal project!
These are the things we agreed upon:
To work as fast as possible, without ‘working towards a certain amount of hours that was planned’:
- Create a planning using 50% of the original estimates, and put the remaining 50% in a big Project buffer.
- Send the Project members to work with that 50% planning, and tell them they should do their best to finish it in that time using the credo’s “good is good enough” and “it doesn’t have to win an award for excellent programming”. also, if they need more time, it’s not a disaster, because the 50% planning also has a 50% chance of making it. This means that a work package will often take 70%, sometimes 100%, and it can also take 40% of the original estimate to build.
To avoid that ‘scarce resources’ fall still:
- Find out what the bottleneck resources are, and create a work buffer before them. In our case it meant our functional designer and one of the developers.
- With the client cooperation we realized that the functional Designer could go on with his work all the time, because if he fell still, the whole project would shift in time
- We took care that necessary pre-work for work that the key developer should do, was ready in time by starting extremely early with it.
Client communication:
- We informed our client that we were going to work this way, which could mean that we deliver earlier as expected, and they should be ready for acceptance testing then, because otherwise we would lose the time we had won before. It was o.k. for them, and if I informed them two weeks before actual delivery, they could set up the team and the environment in time. They were enthusiastic about our approach, since there was a tight deadline, and in this way we would have more certainty about making the deadline in an earlier stage, so they were very willing to cooperate.
Internal communication:
- The Delivery Director and I agreed upon a project status update in the form of a very special graph, described in the book. In this graph only two, relative, values are plotted against each other, namely the project build progress, and the project hours consumption, both as a percentage of the total, see figure for an example:
This graph shows you in one glance how your project is doing: If you stay under the blue line, everything is well. The space between the blue and red line is the orange zone, actions should be taken not to end above the red line, because if you are there, your project will get an overrun. As you can see, at 60% of the project progress measures were taken, because the actual buffer usage line was directing to the red line.
- The data for this sheet I derived from my MS Project planning tool, which I use for the administration of the project
What were the experiences?
Advantages
- We did a very early delivery of important pieces of the project, which gave our client certainty that we would make the deadline
- We made the deadline
- The client was very happy to see what they thought up to work in such an early stage
- We completed the project with a small overrun of 6%, due to a screen that was not thoroughly enough thought out, which cost a lot of time. But thanks to the savings on other project parts, thanks to this method, damage was minimized
- The way in which I reported (with the famous graph) illustrated in a glimpse of an eye how the project was doing
- using the data in MS Project it was easy to drill down to where the losses and gains were, which gave insight for following projects
- Some things actually completed in 50% of the time
- The way we approached the project created commitment at the client’s site to get the requirements clear as early and thorough as possible, and the flexibility in the delivery moment for testing, because they were scarce resources too. Thereby we made a very good impression with them for handling the project in this new way
- Except for the programmer working on the tough screen, there was no stress at all in the team
Disadvantages
- With the way to work according to CCPM I see none
It proved difficult to convince the team that I trusted them to always work as hard as they can during the kickoff session, and during the project the enthusiasm lessened in the team. I have to find a way to get that better. At the evaluation of the project the team told me that I informed everybody about the status of the project, except them (blush), so informing them might be a very good start for keeping up the enthousiasm.
Conclusion
We highly recommend using the Critical Chain Project method. The above mentioned advantages show that it is not so that every problem is solved with this method, but that it is very useful in minimizing losses on project parts where nothing needs to be lost. A big bonus which I hadn’t seen coming was the impression we made on the client, and the cooperation and flexibility on their side that it created.
(Dutch) AMIS kennissessie Scrum
Feb 26th
Komende dinsdag is er bij AMIS een kennissessie over Scrum. Aanmelden kan via deze link.
Datum: dinsdag 3 maart 2009
Tijd: 16:30 tot 21:00, incl. diner
Locatie: AMIS, Edisonbaan 15 in Nieuwegein
Doelgroep: Deze sessie is interessant voor zowel developers, projectmanagers, consultants als sales medewerkers.
Coördinator: Robbrecht van Amerongen
Scrum, Just enough, just in time…
Scrum is een methodiek die ons in staat stelt om snel kwalitatief hoogwaardige software op te leveren; er wordt ontwikkeld en opgeleverd wat volgens de inzichten van vandaag nodig is (in plaats van wat maanden geleden eens is bedacht).
Read the rest of this entry »
Agile software development, the principles. Principle 9 : Continuous attention to technical excellence and good design enhances agility
Dec 28th
This is the ninth of 12 posts about the principles of agile software development. Purpose is to go back to the start of the agile manifesto (http://agilemanifesto.org/principles.html) and discuss the implementation of the 12 principles in real life software engineering. Goals of agility are to go deliver software of higher quality, faster, with a higher acceptance to end-users and able to follow the changing business requirements to strive for competitive advantage.
The question is: is this going to work in practice or is this only based on a nice marketing and sales story.
Principle 9: Continuous attention to technical excellence and good design enhances agility.
The first time I looked at this principle I thought: “How is this possible”. Agility focuses on quickly delivering working software (reading: “Quick and dirty”). I experienced this is not true. Attention to technical excellence is making the agile process working better. Technical excellence can make the development process more flexible. In this context I would like to point out that there is a difference between technical excellence / good design compared to complex design and technical complexity. How many developers, designers and architects (the ivory tower / PowerPoint architects) cannot resist creating complex designs, patterns and code with no other purpose than to show off their technical superiority? How many projects have stranded in complex designs and abstract meta-models without creating any business value? These projects are good to be displayed in the Museum Of Modern Software Development but completely useless in the real world.
Smart technology and smart design
Every developer, architect and designer must work with principles of smart design in their minds. In my opinion there are only two principles: 1) the concept must work and 2) other team members understand and are able apply the principle.
Smart technology and good design has its greatest advantage when it is used for the benefit of the whole application and the whole team. Not just for sub-optimal improvements for a single function or a single developer. Good excellent technology and good design has to make coding easy and the application modular, more flexible and adaptable. By using the right frameworks and supporting tools you are able to deliver higher quality software much faster than you where used to. In practice this means using frameworks for common tasks like authorization, persistence and navigation and tools for building, releasing and deploying your application.
In fact all tasks, that are labor intensive and demand a lot of concentration and focus, are likely candidates to be implemented via frameworks and tools.
Agile software development, the principles. Principle 8: Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Oct 3rd
This is the eight of 12 posts about the principles of agile software development. Purpose is to go back to the start of the agile manifesto (http://agilemanifesto.org/principles.html) and discuss the implementation of the 12 principles in real life software engineering. Goals of agility are to go deliver software of higher quality, faster, with a higher acceptance to end-users and able to follow the changing business requirements to strive for competitive advantage.
The question is: is this going to work in practice or is this only based on a nice marketing and sales story.
Principle 8 : Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely
Sustainable development means constant production of softwarefeatures during a long lasting period. This is done without loss of concentration within the team or a rising amount of errors as a result of fatigue or frustration. Software development is like running a marathon and not 100 meters sprint. You have to keep up to speed but not run so fast that you exhaust yourself or your team members.
I (and many of my colleagues) recall working on projects that start relaxed and easy (deadline is still 4 months in the future) and end up working long hours, nights and weekends fixing the project and delivering the end-result. During this period frustration rises and concentration falls. This often leads to situations where we have to fix errors made during last evening of hard working.
The amount of time and effort the team invests in a development process has to be more or less constant during the whole process. This is not only the case for the developers but also for the other team members / stakeholders like testers, designers, sponsors and users. All these groups suffer when the pace varies a lot. During periods of little activity the focus and interest in the project / product demises and in periods of peek activity the stress and frustration reduces the accuracy needed to perform the work correctly(e.g. testing, deployment etc.).
Having recognized this, why do so many projects keep their deliverables with them as long as possible and then (without announcement) throw their artefacts (designs, products, applications and test results) over the wall. Creating a sustainable development process requires your whole team to work together from the first day of the project. All efforts during the process need to be a team effort guided by the process-step expert (designer, developer, tester, application manager, end user etc.). As a project manager you have to prevent a waiting attitude with the team members. Examples of this undesirable attitude are:
-
Senior users: “so, we have stated our requirements. Let’s see what they (the developers) come up with”
-
Developers: “We have realized all features ad stated in the requirements specification. Even those requirements where that we found confusing, contradictory and wrong. We probably hear about this during the acceptance test (or not)”
-
Testers: “fortunately the requirements do not state anything about usability and performance, so we do not have to do anything about this”
-
Application managers: “I hope the architects do not use the single sing on option that is not supported on our platform”
- Stakeholder: “So, this project has started. I do not have to do anything and delivery will be on time and on budget”
As a project manager you have to make sure the team communicates and discusses the process and the product under construction constantly. This will enforce a common ownership about the product and the process. This way you will keep all team members and stakeholders involved in your project.
You can enforce a constant ownership of the product by requiring the project to deliver a new version every week. This requires a smooth running and well oiled build, test, delivery and deployment process. By scripting these process most of these tasks can be performed automatically. Using the correct tools it is possible to compile, deploy and deliver a new version quickly. Within our factory (using maven, continuum/Hudson, Nexus and subversion) we can deploy a new version from scratch within 5 minutes. This creates a constant state of near-shipability of your software. Your team will have a constant productivity during the whole project and not just a huge productivity during the last few weeks.
Stay Alert and creative
Constant and sustainable development enforces alertness and creativity within the team. Team members are not stressed to the maximum to deliver functionality. They are given the time to come up with a creative solution and are alert on the tings happening with other team members or in the project’s environment. This means that the team has to be focussed and aware of the pace of productivity it has to meet. On the other hand, there has to be room for free time and relaxation. This is also known as the 40-hour work week. This does not mean that you have to limit your work to 40 hours a week. When you gain extra functionality (with the same quality) in 50 hours a week you save a 1 month within a project of 5 months. You have to make sure the quality, creativity and focus does not suffer under the fatigue of the team members. Working e.g. 80 hours a week is asking for errors and sub-optimal solutions in the long rung.
Deliver what the organisation can process
To be sustainable in development the user-organisation not only has to be ready to process new version frequently. The project team needs to take into account the amount of new versions and functionality the user-organisation is able to process also. The productivity of these two groups needs to be aligned with each other. This way the development team can produce software that is immediately tested and (preferably) immediately used by the end-users without stressing the end-users with an unmanageable amount of sub-version. On the other hand the user community is able to give the development team direct feedback on the new version delivered. This way the users and developers create valuable software in a combined team.
Conclusion:
To execute your agile projects you have to make sure all stakeholders in the project are constantly aligned with each other. Make sure all stakeholders are in pace with the speed of the project and they are able to participate and absorb the deliverables from the project. Do not stress the team members in such a way that creativity and alertness suffers. During the whole project’s execution keep the effort off all stakeholders as constant as possible. Make sure that running your project is not like running a 100meter sprint but more like running a marathon.
Good Luck!!
Other posts about the AGILE Principles (soon to come):
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity–the art of maximizing the amount of work not done–is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
On Project Success Factors: Take time for research
Aug 28th
In your project, allocate time (and budget!) for research. I used to think that all activities should contribute directly and specifically to the end goal. I now tend more towards the view that you should plan for research. Of course the research should be targeted, but usually a project has a few subjects that will cause you and your team a headache. And generally you can name these areas in advance! Specific problems, let alone solutions, are unclear at the outset. In order to avoid problems down the line it is wise to start your project with research, proofs of concept, name it what you will, to clarify major stumbling blocks.
The most inspiring example of this approach I have been privy to, has been the implementation of Oracle Spatial at a Dutch government agency. A few years back they wanted to consolidate all geographical data from a dozen regional databases to one Oracle Spatial database, covering the Netherlands. This database was needed to allow the introduction of online services to the public. The database had to be completely refreshed every 24 hours, after which the new database would be put online. The most important questions put to this research project were: is it at all possible to load and index all data within 24 hours and what are the response times of typical queries on the consolidated database.
I had the honour to be responsible for shaping this research project into a fixed price project, with a budget that would have been respectable for any “normal” project. In any normal sense though, the project would have failed. It was impossible to achieve the project goals with the then current Oracle release. Of course the next release, due for shipping six months later, made it possible. For any regular project this would have spelled failure. For a research project of this kind: no worries! We simply put the project on hold for six months and implemented our architectural choices on the new version.
The resulting database has been the crucial and stable foundation of the information flows to all clients and continues to do so. An excellent approach.
