The best solution may not be the best solution. Depending on the scope in which you consider the alternatives, you may arrive at the wrong conclusion. Consider only the problem at hand, there may be an apparently easy solution that in the grander scheme of things is not optimal and perhaps even detrimental. Accepting that you do not decide the scope to apply and are forced to work in a scope much larger than you can oversee can be quite hard. Agreeing to objectives, constraints, decisions that fit in with that larger scope – but seem conflicting with the one you thought to be dealing with – can be difficult and frustrating. Yet, you may have to submit – just as you pay taxes and follow traffic rules.
(from unsplash.com – Grillot edouard)
This may all sound a little abstract. Let me give you an example. The team of database specialists I work with is busy with planning a few major steps for the existing applications & platform. From over a 1000 local instances – database, applications, integration platform, low code platform – we are moving to a small number of central, consolidated and cloud based instances. A major move. Part of this landscape is a substantial number of scheduled jobs that run throughout the day. Most jobs do database intensive work and a large number runs only stored procedures inside the database platform. The scheduling currently is done largely through shell-scripts, some cron and a homegrown mechanism. This will not do for the much larger scale and more complex requirements of the centralized environment. So we need a new solution design for job scheduling.
The team of specialists takes a close look at the challenge. Within their scope, the solution seems an easy one. The database they work with has its own internal job scheduling mechanism that provides almost all functionality required for scheduling database jobs. The knowledge required for developing the scheduled and managing their execution is already available in the team. The mental effort and associated risk is small, their motivation is high. It seems a slam dunk.
However: there is a larger scope that we are working within. The organisation has a plethora of technologies in use, old and new. Dozens of teams use their own solutions for what in essence are the same problems: logging, monitoring, web development, messaging, API design/implementation. And job scheduling. Managing, maintaining and further evolving their existing application portfolio is extremely challenging. One of the ways in which the organisation wants to address this fragmentation of technology, skills and experience is by consolidating on a single technology stack (or at least as much as possible). While motivated exceptions can still exist, all teams are instructed to adopt the same solutions for the same problems. This to allow people to move more easily between teams and products, reduce dependencies on individuals and their knowledge of (esoteric) technologies and homegrown solutions, bring in fresh blood from outside the (aging) pool of current resources and benefit from developments at large in the industry – in the developer community and with vendors.
As part of the unified technology stack, the enterprise wide architecture team has selected a job scheduling solution. It is understood that it is probably not the best fit for any team’s specific situation. But it is a good enough solution for almost any situation. It allows to build a shared knowledge base, reusable components, skills and experience across teams. There still may be instances where for a specific requirement this enterprise scheduling solution needs to be complemented with a one off local solution. These exceptions – when motivated – can be approved (as explicit Architectural Decisions).
The slow but steady movement of the organisation as a whole towards a homogenous technology stack with the associated long term benefits is the overarching goal. And it trumps the (short term, very local and perhaps individual-inspired) optimization of one team. Mont Blanc may be the highest mountain in their scope – but looking at the larger scope there is a much higher mountain to be found.
(depending on the scope, the highest mountain may not be the one you are thinking of, given that Olympus Mons has an altitude of over 21 km.
It is located on Mars. Is that in scope?
