Silos - Shadows - Silver Bullets - Shiny Objects
Architectual Anti-patterns
Architectual Anti-patterns
IT delivery and projects have always been focused on solving problems. Very discrete problems. Each project was unique. In the mainframe era technical limitations (green screens, etc.) limited what could be delivered. IT gave the customer what they could; not what they needed. This led to an interesting "compromise" where business would start asking for solutions they thought they could get rather than of asking for what they needed.
On very large projects another common phenomenon would occur. These projects tended to multi-release, multi-year endeavors. The business knew this was their chance to get what they needed. So requirements gathering was a very long task (it comes to the point where one has to concede that requirements gathering is never ending). It was never complete (how many long projects have scope creep?). To deliver anything, scope would be cut so a release could be delivered. The release was rarely well received and disillusionment means that subsequent releases woould be limited. Reinforcing the business perspective that to get what they need they must get even more requirements into the next large project (Repeat).
The anti-patterns are a result of these systemic project problems. The goal is to move from specific problem solving to one of continuous improvement. The industry has made positive attempts (agile), but success is less dependent on the methodology than on other factors. Agile without an underlying established foundation is just as likely to create an anti-pattern as traditional methods.
Here are the anti-patterns to avoid.
Everybody know Silos are bad. We've been hearing this for decades. Silos are an anti-pattern because they inhibit the enterprise from fully leveraging its assets. There are several common characteristics of a Silo.
Silos usually have their own User Interface (UI). To get to the asset you have to go to the silo. This complicates user experience because they often have to go to several disparate systems to complete a task. This results in longer training times, longer service times, and an increase in product consistency.
Silos usually have their own security implementation. Security is either built into the application or specialized application resources are secured on a platform level. This means that with each silo the enterprises security posture grows. This non standard growth of security assets results in more exposure and requires more support.
The development "language" is specialized. Whether it be the mainframe or client server or clioud based, a silo tends to be written whole cloth in a single programming paradigm. Even if the silos are similar (e.g. mainframe) they often have unique characteristics. Replacing silos often results in just a move of the silo. The result of silos is that IT support model uniquely grows with each new silo and silo speicific skills are harder to acquire.
The silo data layer is unique and over functionlized. Databases are the life-blood of modern applications. In siloed applications the database is tightly coupled and highly tuned to support the application. It makes exposing specific application capabilities difficult. It makes usage of the data by AI/analytics platforms difficult to expose because it requires "cleansing" to make it "analytics ready".
Silos have two operational issues. One is that over time, because of the characteristics above, it becomes hard to support. It requires specific technical skills and application knowledge. The second is that silos limit the ability to continuously improve and leverage newer technologies. They have all the qualities of a locked in solution. Conversion seems to be the only option, but without an enterprise view most conversions result in a new silo.
Shadows are an interesting anti-pattern. They are not IT made. They are often a result of a perception of the lack of IT responsiveness. The business (usually business units) think they have to take control of some part of IT to meet their goals.
One common shadow is when a unit takes on some new technology to meet their needs. This is often done without enterprise knowledge and even IT operational support. The business unit builds up the development team and delivers the solutions. It is often very successful, but it has all the characteristics of a silo. It has its own skills (language), security, UI, and data layer. Longer term it is harder to support as newer technologies come along that will be better suited. It is a silo. A common result is IT department eventually has to take over the technology.
The other shadow is longer and more ominous. It is something that we have all done. It is the creation of assets outside of enterprise visibility. We all have our "private" spreadsheets, reports, and even processes. There is incredible value in these assets, but the enterprise has no way of leveraging them. The existence of these assets are a risk. What if the owner leaves? This is institutional knowledge codified but inaccessible.
Shadows are another form of a silo. They have their own language, UI, and data. They often only have security by obfuscation. The fact that nobody knows about them makes them "secure". The lack of visibility of these assets impedes business transformation and innovation.
Silver bullets are very popular with business financial departments. The pitch is that all you have to do is buy an specific integrated solution and all your problems will go away. This concept is core to CRM, ERP, Integrated X systems, etc. Sometimes a silver bullet may solve the problem; the problem could be a werewolf. More often than not either the silver bullet has to be "customized" or the enterprise adapts their processes to the solution. In some cases, the enterprise is so limited (staff, skills, budget) that a silver bullet is a reasonable answer.
The limitations of the silver bullet are many. Innovation only happens on the vendors schedule and then only to the extent they allow it. The data is in a format to serve the solution and is not easily exposed for other uses AI, analytics, integration). The enterprise is completely dependent on the vendor and their support model. Security is always application/platform based. The enterprise is locked-in. Converting to another solution will be very expensive and time consuming.
A silver bullet is a silo. It has its own language, security, UI, and data layer. It is a silo where the experts are outside the enterprises' control.
Shiny objects are very common. Did you ever watch sports on the weekend and see a commercial hawking some technology - AI, blockchain, etc.? Did you ever wonder who they were selling to? The message - everyone is using this technology and if you don't you are falling far behind.
Its intent is to feed the FOMO in your executives. It is expected that the executives will rush to fund a pilot on this technology.
Silver bullet pilots have some interesting characteristics. Usually, to limit risk, a low impact problem is selected. The need for the technology is not in the equation (it usually can be done another way), the goal is to see what the technology can do.
A specialized team of some of the organization's best technicians are assigned. The vendor either has a major role (and some cost that allows the vendor to "train up" new staff on the hot technology) or the organization tries to go it on their own using some open source or easily acquired technology.
The team spends much of the time just "getting" the technology to work. Less time was spent picking the project or trying to understand its long term best use.
Shiny objects follow a similar path as Shadows and Silver Bullets. If successful, demand becomes an issue. Since this technology is now critical, a new team is formed to make it repeatable and supportable (by determining the long term best use and structuring it). The pilot becomes another hard to support silo because the team is moving too fast to "refactor" the solution. (I have seen organizations with 3 silos of the same technology, none really successful).
Unsuccessful projects, if put in production, are another type of silo.
Why do we have these anti-patterns? Well, they sort of worked, at least for awhile. It was thought that keeping projects small would result in more success. This is true, but as the project functionality is bound, so is the long term thinking. Reuse, continuous improvement, and product management may be the answer, but only when those are based on a sound architectural foundation.
The Right Strategy is one such architectural foundation.