3 - Dealing with Software Packages and Software Vendors

“If you don’t know what you want,” the doorman said, “you end up with a lot you don’t.”

In many organizations, the language that describes the software package in use—usually a COTS product that has been chosen for its features—is less than favorable. Clearly this software, often considered legacy, did not magically appear in the organization; someone made the decision to purchase a software package as a response to a business problem. There are good reasons to not reinvent the wheel but leverage packaged software instead. Unfortunately, the state of many software packages these days is such that they don’t behave like modern applications should. In this chapter, I will discuss the criteria that you should consider when choosing a software package and how you can work to improve the software package you already have, and I will provide some exercises at the end to adapt these guidelines for yourself. Let’s talk first about the idea behind software packages.

The original purpose of software packages was to support commodity processes in your organization. These processes are very similar to those of other organizations and do not differentiate you from your competitors. And even though many of these software packages are now delivered as software as a service (SaaS), your organization has legacy package solutions that you have to maintain.

The problem is that many organizations that adopted software packages ended up customizing the product so much that the upgrade path has become expensive. For example, I have seen multiple Siebel upgrades that cost many millions of dollars. When the upgrade path is expensive, it means that newer, better, safer functionality that comes with newer versions of the package is often not available to the organization for years. Besides this downside, heavy customization over time to make the software support all the requirements from business also means that each further change becomes more expensive and that technical debt increases over time.

If we also recognize that the concept of an end-state architecture is not applicable anymore, it becomes clear that software packages should be designed so that they can follow a life cycle that includes eventual decommission for a new, different system by making it easy to decouple them from the rest of the architecture. Every component in the architecture should be seen as temporary so that you are not bound to a specific software vendor. The IT market evolves so quickly that the safe bet today might be a liability in the near future. Dealing with a software package for which the vendor has gone under or where the specific package is not strategic to the vendor anymore is a nightmare, as defect fixes and other support become harder and harder to get. I have been in this situation multiple times and ended up trying to reverse engineer the software in my team to deal with problems, which is expensive and time consuming.

Application architecture is a key factor to determine the speed at which you can deliver and is one of the hardest and slowest things to change. Considering this, the choice of introducing a new software package into your organization should be one that considers the impact to the overall architecture and not something that should be driven purely by functionality alone.

A lot of IT delivery nowadays is closer to assembly with LEGO blocks than it is to handcrafting something with Play-Doh. To allow for this, the applications you leverage should be modular, leveraging an open architecture and good engineering practices so that you can maximize benefits from them. If that’s not the case, it would be like having LEGO blocks that all have different connecting mechanisms, so you end up putting glue between the blocks because nothing fits together otherwise. This, of course, defeats the purpose of the module blocks and will make changes to the structure more difficult.

Warlike arguments have been fought over which IT product to choose for a project or business function. Should you use SalesForce, Siebel, or Microsoft as your CRM system, for example? Just looking at the functionality is not sufficient anymore, because as much as it should be our preference, it is very unlikely that an organization will use the product as is. The application architecture the product will be part of will continue to evolve, which often requires changes to the software product.

Architecture and engineering principles play a much larger role than in the past due to continuous evolution of the architecture. This puts a very different view on product choice. Of course, the choice is always contextual for each company and each area of business. Your decision framework should guide the decision to make sure all factors are considered. And while the decision might be different for each of you, what I can do is provide a technology decision framework (TDF) that helps you think more broadly about technology choices before you make them.

My TDF is based on three dimensions for you to evaluate: (1) functionality, (2) architecture maturity, and (3) engineering principles.

Very often the functionality provided by the software package is the key decision factor. The closer the functionality aligns with the process that you want to support, the better a choice it will be. For you to determine whether a software package is suitable or whether you should build a custom system (which hopefully leverages open-source libraries and modules, so you aren’t starting from scratch) requires that you take a good, hard look at your organization. Two factors will be important in this decision: your flexibility in the process you are trying to support and your engineering capabilities. If you are not very flexible and you have a custom-made process, then leveraging a software product will likely require a lot of expensive customizations. If you don’t have a strong engineering capability either in-house or through one of your strategic partners, then leveraging a software package is perhaps the better choice. You need to understand where you stand on the continuum from flexible process and low engineering capability (= package) to a custom-made process and high engineering capability (= custom solution).

If you land on the side of a software package, then create an inventory of the required functionality as requirements or user stories, and evaluate the candidate packages. Ideally, you want real business users to be involved in the evaluation to make sure it is functionally appropriate for the business. The idea is that a package is giving you a lot right out of the box, and it shouldn’t be too much hassle to get a demo installed in your environment for this purpose. If it is a hassle, then that’s a warning sign for you.

Application architecture maturity is important to the ongoing support of your application, as a well-architected application will make it easier for you to manage and maintain the application. If you build an application yourself, you will have to deal with architecture considerations, such as scaling and monitoring, and the better your IT capability is, the more you are able to build these architecture aspects yourself. Otherwise, you can choose a package solution to do this for you.

Four aspects that you can use to start the assessment of architecture maturity are as follows:

Engineering principles increase in importance the more you believe that the application will have to evolve in the future; and this, in turn, is often driven by the strategic importance of the application for your customer interactions. Good engineering principles in an application allow you to quickly change things and scale up delivery to support increasing volumes of change. The better skilled your IT department is, the more it will be able to leverage these principles and patterns. If you don’t have strong IT capabilities, then you will focus more on the built-in architecture features. Here are a few things to look out for:

To help you better select the most appropriate product for your needs, score each of the proposed products on the four areas we’ve discussed: functionality, architecture, engineering capability, and in-house IT capability. I have provided an example scorecard in Table 3.1, which you can use as a starting point for your own evaluation.

Table 3.1: Example scorecard: New applications should be evaluated on four dimensions, not just functionality

While the advice in the last section is great for introducing new applications, you will still have to deal with your existing legacy application portfolio. I will describe later how to evolve your application architecture if you have control over it. Here, we will look at what you can do if the application is not fully in your control.

You are probably already working with a list of applications, and some of them are supported by software vendors who created the software or made specific changes for you, or both. Yet when you look at the application, it is quite possible that it is not following the modern architecture and engineering principles as I described earlier. You will likely not want to invest in completely replacing these systems immediately, so you have to find other ways to deal with them There are four principle options that I recommend exploring with such software vendors: (1) leverage your system integrators, (2) leverage user groups, (3) strangle the application, and/or (4) incentivize the vendor to improve their product.

Of course, not all vendors and applications are created equally. There are multimillion-dollar organizations that you have to deal with differently than with your small-scale vendor that has only one application to support. I think the four basic patterns apply across the spectrum, but you should tailor them to your context.

Having spent my entire career either working for a software vendor or system integrator, I find it surprising that so much is left on the table when it comes to using the relationship effectively beyond the immediate task at hand. If you work with a large system integrator (SI) to maintain and develop an application, it is likely that the SI is working with the same application in many other places. While you have some leverage with the software vendor, the SI can use the leverage he has across several clients to influence the application vendor. The better and more aligned your organization is with your SI on how applications should be developed, the easier it will be to successfully influence the software vendor. Better leverage with software application vendors to change their architecture is only one of many benefits that you can derive from changing your relationship with your SI.

For most popular applications, there are user groups, which can be another powerful channel to provide feedback to the vendor. Sometimes these are organized by the vendor itself; sometimes they are independent. In either case, it is worthwhile to find allies who also want to improve the application architecture in line with modern practices. Having a group of clients approach the vendor with the same request can be very powerful. A few years back, I was working with an Agile software that was unable to provide reporting based on story points, relying instead on hour tracking. The product vendor always told my client, my colleagues, and me that our request was unique and hence not a high priority for them. We could only get traction once we had reached out to some other organizations that, unsurprisingly, had issued the same request and had received the same response. The vendor was clearly not transparent with us. Once we had found an alliance of customers, the vendor took our feedback more seriously and fixed the problem.

I encourage you to look for these user groups as a way to find potential allies as well as work-arounds and “hacks” that you can leverage in the meantime. By now, people worldwide have solved how to leverage DevOps practices for applications that are not very suitable on paper for the implementation of DevOps. And the good news is that DevOps enthusiasts are usually very happy to share that information.

As discussed in the previous chapter, when the application is not changing and you have to divest from it, you can use an analogy to the strangler pattern in software development to slowly move away from the application. Reduce the investment in the application and reinvest to build new functionality somewhere else that is more aligned with the architecture you have in mind. Be transparent that you are doing this because the software vendor is not providing the capabilities that you are looking for, but you would reconsider if and when those capabilities were available. This will incentivize the software vendor to look into investing into a better architecture (perhaps the reason that the capabilities don’t exist is simply because no one ever asked for them before). Make sure to explain why locked-down architecture and tools are not appropriate going forward and that your requirements for modern architecture require changes in the application architecture. If the software vendor decides that those capabilities are just not the right ones for their application, then not investing any further into the application and spending your money somewhere else is the right thing to do anyway to enable the next evolution of your architecture.

I always prefer the carrot over the stick; you, too, should look for win-win situations. Improvements in the architecture and engineering of an application will lead to benefits on your side, which you can use to incentivize the vendor. What is usually even more effective is to show how changes will make the application more attractive for your organization and how more and more licenses will be required over time. This is the ultimate incentive for vendors to improve their architecture. And of course, you can present publicly how great the application is and hence create new customers—a win for both parties.

As I said in the beginning of this chapter, the opinion on software packages in many organizations is not great. It is therefore surprising that organizations do not actively manage what software they do use and how little effort goes into engaging their software vendors to improve the situation. I truly believe that vendors would be happy to come to the party more effectively if more organizations would ask the right questions. After all, why would a software vendor invest in DevOps and Agile–aligned architectures when all every customer is asking for is more functionality and no one is paying for architecture improvements? If companies engaged vendors to discuss the way they want to manage the software package and how important DevOps practices are for them in that process, vendors would invest more to improve those capabilities.

If all else fails and you feel courageous and curious, then you can ignore the guidance from your vendors and attempt to adopt DevOps techniques yourself, even to software products that don’t easily lend themselves to those techniques. This is how you start:

In my team, we have used these techniques for applications like Siebel, SAP, Salesforce, and Pega.*

The above techniques will, I hope, help you to better drive your own destiny and be part of a thriving ecosystem where IT is a real enabler. The last piece of the ecosystem that I want to explore is the role of the system integrator, a topic obviously close to my heart, which I’ll address in the next chapter.

First Steps for Your Organization

Determine Guidelines for New Applications

Based on the sample scorecard provided in the Table 3.1, derive a scorecard for your organization. Take the next product decision (or a historic one if nothing is coming up) and apply this scorecard to see how it differs from your current process. Determine whether or not this scorecard, with an architecture focus, provides a different result. I would recommend inviting stakeholders across the organization to a workshop to discuss results and next steps to change your evaluation process going forward.

Strengthen Your Architecture by Creating an Empowering Ecosystem

So, you already have software packages in your organization like so many others. In the previous chapter, we did an analysis of your application portfolio, which you can leverage now to determine which software packages are strategic for your organization.

Based on the previous application portfolio analysis (or another means), determine a small subset of strategic applications (such as the first minimum viable cluster) to devise a strategy for creating an empowered ecosystem around them.
Now pick these strategic packages and run the scorecard from this chapter. You can largely ignore the functional aspects, as they are used more for the choice between package and custom software. You could, however, use the full scorecard in case you are willing to reconsider whether your current choice is the right one. Given that you are doing this after the fact, you will already know how suitable the package was by the amount of customizations that your organization has already made.
Where you identify weaknesses in your software package, determine your strategy for them. How will you work with the software vendor to improve the capabilities? Will you work with them directly? Will you leverage a system integrator or engage with a user group?
Results take time. Determine a realistic review frequency to see whether or not your empowered ecosystem is helping you improve the applications you are working with. You can leverage the principles for measuring technical debt from the previous chapter as a starting point if you don’t have any other means to measure the improvements in your packaged applications.

		Product A	Product B	Product C
Functionality	Func. Area 1
	Func. Area 2
	Func. Area 3
Architecture	Auto Scaling
	Self-Healing
	Monitoring
	Changeability
Engineering Capability	Source Code
	APIs
	Modularity
	Cloud Enablement
In-House IT Capability