The tools you work with have a lot of impact on what you can accomplish and the more sophisticated the tools the better, especially in software. Beagle Research just completed a study into using a DevOps strategy with the Salesforce Lightning Platform. You can get a copy of it here. The work was sponsored by Copado a DevOps solutions provider. DevOps is a strategy for building, changing and deploying enterprise software that can also be used with a Scrum or Agile methodology as well as others. More than concentrating on code and coding, DevOps is more holistic looking at culture and infrastructure in its broadest manifestation.
Even if you’ve never built systems you can surmise that planning, developing, assessing, testing and deploying software are all critical milestones and they’re often spread across technical departments of IT like development and operations, hence the name.
It can be challenging to compare enterprise software strategies. For instance, using an on-premise hardware and software stack has been common for decades but with the development of cloud computing, users find they can eliminate having to care about a good deal of their development environments leaving it all to the cloud vendor. How do you compare overhead and costs between cloud and on-premise? What are the effects on speed to market, reliability, security? To control for some variables and enable us to make an apples-to-apples comparison, we chose to research only companies developing and maintaining systems using Salesforce Lightning and a DevOps strategy.
Companies ranged in size from fewer than 100 employees to more than 10,000. There were similar measures for number of salesforce users and number of developers as well as the number of production orgs. Two-thirds or 67 percent said they run between 2 and 7 production orgs. Most of the respondents were C-level executives (48 percent) or upper management (40 percent).
We found that DevOps is delivering value for most of its users though the larger organizations have greater challenges, more on that in a moment; 17 percent claim over $5 million in benefits from using a DevOps strategy. These people have a good understanding that software flexibility drives business agility and impressively, 54 percent say their lead time for making changes to their Salesforce orgs is between one day and one week. Compare that to a more traditional process that takes weeks or months.
But we also identified an elite group that operates even faster–21 percent say their lead time for making changes is less than a day, and 8 percent say it takes less than an hour. Taken together 83 percent can make changes in a week or less.
In other recent research I’ve been involved in, delivering running, tested and deployable code was much slower. Clearly, if a business depends on its ability to quickly change to meet changing market demands this is where you want to be.
On the other hand
As you might expect though, the benefits of a DevOps strategy were not evenly distributed across all users. Generally, smaller businesses with smaller development groups did better overall at establishing DevOps programs and at excelling within them.
The most successful businesses using DevOps are those that use a well-integrated set of tools to move through development and deployment. Many organizations, especially smaller ones, use a combination of in-house developed and opensource management tools. At best the great variety of tool choices suggested to me that some best practices are still being worked out.
Even with Salesforce Lightning and a DevOps approach you can still have issues and almost everyone had the experience of deploying a release to a production org and having a service degradation. A plurality of respondents, 43 percent, said a problem occurred up to 15 percent of the time and the vast majority or 86 percent said service degradations happen less than half of the time. This is an important snapshot of the state of the industry. Speed of delivery slightly exceeds stability of releases indicating a need to bring the two metrics more in alignment.
Some best practices considerations
- A strong majority (60 percent) say each developer in the business has a private development environment.
- Also, 77 percent say they use version control to store code and click-based Salesforce customizations.
- Most synchronize their development environments with the latest changes from other teams with 41 percent doing this on-demand or at most once per day and 42 percent saying they do this between once per day and once a week.
- 75 percent say changes made in version control trigger automation tests.
- 87 percent have confidence that when automated tests pass the software is ready for release. However, meta-analysis of the data strongly suggests that the greater a team’s confidence in their tests, the higher their change failure rate. Skeptics who were neutral on this question experienced a 40% lower change fail rate than those who expressed strong confidence in their tests.
It’s good to be skeptical.
Part of the allure of the digital disruption is having the capacity to change a business process to take advantage of changing market conditions and many businesses are already having that experience. Big data and analytics tell us what needs attention but then we still need to change our systems’ behaviors. Flexible software contributes to a business’ agility and that’s good. But that speed and flexibility need to be balanced by security and what I can only call the bulletproof-ness of the new or changed code.
The businesses most able to reap the rewards of DevOps tend to be smaller though large enterprises have their bragging points. While larger businesses already see benefits from a DevOps strategy, they are the ones with the greatest potential to do more. What’s holding them back?
In any organization size breeds complexity which causes business friction. We don’t have all the data to say so unequivocally, but it seems that bigger organizations have more walls to break down.
It looks to me like the development tools are pretty good. Not enough businesses have well integrated management suites to handle the complexity and it also seems like culture forms stovepipes which causes less stellar performance. If that’s so there’s still some cultural work to be done enhancing communications within and between developer groups and the business. DevOps tools can be a big part of that help but as with the psychiatrist trying to change a lightbulb, the bulb still has to want to change.