DevOps Disambiguation: The Journey of Software-Based Services
In This Article
When having conversations with customers or colleagues outside the DevOps community, we often find ourselves misunderstanding one another when discussing cultural and technical practices. We're currently seeing almost daily advances in how to develop, build, deploy, and operate software-based services. Keeping up with this pace of innovation, and the accompanying acronyms and buzzwords can begin to feel like a full-time job.
In the article below, we take a deep dive into the world of software-based services to explore:
The journey of software-based services over the past 20 years, including advancements that have been made during this time.
The common terms, acronyms and buzzwords associated with the delivery and operation of software-based services, and how these have been developed in-line with the various advancements in this space.
The main methodologies used to build, deploy and operate software-based services, and how these interact with and relate to each other.
The new and emerging buzzwords resulting from further technological innovations within software-development.
During the 90's, the use of computers started to increase rapidly within organisations, however the commercial-off-the-shelf software (COTS) wasn't always tailored enough to support business processes. In other words, companies needed better software. One option was to partner with a software company; however, the results were not always satisfactory, and with programming languages becoming simpler and software developers more readily available, companies soon began developing their own software instead.
When it comes to non-software companies that predominantly use the COTS software, the IT department is often heavily outsourced with a relatively small budget. In comparison, rapid delivery of quality, bespoke software-based services require a very different type of IT Department, accompanied by a much larger budget. Unfortunately, IT departments are typically seen as a cost-centre, meaning they have to make do with budgets that are never quite large enough to fund all needs of a software-house style team.
After a fair amount of trial and error, the technology industry arrived at methodologies that helped non-software companies to overcome these challenges and allow their IT departments to deliver quality software-based services at an acceptable cost and speed.
Let's start by imagining that a product owner (PO) is providing requirements for the functionality of a fully operational email client upfront. How easy is it for a human to achieve this? How easy is it to imagine all the fine details, components and how they will interact and influence each other without seeing or trying them? Do we know everything we do not know at this stage? It is no wonder so much reworking was required when using this method, known as the "Waterfall" approach.
Contrast the above scenario to having to provide requirements for the "Receive Email" functionality only. The scope is small - easy for the PO to imagine what is needed and simple to describe in enough detail for a delivery team to understand and deliver the desired outcome. This deliverable, and the changes and tweaks involved in creating it, can now be used as an example of real-life "working software" that provides a strong foundation for the next piece of functionality. This approach is known as "Agile".
Agile takes into account human strengths and weaknesses. We can do a great job when working with something relatively small and tangible but very few of us can handle something large and nebulous with multiple dependencies and unclear risks. This was the root of its success.
- Agile: A family of software development frameworks built around a set of common principles and values, a number of which are listed below:
- Encouragement of inter/intra-team collaboration, minimisation of red-tape and silos
- More is gained through personal communication and close collaboration - the main deliverable in software development is working software not extensive documentation
- Close collaboration with customers is the best way to discover what is needed, no contracts can substitute this
- When planning, it is important to leave room for changes as they are an important and inevitable part of the plan
- Minimising the amount of upfront planning and design - work is broken down into small increments which are then typically executed in small batches.
- Agile is supported by a set of development practices including; Scaled Agile Framework (SAFe), continuous integration (CI), continuous deployment/delivery (CD), and cross-functional teams (T-shaped staff) to name a few.
Intended Audience: scrum masters, product owners, developers, testers
Monthly web searches: 1.6m
SAFe: Scaled Agile Framework. A popular framework allowing Agile methodology to be successfully used at an enterprise scale.
Intended Audience: same as Agile
Monthly web searches: 30k
Continuous Integration. A practice used in software development whereby team members integrate their work on a regular basis, often multiple times a day. Each integration triggers an automated build and testing, ensuring rapid detection of any integration errors, thus avoiding major integration issues.
Continuous Delivery. The same principle as Continuous Deployment, however the "deploy to production" task is triggered manually, allowing for additional quality assurance checks.
Continuous Deployment. The ability to deliver software from the point of committing code to "production" without manual intervention using automated build, test, deployment and release management.
Intended Audience: CI - Developers, CD – Developers, Operations, QA, Shared services
Monthly web searches: CI – 90k, C. Delivery – 15k, C. Deployment – 7k
T-Shaped staff: A key pillar of Agile and DevOps, both of which utilised ideas and approaches from the Toyota Production System (TPS) which enabled Toyota to survive the oil crisis in the 70's. Workers in the TPS model pose a multitude of skills, which enable them to work at different parts of a conveyor belt. This makes it very easy to redeploy staff, as well as increasing the speed and quality of production as staff on the conveyor belt understand what happens both up-and-downstream from them. This is the essence of being 'T-shaped'.
Intended Audience: Anyone
Monthly web searches: 15.5k
Unfortunately, the improvements in developing software, brought out by Agile, resulted in a lack of balance in the chain: Build -> Deploy -> Operate. Developers could now quickly produce software; however, the rest of the downstream processes could not support deploying it fast enough.
Furthermore, support teams could not keep up with the rate of change in an application they were responsible for, leading to a decline in the quality of support. As a result, Operations started to escalate issues to developers, which in turn resulted in unbalanced writing software, thus disrupting the improvements that were made in the first place. This situation created a reaction against the silos and inflexible standards that resulted from existing practices, and the need for systemwide improvements now known as DevOps.
DevOps: Development Operations. A methodology that enables teams to deliver better quality software-based services faster to their customers. The three core focus areas for DevOps are tools, processes and culture, applicable from "design" right through to the "maintain" phase of a Software Development LifeCycle.
It is important to remember that all three areas are interrelated. For example, tooling affects culture and processes as much as culture affects processes and tools.
Intended Audience: Developers, Operations, QA, Shared services (e.g. Release Control)
Monthly web searches: 700k
CALMS: Culture, Automation, Lean (management), Measurement, and Sharing. A model that helps us to understand the key DevOps principles and methodologies, whilst also enabling us to measure the success of DevOps implementation within an organisation.
Intended Audience: same as DevOps
Monthly web searches: 2k
SDLC: Software Development LifeCycle. A framework used to deliver high-quality software-based services to customers, with seven distinct phases: analysis -> feasibility study -> design -> develop -> test -> deploy -> maintain.
Intended Audience: Users, Product Owners, IT Department
Monthly web searches: 345k
SSDLC: Secure SDLC. Enhanced SDLC with an emphasis on delivering software in a secure way.
Intended Audience: same as SDLC
Monthly web searches: 5k
SRE: Site Reliability Engineering. An implementation of DevOps in the Operations space. SRE teams originated at Google and were mostly comprised of software developers, who's responsibilities consisted of Ops (availability, capacity, performance, change management and all levels of support) and Devs (development of software solutions to improve operational aspects of the systems a software team looks after). Outside the context of Google, an SRE team is typically made up of systems engineers/administrators (sysadmins), who's areas of responsibility are similar to those of a Google SRE but rather than writing software they mainly focus on the automation of operational tasks.
Intended Audience: Developers, Infrastructure
Monthly web searches: 12k
It is worth noting that whilst sysadmins are responsible for SRE in "traditional environments", this is not the case in Cloud environments, where developers are starting to own a great deal of these tasks thanks to the invention of IaC.
Infrastructure as Code (IaC): The concept of managing the complete lifecycle of infrastructure using development practices, and quickly becoming the standard way of managing infrastructure residing in a Cloud environment.
Intended Audience: Developers, Infrastructure teams
Monthly web searches: 5k
Just as we thought Agile, DevOps and SRE had fixed it all, we realised there's room for further improvements. Ever stricter regulations on the handling of customer data, and protection of Intellectual Property e.g. trading algorithms, have led to a significant increase in requirements when it comes to developing and operating software. It is too costly to scale a "traditional" InfoSec department to enable full support at Agile/DevOps speed. The inability of delivery teams to keep up with the volume and complexity of information security requirements (Application Security in particular), has become a significant blocker for Agile delivery teams. Hence, the introduction of DevSecOps.
DevSecOps: DevOps which ensures the delivery of software-based solution in a secure way. This is the latest addition to the batch of "continuous" processes: "Continuous Security" at every stage of an application or infrastructure lifecycle. Some see DevSecOps as a wrapper around DevOps which could lead to the "Dev" element not partaking and leaving the InfoSec team to lead the initiative. This, of course, does not achieve the desired outcome - "everyone is responsible for security".
Intended Audience: Same as DevOps plus Information Security (Application Security)
Monthly web searches: 35k
Now that we've explored the background and history of existing terms, as well as improvements in the delivery of software-based services over the past 20 years, let's look at what the future holds – including new and emerging buzzwords.
Hyperautomation: An end to end automation which makes use of Artificial Intelligence (AI), Robotic Process Automation (RPA) and Machine Learning (ML) to augment humans and further automate tasks that currently require human interaction. There is a particular emphasis on using AI & ML to enable autonomous decision making, for example not only attempting to fix the issue of too many alerts, but also taking actions based on those alerts.
Intended Audience: Anyone
Monthly web searches: 0.5k
AIOps: Artificial Intelligence for IT Operations: The use of AI, ML and Big Data to detect and resolve IT problems. For example, when a storage device goes down, a monitoring system sends alerts about all systems that use the device, in addition to alerts about the device itself. AIOps aims to suppress all alerts apart from the relevant ones, resulting in faster identification and resolution of the root cause.
Intended Audience: IT Operations
Monthly web searches: 1k
GitOps: A way of delivering software-based services in environments built on Kubernetes. Developers and Operations collaborate using flows that are based on capabilities offered by the git-version control system. The resulting source code is then piped into a Continuous Deployment style system which then releases it into production. The practice was developed at WeaveWorks in 2017 who used git repositories to manage the lifecycle of Kubernetes configuration. Hence, it is best known in the context of Kubernetes, however there is also potential for it to be applied to environments built using cloud-native technologies.
Intended Audience: Same as DevOps
Monthly web searches: 6k
DataOps: DevOps for data. An approach that applies to the entire lifecycle of data analytics, from collection to delivery, and aims to reduce the cycle time.
Intended Audience: Analytics and data teams
Monthly web searches: 7k
Given the trends we've discussed above, what are the next steps for DevOps? What should DevOps toolchains be like to support the speed of modern delivery? Is having chatbots assisting developers sufficient? Should solutions produced by No-Code be centrally supported? How should delivery teams make the most out of AIOps?
When evaluating DevOps and Agile frameworks, it's important to understand a wide range of challenges. There is an increasing push to automate business processes, adapt cloud technologies and the need for better collaboration between IT and the business. At WWT, our DevOps experts can help you evaluate, design, implement and operate the right DevOps processes and tools to enhance your operational efficiency and drive business outcomes.