From #KubeCon, Three Things Happening with the Kubernetes Market

This year’s KubeCon+CloudNativeCon was, to say the least, an experience. Normally sunny San Diego treated conference-goers to torrential downpours. The unusual weather turned the block party event into a bit of a sog. My shoes are still drying out. The record crowds – this year’s attendance was 12,000 up from last year’s 8000 in Seattle – made navigating the show floor a challenge for many attendees.

Despite the weather and the crowds, this was an exciting KubeCon+CloudNativeCon. On display was the maturation of the Kubernetes and container market. Both the technology and the best practices discussions were less about “what is Kubernetes” and, instead more about “how does this fit into my architecture?” and “how enterprise-ready is this stuff?” This shift from the “what” to the “how” is a sign that Kubernetes is heading quickly to the mainstream. There are other indicators at Kubecon+CloudNativeCon that, to me, show Kubernetes maturing into a real enterprise technology.

First, the makeup of the Kubernetes community is clearly changing. Two years ago, almost every company at KubeCon+CloudNativeCon was some form of digital forward company like Lyft or cloud technology vendor such as Google or Red Hat. Now, there are many more traditional companies on both the IT and vendor side. Vendors such as HPE, Oracle, Intel, and Microsoft, mainstays of technology for the past 30 years, are here in force. Industries like telecommunications (drawn by the promise of edge computing), finance, manufacturing, and retail are much more visible than they were just a short time ago. While microservices and Kubernetes are not yet as widely deployed as more traditional n-Tier architectures and classic middleware, the mainstream is clearly interested.

Another indicator of the changes in the Kubernetes space is the prominence of security in the community. Not only are there more vendors than ever, but we are seeing more keynote time given to security practices. Security is, of course, a major component of making Kubernetes enterprise-ready. Without solid security practices and technology, Kubernetes will never be acceptable to a broad swatch of large to mid-sized businesses. That said, there is still so much more that needs to be done with Kubernetes security. The good news is that the community is working on it.

Finally, there is clearly more attention being paid to operating Kubernetes in a production environment. That’s most evident in the proliferation of tracing and logging technology, from both new and older companies, that were on display on the show floor and mainstage. Policy management was also an important area of discussion at the conference. These are all examples of the type of infrastructure that Operations teams will need to manage Kubernetes at scale and a sign that the community is thinking seriously about what happens after deployment.

It certainly helps that a lot of basic issues with Kubernetes have been solved but there is still more work to do. There are difficult challenges that need attention. How to migrate existing stateful apps originally written in Java and based on n-Tier architectures is still mostly an open question. Storage is another area that needs more innovation, though there’s serious work underway in that space. Despite the need for continued work, the progress seen at KubeCon+CloudNativeCon NA 2019 point to future where Kubernetes is a major platform for enterprise applications.  2020 will be another pivotal year for Kubernetes, containers, and microservices architectures. It may even be the year of mainstream adoption. We’ll be watching.

Canonical Takes a Third Path to Support New Platforms

We are in the midst of another change-up in the IT world. Every 15 to 20 years there is a radical rethink of the platforms that applications are built upon. During the course of the history of IT, we have moved from batch-oriented, pipelined systems (predominantly written in COBOL) to client-server and n-Tier systems that are the standards of today. These platforms were developed in the last century and designed for last century applications. After years of putting shims into systems to accommodate the scale and diversity of modern applications, IT has just begun to deploy new platforms based on containers and Kubernetes. These new platforms promise greater resiliency and scalability, as well as greater responsiveness to the business. Continue reading “Canonical Takes a Third Path to Support New Platforms”

The Amalgam Insiders have 5 Key Questions for VMworld

(Editor’s Note: This week, Tom Petrocelli and Hyoun Park will be blogging and tweeting on key topics at VMworld at a time when multi-cloud management is a key issue for IT departments and Dell is spending billions of dollars. Please follow our blog and our twitter accounts TomPetrocelli, Hyounpark, and AmalgamInsights for more details this week as we cover VMworld!)

As Amalgam Insights prepares to attend VMworld, it is an especially interesting time from both an M&A and a strategic perspective as VMware completes acquisitions of its sibling company Pivotal and end-user security startup Carbon Black. As these acquisitions are in progress and Amalgam Insights has the opportunity to question executives at the top of Dell Technologies, including Pat Gelsinger and Michael Dell, Amalgam Insights will be looking forward to answers to the following questions:

1. How will VMware accelerate Pivotal’s growth post-acquisition? Back in 2013 when Pivotal was first founded, I stated in an interview that

“Pivotal is the first application platform that combines cloud, Big Data, and rapid application development and it represents a fundamental shift in enterprise IT. By creating an enterprise-grade Big Data application platform, Pivotal has the opportunity to quickly unlock value from transactional data that has traditionally been archived and ignored without requiring a long period of up training, integration, and upfront development time.”

 

The potential for Pivotal was immense. Even in light of The Death of Big Data, Pivotal still has both the toolkits and methodology to support intelligent analytic and algorithm-based application architectures at a time when VMware needs to increase its support there in light of the capabilities of IBM-Red Hat, Oracle, and other competitors. We’re looking forward to getting some answers!

2. How will the Carbon Black acquisition be integrated into VMware’s security and end-user computing offerings? Carbon Black is a Boston-area security startup focused on discovering malicious activity on endpoints and will be a strong contributor to WorkspaceONE as VMware seeks to manage and secure the mobile-cloud ecosystem. And along with NSX Cloud for networking and CloudHealth Technologies for multi-cloud management, Carbon Black will help VMware to tell a stronger end-to-end cloud story. But the potential and timeline for integration will end up defining the success of this 2 billion+ dollar acquisition.

3. Where does CloudHealth Technologies fit into VMware’s multi-cloud management story? Although this 500 million dollar acquisition looked promising when it occurred last year, the Dell family previously invested in Enstratius to manage multi-cloud environments and that acquisition ended up going nowhere. What did VMware learn from the last time around and how will CloudHealth Technologies stay top of mind with all these other acquisitions going on?

4. Where is VMware going with its machine learning and AI capabilities for data center management? I can’t take credit for this one, as the great Maribel Lopez brought this up (go ahead and follow her on LinkedIn!). But VMware needs to continue advancing the Software-Defined Data Center and to ease client challenges in supporting hybrid cloud environments.

5. How is VMware bringing virtualization and Kubernetes together? With VMware’s acquisitions of Heptio and Bitnami, VMware has put itself right in the middle of the Kubernetes universe. But virtualization and Kubernetes are the application support equivalent of data center and cloud, two axes on the spectrum of what is possible. How will VMware simplify this componentization for clients who are seeking hybrid cloud help?

We’ll be looking for answers to these questions and more as we roam the halls of Moscone and put VMware and Dell executives to the test! Stay tuned for more!

VMware plus Pivotal Equals Platforms

(Editor’s Note: This week, Tom Petrocelli and Hyoun Park will be blogging and tweeting on key topics at VMworld at a time when multi-cloud management is a key issue for IT departments and Dell is spending billions of dollars. Please follow our blog and our twitter accounts TomPetrocelli, Hyounpark, and AmalgamInsights for more details this week as we cover VMworld!)

On August 22, 2019, VMware announced the acquisition of Pivotal. The term “acquisition” seems a little weird here since both are partly owned by Dell. It’s a bit like Dell buying Dell. Strangeness aside, this is a combination that makes a lot of sense.

For nearly eight years now, the concept of a microservices architecture has been taking shape. Microservices is an architectural idea wherein applications are broken up into many, small, bits of code – or services – that provide a limited set of functions and operate independently. Applications are assembled Lego-like, from component microservices. The advantages of microservices are that different parts of a system can evolve independently, updates are less disruptive, and systems become more resilient because system components are less likely to harm each other. The primary vehicle for microservices are containers (which I’ve covered in my Market Guide: Seven Decision Points When Considering Containers), that are deployed in clusters to enhance resiliency and more easily scale up resources.

The Kubernetes open-source software has emerged as the major orchestrator for containers and provides a stable base to build microservice platforms. These platforms must deploy not only the code that represents the business logic, but a set of system services, such as network, tracing, logging, and storage, as well. Container cluster platforms are, by nature, complex assortments of many moving parts – hard to build and hard to maintain.

The big problem has been that most container technology has been open-source and deployed piecemeal, leaving forward-looking companies to assemble their own container cluster microservices platforms. Building out and then maintaining these DIY platforms requires continued investment in people and other resources. Most companies either can’t afford or are unwilling to make investments in this amount of engineering talent and training. Subsequently, there are a lot of companies that have been left out of the container platform game.

The big change has been in the emergence of commercial platforms (many of which were discussed in my SmartList Market Guide on Service Mesh and Building Out Microservices Networking), based on open-source projects, that bring to IT everything it needs to deploy container-based microservices. All the cloud companies, especially Google, which was the original home of Kubernetes, and open-source software vendors such as Red Hat (recently acquired by IBM) with their OpenShift platform, have some form of Kubernetes-based platform. There may be as many as two dozen commercial platforms based on Kubernetes today.

This brings us to VMware and Pivotal. Both companies are in the platform business. VMware is still the dominant player in Virtual Machine (VM) hypervisors, which underpin most systems today, and are marketing a Kubernetes distribution. They also recently purchased Bitnami, a company that makes technology for bundling containers for deployment. At the time, I said:

“This is VMware doubling down on software for microservices and container clusters. Prima facie, it looks like a good move.”

Pivotal markets a Kubernetes distribution as well but also one of the major vendors for Cloud Foundry, another platform that runs containers, VMs, and now Kubernetes (which I discuss in my Analyst Insight: Cloud Foundry and Kubernetes: Different Paths to Microservices). The Pivotal portfolio also includes Spring Boot, one of the primary frameworks for building microservices in Java, and an extensive Continuous Integration/Continuous Deployment capability based on BOSH (part of Cloud Foundry), Concourse, and other open source tools.

Taken together, VMware and Pivotal offer a variety of platforms for newer microservices and legacy VM architectures that will fit the needs of a big swatch of large enterprises. This will give them both reach and depth in large enterprise companies and allow their sales teams to sell whichever platform a customer needs at the moment while providing a path to newer architectures. From a product portfolio perspective, VMware plus Pivotal is a massive platform play that will help them to compete more effectively against the likes of IBM/Red Hat or the big cloud vendors.

On their own, neither VMWare or Pivotal had the capacity to compete against Red Hat OpenShift, especially now that that Red Hat has access to IBM’s customer base and sales force. Together they will have a full range of technology to bring to bear as the Fortune 500 moves into microservices. The older architectures are also likely to remain in place either because of legacy reasons or because they just fit the applications they serve. VMware/Pivotal will be in a position to service those companies as well.

VMware could easily have decided to pick up any number of Kubernetes distribution companies such as Rancher or Platform9. None of them would have provided the wide range of platform choices that Pivotal brings to the table. And besides, this keeps it all in the Dell family.

Kubernetes Grows Up – The View from KubeCon EU 2019

Our little Kubernetes is growing up.

By “growing up” I mean it is almost in a state that a mainstream company can consider it fit for production. While there are several factors that act as a drag against mainstream reception, a lack of completeness has been a major force against Kubernetes broader acceptance. Completeness, in this context, means that all the parts of an enterprise platform are available off the shelf and won’t require a major engineering effort on the part of conventional IT departments.

The good news from KubeCon+CloudNativeCon EU 2019 in Barcelona, Spain (May 20 – 23 2019) is that the Kubernetes and related communities are zeroing in on that ever so important target. There are a number of markers pointing toward mainstream acceptance. Projects are filling out the infrastructure – gaining completeness – and the community is growing.

Project Updates

While Kubernetes may be at the core, there are many supporting projects that are striving to add capabilities to the ecosystem that will result in a more complete platform for microservices. Some of the projects featured in the project updates show the drive for completeness. For example, OpenEBS and Rook are two projects striving to make container storage more enterprise friendly. Updates to both projects were announced at the conference. Storage, like networking, is an area that must be tackled before mainstream IT can seriously consider container microservices platforms based on Kubernetes.

Managing microservices performance and failure is a big part of the ability to deploy containers at scale. For this reason, the announcement that two projects that provide application tracing capabilities, OpenTracing and OpenCensus, were merging into OpenTelemetry is especially important. Ultimately, developers need a unified approach to gathering data for managing container-based applications at scale. Removing duplication of effort and competing agendas will speed up the realization of that vision.

Also announced at KubeCon+CloudNativeCon EU 2019 were updates to Helm and Harbor, two projects that tackle thorny issues of packaging and distributing containers to Kubernetes. These are necessary parts of the process of deploying Kubernetes applications. Securely managing container lifecycles through packaging and repositories is a key component of DevOps support for new container architectures. Forward momentum in these projects is forward movement toward the mainstream.

There were other project updates, including updates to Kubernetes itself and Crio-io. Clearly, the community is filling in the blank spots in container architectures, making Kubernetes a more viable application platform for everyone.

The Community is Growing

Another gauge pointing toward mainstream acceptance is the growth in the community. The bigger the community, the more hands to do the work and the better the chances of achieving feature critical mass. This year in Barcelona, KubeCon+CloudNativeCon EU saw 7700 attendees, nearly twice last year in Copenhagen. In the core Kubernetes project, there are 164K commits and 1.2M comments in Github. This speaks to broad involvement in making Kubernetes better. Completeness requires lots of work and that is more achievable when there are more people involved.

Unfortunately, as Cheryl Hung, Director of Ecosystems at CNCF says, only 3% of contributors are women. The alarming lack of diversity in the IT industry shows up even in Kubernetes despite the high-profile women involved in the conference such as Janet Kuo of Google. Diversity brings more and different ideas to a project and it would be great to see the participation of women grow.

Service Mesh Was the Talk of the Town

The number of conversations I had about service mesh was astounding. It’s true that I had released a pair of papers on it, one just before KubeCon+CloudNativeCon EU 2019. That may have explained why people want to talk to me about it but not the general buzz. There was service mesh talk in the halls, at lunch, in sessions, and from the mainstage. It’s pretty much what everyone wanted to know about. That’s not surprising since a service mesh is going to be a vital part of large scale-out microservices applications. What was surprising was that even attendees who were new to Kubernetes were keen to know more. This was a very good omen.

It certainly helped that there was a big service mesh related announcement from the mainstage on Tuesday. Microsoft, in conjunction with a host of companies, announced the Service Mesh Interface. It’s a common API for different vendor and project service mesh components. Think of it as a lingua franca of service mesh. There were shout-outs to Linkerd and Solo.io. The latter especially had much to do with creating SMI. The fast maturation of the service mesh segment of the Kubernetes market is another stepping stone toward the completeness necessary for mainstream adoption.

Already Way Too Many Distros

There were a lot of Kubernetes distributions a KubeCon+CloudNativeCon EU 2019. A lot. Really.  A lot. While this is a testimony the growth in Kubernetes as a platform, it’s confusing to IT professionals making choices. Some are managed cloud services; others are distributions for on-premises or when you want to install your own on a cloud instance. Here’s some of the Kubernetes distros I saw on the expo floor.  I’m sure I missed a few:

Microsoft Azure Google Digital Ocean Alibaba
Canonical (Ubuntu) Oracle IBM Red Hat
VMWare SUSE Rancher Pivotal
Mirantis Platform9

 

From what I hear this is a sample, not a comprehensive, list. The dark side of this enormous choice is confusion. Choosing is hard when you get beyond a handful of options. Still, only five years into the evolution of Kubernetes, it’s a good sign to see this much commercial support for it.

The Kubernetes and Cloud Native architecture is like a teenager. It’s growing rapidly but not quite done. As the industry fills in the blanks and as communities better networking, storage, and deployment capabilities, it will go mainstream and become applicable to companies of all sizes and types. Soon. Not yet but very soon.

Docker Enterprise 3.0 is the Docker We’ve Been Waiting For

For the past few years, one of the big questions in the software industry has been what direction Docker would take. Much of their unique intellectual property, such as Docker images, had been open sourced and many of their products have underperformed. Docker Swarm is an excellent example of a product that was too little too late. While loved by Docker customers I spoke with, Docker Swarm simply couldn’t surf the swell that is the

Please register or log into your Amalgam Insights Community account to read more.
Log In Register

Google Goes Corporate at Google Next

There’s no doubt that Google exists to make money. They make money by getting companies to buy their services. When it comes to selling ads on search engines, Google is number one. When it comes to their cloud business, Google is… well, number three.

I’m guessing that irks them a bit especially since they sit behind a company whose main business is selling whatever stuff people want to sell and a company that made its name in the first wave of PCs. Basically, a department store and a dinosaur are beating them at what should be their game.

Please register or log into your Amalgam Insights Community account to read more.
Log In Register

Coming Attractions: Groundbreaking Service Mesh Research

In early January, I started researching the service mesh market. To oversimplify, a service mesh is a way of providing for the kind of network services necessary for enterprise applications deployed using a microservices architecture. Since most microservices architectures are being deployed within containers and, most often, managed and orchestrated using Kubernetes, service mesh technology will have a major impact on the adoption of these markets.

As I began writing the original paper, I quickly realized that an explanation of service mesh technology was necessary to understand the dynamic of the service mesh market. Creating a primer on service mesh and a market guide turned out to be too much for one paper. It was unbearably long. Subsequently, the paper was split into two papers, a Technical Guide and a Market Guide.

The Technical Guide is a quick primer on service mesh technology and how it is used to enhance microservices architectures, especially within the context of containers and Kubernetes. The Market Guide outlines the structure of the market for service mesh products and open source projects, discusses many of the major players, and talks to the current Istio versus Linkerd controversy. The latter is actually a non-issue that has taken on more importance than it should given the nascence of the market.

The Technical Guide will be released next week, just prior to Cloud Foundry Summit. Even though service mesh companies seem to be focused on Kubernetes, anytime there is a microservices architecture, there will be a service mesh. This is true for microservices implemented using Cloud Foundry containers.

The Market Guide will be published roughly a month later, before Red Hat Summit and KubeCon+CloudNative Summit Europe, which I will be attending. Most of the vendors discussed in the Market Guide will be in attendance at one or the other conference. Read the report before going so that you know who to talk to if you are attending these conferences.

A service mesh is a necessary part of emerging microservices architectures. These papers will hopefully get you started on your journey to deploying one.

Note: Vendors interested in leveraging this research for commercial usage are invited to contact Lisa Lincoln (lisa@amalgaminghts.com).

 

Tom Petrocelli Clarifies How Cloud Foundry and Kubernetes Provide Different Paths to Microservices

DevOps Research Fellow Tom Petrocelli has just published a new report describing the roles that Cloud Foundry Application Runtime and Kubernetes play in supporting microservices. This report explores when each solution is appropriate and provides a set of vendors that provide resources and solutions to support the development of these open source projects.

Organizations and Vendors mentioned include: Cloud Foundry Foundation, Cloud Native Computing Foundation, Pivotal, IBM, Suse, Atos, Red Hat, Canonical, Rancher, Mesosphere, Heptio, Google, Amazon, Oracle, and Microsoft

To download this report, which has been made available at no cost until the end of February, go to https://amalgaminsights.com/product/analyst-insight-cloud-foundry-and-kubernetes-different-paths-to-microservices

The View from KubeCon+CloudNativeCon – Containers and Kubernetes Become Enterprise Ready

In case there was any doubt about the direction containers and Kubernetes are going, KubeCon+CloudNativeCon 2018 in Seattle should have dispelled them. The path is clear – technology is maturing and keeps adding more features that make it conducive to mission-critical, enterprise applications. From the very first day, the talk was about service meshes and network functions, logging and traceability, and storage and serverless compute. These are couplets that define the next generation of management, visibility, and core capabilities of a modern distributed application. On top of that is emerging security projects such as SPIFFE & SPIRE, TUF, Falco, and Notary. Management, visibility, growth in core functionality, and security. All of these are critical to making container platforms enterprise ready.

“The future of containers and Kubernetes as the base of the new stack was on display at KubeCon+CloudNativeCon and it’s a bright one.

Tom Petrocelli, Research Fellow, Amalgam Insights”

If the scope of KubeCon+CloudNativeCon and the Cloud Native Computing Foundation (CNCF) is any indication, the ecosystem is also growing. This year there were 8000 people at the conference – a sellout. The CNCF has grown to 300+ vendor members there are 46,000 contributors to its projects. That’s a lot of growth compared to just a few years ago. This many people don’t flock to sinking projects.

Please register or log into your Amalgam Insights Community account to read more.
Log In Register