Posted on Leave a comment

How Red Hat Runs

This past week at Red Hat Summit 2019 (May 7 – 9 2019) has been exhausting. It’s not an overstatement to say that they run analysts ragged at their events, but that’s not why the conference made me tired. It was the sheer energy of the show, the kind of energy that keeps you running with no sleep for three days straight. That energy came from two sources – excitement and fear.

Two announcements, in particular, generated joy amongst the devoted Red Hat fans. The first was the announcement of Red Hat Enterprise Linux version 8, better known as RHEL8. RHEL is the granddaddy of all major Linux distributions for the data center. RHEL8, however, doesn’t seem all that old. As well as all the typical enhancements to the kernel and other parts of the distro, Red Hat has added two killer features to RHEL.

The first, the web console, is a real winner. It provides a secure browser-based system to manage all the features of Linux that one typically needs a command line on the server to perform. Now, using Telnet or SSH to log in to a remote box and do a few adjustments is no big deal when you have a small number of machines, physical or virtual, in a data center. When there are thousands of machines to care for, this is too cumbersome. With web console plus Red Hat Satellite, the same type of system maintenance is much more efficient. It even has a terminal built in if the command line is the only option. I predict that the web console will be an especially useful asset to new sysadmins who have yet to learn the intricacies of the Linux command line (or just don’t want to).

The new image builder is also going to be a big help for DevOps teams. Image builder uses a point and click interface to build images of software stacks, based on RHEL of course, that can be instantiated over and over. Creating consistent environments for developers and testing is a major pain for DevOps teams. The ability to quickly and easily create and deploy images will take away a major impediment to smooth DevOps pipelines.

The second announcement that gained a lot of attention was the impending GA of OpenShift 4 represents a major change in the Red Hat container platform. It incorporates all the container automation goodness that Red Hat acquired from CoreOS, especially the operator framework. Operators are key to automating container clusters, something that is desperately needed for large scale production clusters. While Kubernetes has added a lot of features to help with some automation tasks, such as autoscaling, that’s not nearly enough for managing clusters at hyperscale or across hybrid clouds. Operators are a step in that direction, especially as Red Hat makes it easier to use Operators.

Speaking of OpenShift, Satya Nadella, CEO of Microsoft appeared on the mainstage to help announce Azure Red Hat OpenShift. This would have been considered a mortal sin at pre-Nadella Microsoft and highlights the acceptance of Linux and open source at the Windows farm. Azure Red Hat OpenShift is an implementation of OpenShift as a native Azure service. This matters a lot to those serious about multi-cloud deployments. Software that is not a native service for a cloud service provider do not have the integrations for billing, management, and especially set up that native services do. That makes them second class citizens in the cloud ecosystem. Azure Red Hat OpenShift elevates the platform to first-class status in the Azure environment.

Now for the fear. Although Red Hat went to considerable lengths to address the “blue elephant in the room”, to the point of bringing Ginny Rometty, IBM CEO on stage, the unease around the acquisition by IBM was palpable amongst Red Hat customers. Many that I spoke to were clearly afraid that IBM would ruin Red Hat. Rometty, of course, insisted that was not the case, going so far as to say that she “didn’t spend $34B on Red Hat to destroy them.”

That was cold comfort to Red Hat partners and customers who have seen tech mergers start with the best intentions and end in disaster. Many attendees I spoke drew parallels with the Oracle acquisition of Sun. Sun was, in fact, the Red Hat of its time – innovative, nimble, and with fierce loyalists amongst the technical staff. While products created by Sun still exist today, especially Java and MySQL, the essence of Sun was ruined in the acquisition. That is a giant cloud hanging over the IBM-Red Hat deal. For all the advantages that this deal brings to both companies and the open source community, the potential for a train wreck exists and that is a source of angst in the Red Hat and open source world.

In 2019, Red Hat is looking good and may have a great future. Or it is on the brink of disaster. The path they will take now depends on IBM. If IBM leaves them alone, it may turn out to be an amazing deal and the capstone of Rometty and Jim Whitehurst’s careers. If IBM allows internal bureaucracy and politics to change the current plan for Red Hat, it will be Sun version 2. Otherwise, it is expected that Red Hat will continue to make open source enterprise-friendly and drive open source communities. That would be very nice indeed.

Posted on Leave a comment

Red Hat Hybrid Cloud Management Gets Financial with Cloud Cost Management

Key Stakeholders: CIO, CFO, Accounting Directors and Managers, Procurement Directors and Managers, Telecom Expense Personnel, IT Asset Management Personnel, Cloud Service Managers, Enterprise Architects

Why It Matters: As enterprise cloud infrastructure continues to grown 30-40% per year and containerization becomes a top enterprise concern, IT must have tools and a strategy for managing the cost of storage and compute associated with both hybrid cloud and container spend. With Cloud Cost Management, Red Hat provides an option for its considerable customer base.

Key Takeaways: Red Hat OpenShift customers seeking to managing the computing costs associated with hybrid cloud and containers should starting trialing Cloud Cost Management when it becomes available in 2019. Effective cost management strategies and tools should be considered table stakes for all enterprise-grade technologies.

Amalgam Insights is a top analyst firm in the analysis of IT subscription cost management, as can be seen in our:

In this context, Red Hat’s intended development of multi-cloud cost management integrated with CloudForms is an exciting announcement for the cloud market. This product, scheduled to come out in early 2019, will allow enterprises supporting multiple cloud vendors to support workload-specific cost management, which Amalgam Insights considers to be a significant advancement in the cloud cost management market.

And this product comes at a time when cloud infrastructure cost management has seen significant investment including VMware’s $500 million purchase of Boston-based CloudHealth Technologies, the 2017 $50 million “Series A” investment in CloudCheckr, investments in this area by leading Telecom and Technology Expense Management vendors such as Tangoe and Calero, and recent acquisitions and launches in this area from the likes of Apptio, BMC, Microsoft, HPE, and Nutanix.

However, the vast majority of these tools are currently lacking in the granular management of cloud workloads that can be tracked at a service level and then appropriately cross-charged to a project, department, or location. This capability will be increasingly important as application workloads become increasingly nuanced and revenue-driven accounting of IT becomes increasingly important. Amalgam Insights believes that, despite the significant activity in cloud cost management, that this market is just starting to reach a basic level of maturity as enterprises continue to increase their cloud infrastructure spend by 40% per year or more and start using multiple cloud vendors to deal with a variety of storage, computing, machine learning, application, service, integration, and hybrid infrastructure needs.

Red Hat Screenshot of Hybrid Cloud Cost Management

As can be seen from the screenshot, Red Hat’s intended Hybrid Cloud Cost Management offering reflects both modern design and support for both cloud spend and container spend. Given the enterprise demand for third-party and hybrid cloud cost management solutions, it makes sense to have an OpenShift-focused cost management solution.

Amalgam Insights has constantly promoted the importance of formalized technology cost management initiatives and their ability in reducing IT cost categories by 30% or more. We believe that Red Hat’s foray into Hybrid Cloud Cost Management has an opportunity to compete with a crowded field of competitors in managing multi-cloud and hybrid cloud spend. Despite the competitive landscape already in play, Red Hat’s focus on the OpenShift platform as a starting point for cost management will be valuable for understanding cloud spend at container, workload, and microservices levels that are currently poorly understood by IT executives.

My colleague Tom Petrocelli has noted that “I would expect to see more and more development shift to open source until it is the dominant way to develop large scale infrastructure software.” As this shift takes place, the need to manage the financial and operational accounting of these large-scale projects will become a significant IT challenge. Red Hat is demonstrating its awareness of this challenge and has created a solution that should be considered by enterprises that are embracing both Open Source and the cloud as the foundations for their future IT development.

Recommendations

Companies already using OpenShift should look forward to trialling Cloud Cost Management when it comes out in early 2019. This product provides an opportunity to effectively track the storage and compute costs of OpenShift workloads across all relevant infrastructure. As hybrid and multi-cloud management becomes increasingly common, IT organizations will need a centralized capability to track their increasingly complex usage associated with the OpenShift Container Platform.

Cloud Service Management and Technology Expense Management solutions focused on tracking Infrastructure as a Service spend should consider integration with Red Hat’s Cloud Cost Management solution. Rather than rebuild the wheel, these vendors can take advantage of the work already done by RedHat to track container spend.

And for Red Hat, Amalgam Insights provides the suggestion that Cloud Cost Management become more integrated with CloudForms over time. The most effective expense management practices for complex IT spend categories always include a combination of contracts, inventory, invoices, usage, service orders, service commitments, vendor comparisons, and technology category comparisons. To gain this holistic view that optmizes infrastructure expenses, cloud procurement and expense specialists will increasingly demand this complete view across the entire lifecycle of services.

Although this Cloud Cost Management capability has room to grow, Amalgam Insights expects this tool to quickly become a mainstay, either as a standalone tool or as integrated inputs within an enterprise’s technology expense or cloud service management solution. As with all things Red Hat, Amalgam Insights expects rapid initial adoption within the Red Hat community in 2019-2020 which will drive down enterprise infrastructure total cost of ownership and increase visibility for enterprise architects, financial controllers, and accounting managers responsible for responsible IT cost management.

Posted on Leave a comment

Observations on the Future of Red Hat from Red Hat Analyst Day

On November 8th, 2018, Amalgam Insights analysts Tom Petrocelli and Hyoun Park attended the Red Hat Analyst Day in Boston, MA. We had the opportunity to visit Red Hat’s Boston office in the rapidly-growing Innovation District, which has become a key tech center for enterprise technology companies. In attending this event, my goal was to learn more about the Red Hat culture that is being acquired as well as to see how Red Hat was taking on the challenges of multi-cloud management.

Throughout Red Hat’s presentations throughout the day, there was a constant theme of effective cross-selling, growing deal sizes including a record 73 deals of over $1 million in the last quarter, over 600 accounts with over $1 million in business in the last year, and increased wallet share year-over-year for top clients with 24 out of 25 of the largest clients increasing spend by an average of 15%. The current health of Red Hat is undeniable, regardless of the foibles of the public market. And the consistency of Red Hat’s focus on Open Source was undeniable across infrastructure, integration, application development, IT automation, IT optimization, and partner solutions, which demonstrated how synchronized and focused the entire Red Hat executive team presenters were, including Continue reading Observations on the Future of Red Hat from Red Hat Analyst Day

Posted on

Is IBM’s Acquisition of Red Hat the Biggest Acquihire of All Time?

Estimated Reading Time: 11 minutes

Internally, Amalgam Insights has been discussing why IBM chose to acquire Red Hat for $34 billion dollars fairly intensely. Our key questions included:

  • Why would IBM purchase Red Hat when they’re already partners?
  • Why purchase Red Hat when the code is Open Source?
  • Why did IBM offer a whopping $34 billion, $20 billion more than IBM currently has on hand?

As a starting point, we posit that IBM’s biggest challenge is not an inability to understand its business challenges, but a fundamental consulting mindset that starts with the top on down. By this, we mean that IBM is great at identifying and finding solutions on a project-specific basis. For instance, SoftLayer, Weather Company, Bluewolf, and Promontory Financial are all relatively recent acquisitions that made sense and were mostly applauded at the time. But even as IBM makes smart investments, IBM has either forgotten or not learned the modern rules for how to launch, develop, and maintain software businesses. At a time when software is eating everything, this is a fundamental problem that IBM needs to solve.

The real question for IBM is whether IBM can manage itself as a modern software company.

Continue reading Is IBM’s Acquisition of Red Hat the Biggest Acquihire of All Time?

Posted on 1 Comment

Tom Petrocelli Provides Context for IBM’s Acquisition of Red Hat

Tom Petrocelli, Amalgam Insights Research Fellow

In light of yesterday’s announcement that IBM is planning to acquire Red Hat for $34 billion, we’d like to share with you some of our recent coverage and mentions of Red Hat to provide context for this gargantuan acquisition.

To learn more about the state of Enterprise Free Open Source Software and the state of DevOps, make sure you continue to follow Tom Petrocelli on this website and his Twitter account.

Posted on 1 Comment

Market Milestone: Red Hat Acquires CoreOS Changing the Container Landscape

Red Hat Acquires CoreOS

We have just published a new document from Tom Petrocelli analyzing Red Hat’s $250 million acquisition of CoreOS and why it matters for DevOps and Systems Architecture managers.

This report is recommended for CIOs, System Architects, IT Managers, System Administrators, and Operations Managers who are evaluating CoreOS and Red Hat as container solutions to support their private and hybrid cloud solutions. In this document, Tom provides both the upside and concerns that your organization needs to consider in evaluating CoreOS.

This document includes:
A summary of Red Hat’s Acquisition of CoreOS
Why It Matters
Top Takeaways
Contextualizing CoreOS within Red Hat’s private and hybrid cloud portfolio
Alternatives to Red Hat CoreOS
Positive and negative aspects fcr current Red Hat and CoreOS customers

To download this report, please go to our Research section.

Posted on

February 25: From BI to AI (Aporia, cnvrg.io, Decodable, Equalum, Grata, Hasura, Mage, nRoad, Redpanda, SeMI Technologies, thatDot)

If you would like your announcement to be included in Amalgam Insights’ weekly data and analytics roundups, please email lynne@amalgaminsights.com.

Funding

Aporia Raises $25 Million to Grow its Machine Learning Observability Platform

On February 22, Aporia, a machine learning observability platform, announced that it had raised $25M in a Series A funding round. Tiger Global Management led the round, with participation from existing investors TLV Partners and Vertex Ventures, and new investors Samsung NEXT and Tal Ventures. The funding will go towards hiring and global expansion.

Decodable Raises $20M Series A Funding Round For Its Realtime Data Platform

Decodable, a realtime data engineering platform, raised a $20M A round this week. Bain Capital Ventures and Venrock led the funding round, with additional participation from individual investors including former US Chief Data Scientist DJ Patil, DataDog CEO Olivier Pomel, Cockroach Labs CEO Spencer Kimball, and Redis CRO and President Jason Forget. Decodable also debuted the Decodable Real-Time Data Platform, which supports functions like event-driven micro services, data mesh deployment, realtime data integration and ML/AI pipelines, and data governance and regulatory compliance.

Grata Closes $25 Million A Round For Its Data Intelligence Engine

Grata, a data intelligence engine, announced February 22 that it had raised $25M in a Series A funding round led by Craft Ventures. Existing investors Accomplice, Bling, and Touchdown Ventures also participated, along with new investors Altai Ventures, Eigen Ventures, and Teamworthy Ventures. The funding will go towards further product development. Grata uses proprietary machine learning and natural language processing models to process unstructured data from websites into insights on private companies, made available in a search-based interface.

GraphQL Engine Provider Hasura Announces $100M in Series C Funding

Hasura, a GraphQL engine provider, has raised a $100M Series C funding round. Greenoaks led the round, with participation from existing investors Lightspeed Venture Partners, Nexus Venture Partners, and Vertex Ventures. Hasura will use the funding for R+D and global expansion of their go-to-market strategy.

Streaming Data Platform Redpanda Raises $50M Series B

Redpanda, a data streaming platform, announced February 23 that they had raised a $50M Series B Funding Round led by GV. Haystack VC also participated, as did Lightspeed Venture Partners (busy week for Lightspeed, also participating in the Hasura C round!). The funding will go towards hiring for their engineering and go-to-market teams.

SeMI Technologies Raises $16M Series A Round For AI-Based Search Database

SeMI Technologies, providers of open source vector search engine Weaviate, announced a $16M Series A funding round February 22. Cortical Ventures and New Enterprise Associates co-led the round. The funding will go towards hiring, community development, and product improvement including increasing potential use cases and creating and improving the ML models Weaviate is based on.

Launches and Updates

cnvrg.io Announces AI Blueprints, Customizable ML Pipelines

On February 22, Cnvrg.io, an AI/ML platform provider, debuted cnvrg.io AI Blueprints. AI Blueprints is a curated open-source library of machine learning model APIs and customizable pipelines, allowing companies to quickly piece together models to analyze their data. Availability of cnvrg.io AI Blueprints is planned for the first half of 2022.

Equalum Releases v3.0 of their Continuous Data Integration Platform 3.0

Equalum released version 3.0 of their “continuous” data integration platform this week. New features include expanded support for cloud targets across AWS, Azure, and GCP; enhanced binary parsers for Oracle logs and SQL replication; improvements to replication groups to allow for extensive data migrations and cross-platform data warehousing; and no-code data integration capabilities for streaming ETL and ELT data, as well as batch ETL and change data capture.

Mage Debuts Low Code AI Ranking Model Tool for Product Developers

On February 24, Mage announced the general availability of its low code AI tool. Mage is targeted towards product developers needing to build AI ranking models to increase user engagement and retention.

nRoad Launches Unstructured Data Processing Platform Convus

nRoad, an NLP startup, introduced its Convus platform February 23. Convus provides machine learning models for financial services to extract insights from unstructured data. This allows FinTech businesses to avoid manual data extraction and entry while incorporating information in documents into business processes.

thatDot Releases Complex Event Processing Engine Quine Streaming Graph

thatDot, complex event processing software providers, debuted Quine Streaming Graph, an open source event processing engine based on streaming graph data. Developers can use Quine to quickly build complex event processing workflows to apply to streaming graph data using “recipes.” Recipes currently available include blockchain realtime tag propagation, CDN cache efficiency analysis, and Apache server log observability.

Posted on 4 Comments

5 Predictions That Will Transform Corporate Training in 2018

At Amalgam Insights, we have been focused on the key 2018 trends that will change our ability to manage technology at scale. Tom Petrocelli provided his key Developer Operations and enterprise collaboration predictions for 2018 in mid-December. To continue that trend, Todd Maddox provides 5 key predictions that will shape enterprise learning in 2018 as markets reach new heights, corporate training embraces new scientific principles, and retention replaces compliance as a key training driver.

  1. VR/AR Enterprise Application Budget to Surpass $1 Billion in 2018
  2. eLearning (Computer-Based Training) Market to Approach $180 billion in 2018
  3. Commercial Training Sector to Embrace Neuroscience of Optimized Learning
  4. Continued Exponential Growth of Artificial Intelligence (AI) as a Driving Force Behind the User Interface (UI)
  5. Training for Retention: The Rule, Not the Exception in 2018

Continue reading 5 Predictions That Will Transform Corporate Training in 2018

Posted on

What Happened In Tech? – AI has its Kardashians Moment with OpenAI’s Chaotic Weekend

The past week has been “Must See TV” in the tech world as AI darling OpenAI provided a season of Reality TV to rival anything created by Survivor, Big Brother, or the Kardashians. Although I often joke that my professional career has been defined by the well-known documentaries of “The West Wing,” “Pitch Perfect,” and “Sillcon Valley,” I’ve never been a big fan of the reality TV genre as the twist and turns felt too contrived and over the top… until now.

Starting on Friday, November 17th, when The Real Housewives of OpenAI started its massive internal feud, every organization working on an AI project has been watching to see what would become of the overnight sensation that turned AI into a household concept with the massively viral ChatGPT and related models and tools.

So, what the hell happened? And, more importantly, what does it mean for the organizations and enterprises seeking to enter the Era of AI and the combination of generative, conversational, language-driven, and graphic capabilities that are supported with the multi-billion parameter models that have opened up a wide variety of business processes to natural language driven interrogation, prioritization, and contextualization?

The Most Consequential Shake Up In Technology Since Steve Jobs Left Apple

The crux of the problem: OpenAI, the company we all know as the creator of ChatGPT and the technology provider for Microsoft’s Copilots, was fully controlled by another entity, OpenAI, the nonprofit. This nonprofit was driven by a mission of creating general artificial intelligence for all of humanity. The charter starts with“OpenAI’s mission is to ensure that artificial general intelligence (AGI) – by which we mean highly autonomous systems that outperform humans at most economically valuable work – benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.”

There is nothing in there about making money. Or building a multi-billion dollar company. Or providing resources to Big Tech. Or providing stakeholders with profit other than highly functional technology systems. In fact, further in the charter, it even states that if a competitor shows up with a project that is doing better at AGI, OpenAI commits to “stop competing with and start assisting this project.”

So, that was the primary focus of OpenAI. If anything, OpenAI was built to prevent large technology companies from being the primary force and owner of AI. In that context, four of the six board members of OpenAI decided that open AI‘s efforts to commercialize technology were in conflict with this mission, especially with the speed of going to market, and the shortcuts being made from a governance and research perspective.

As a result, they ended up firing both the CEO, Sam, Altman and removed President COO Greg Brockman, who had been responsible for architecting that resources and infrastructure associated with OpenAI, from the board. That action begat this rapid mess and chaos for this 700+ employee organization which was allegedly about to see an 80 billion dollar valuation

A Convoluted Timeline For The Real Housewives Of Silicon Valley

Friday: OpenAI’s board fires its CEO and kicks its president Greg Brockman off the board. CTO Mira Murati, who was called the night before, was appointed temporary CEO. Brockman steps down later that day.

Saturday: Employees are up in arms and several key employees leave the company, leading to immediate action by Microsoft going all the way up to CEO Satya Nadella to basically ask “what is going on? And what are you doing with our $10 billion commitment, you clowns?!” (Nadella probably did not use the word clowns, as he’s very respectful.)

Sunday: Altman comes in the office to negotiate with Microsoft and OpenAI’s investors. Meanwhile, OpenAI announces a new CEO, Emmett Shear, who was previously the CEO of video game streaming company Twitch. Immediately, everyone questions what he’ll actually be managing as employees threaten to quit, refuse to show up to an all-hands meeting, and show Altman overwhelming support on social media. A tumultuous Sunday ends with an announcement by Microsoft that Altman and Brockman will lead Microsoft’s AI group.

Monday: A letter shows up asking the current board to resign with over 700 employees threatening to quit and move to the Microsoft subsidiary run by Altman and Brockman. Co-signers include board member and OpenAI Ilya Sutskever, who was one of the four board votes to oust Altman in the first place.

Tuesday: The new CEO of OpenAI, Emmett Shear, states that he will quit if the OpenAI board can’t provide evidence of why they fired Sam Altman. Late that night, Sam Altman officially comes back to OpenAI as CEO with a new board consisting initially of Bret Taylor, former co-CEO of Salesforce, Larry Summers (former Secretary of the Treasury), and Adam d’Angelo, one of the former board members who voted to figure Sam Altman. Helen Toner of Georgetown and Tasha McCauley, both seen as ethical altruists who were firmly aligned with OpenAI’s original mission, both step down from the board.

Wednesday: Well, that’s today as I’m writing this out. Right now, there are still a lot of questions about the board, the current purpose of OpenAI, and the winners and losers.

Keep In Mind As We Consider This Wild And Crazy Ride

OpenAI was not designed to make money. Firing Altman may have been defensible from OpenAI’s charter perspective to build safe General AI for everyone and to avoid large tech oligopolies. But if that’s the case, OpenAI should not have taken Microsoft’s money. OpenAI wanted to have its cake and eat it as well with a board unused to managing donations and budgets at that scale.

Was firing Altman even the right move? One could argue that productization puts AI into more hands and helps prepare society for an AGI world. To manage and work with superintelligences, one must first integrate AI into one’s life and the work Altman was doing was putting AI into more people’s hands in preparation for the next stage of global access and interaction with superintelligence.

At the same time, the vast majority of current OpenAI employees are on the for-profit side and signed up, at least in part, because of the promise of a stock-based payout. I’m not saying that OpenAI employees don’t also care about ethical AI usage, but even the secondary market for OpenAI at a multi-billion dollar valuation would help pay for a lot of mortgages and college bills. But tanking the vast majority of employee financial expectations is always going to be a hard sell, especially if they have been sold on a profitable financial outcome.

OpenAI is expensive to run: probably well over 2 billion dollars per year, including the massive cloud bill. Any attempt to slow down AI development or reduce access to current AI tools needs to be tempered by the financial realities of covering costs. It is amazing to think that OpenAI’s board was so naïve that they could just get rid of the guy who was, in essence, their top fundraiser or revenue officer without worrying about how to cover that gap.

Primary research versus go-to-market activities are very different. Normally there is a church-and-state type of wall between these two areas exactly because they are to some extent at odds with each other. The work needed to make new, better, safer, and fundamentally different technology is often conflicted with the activity used to sell existing technology. And this is a division that has been well established for decades in academia where patented or protected technologies are monetized by a separate for-profit organization.

The Effective Altruism movement: this is an important catchphrase in the world of AI, as it is not just defined as a dictionary definition. This is a catchphrase for a specific view of developing artificial general intelligence (superintelligences beyond human capacity) with the goal of supporting a population of 10^58 millennia from now. This is one extreme of the AI world, which is countered by a “doomer” mindset thinking that AI will be the end of humanity.

Practically, most of us are in between with the understanding that we have been using superhuman forces in business since the Industrial Revolution. We have been using Google, Facebook, data warehouses, data lakes, and various statistical and machine learning models for a couple of decades that vastly exceed human data and analytic capabilities.

And the big drama question for me: What is Adam d’Angelo still doing on the board as someone who actively caused this disaster to happen? There is no way to get around the fact that this entire mess was due to a board-driven coup and he was part of the coup. It would be surprising to see him stick around for more than a few months especially now that Bret Taylor is on board, who provides an overlap of experiences and capabilities that d’Angelo possesses, but at greater scale.

The 13 Big Lessons We All Learned about AI, The Universe, and Everything

First, OpenAI needs better governance in several areas: board, technology, and productization.

  1. Once OpenAI started building technologies with commercial repercussions, the delineation between the non-profit work and the technology commercialization needed to become much clearer. This line should have been crystal clear before OpenAI took a $10 billion commitment from Microsoft and should have been advised by a board of directors that had any semblance of experience in managing conflicts of interest at this level of revenue and valuation. In particular, Adam d’Angelo as the CEO of a multi-billion dollar valued company and Helen Toner of Georgetown should have helped to draw these lines and make them extremely clear for Sam Altman prior to this moment.
  2. Investors and key stakeholders should never be completely surprised by a board announcement. The board should only take actions that have previously been communicated to all major stakeholders. Risks need to be defined beforehand when they are predictable. This conflict was predictable and, by all accounts, had been brewing for months. If you’re going to fire a CEO, make sure your stakeholders support you and that you can defend your stance.
  3. You come at the king, you best not miss.” As Omar said in the famed show “The Wire,” you cannot try to take out the head of an organization unless your followup plan is tight.
  4. OpenAI’s copyright challenges feel similar to when Napster first became popular as a streaming platform for music. We had to collectively figure out how to avoid digital piracy while maintaining the convenience that Napster provided for supporting music and sharing other files. Although the productivity benefits make generative AI worth experimenting with, always make sure that you have a back up process or capability for anything supported with generative AI.

    OpenAI and other generative AI firms have also run into challenges regarding the potential copyright issues associated with their models. Although a number of companies are indemnifying clients from damages associated with any outputs associated with their models, companies will likely still have to stop using any models or outputs that end up being associated with copyrighted material.

    From Amalgam Insights’ perspective, the challenge with some foundational models is that training data is used to build the parameters or modifiers associated with a model. This means that the copyrighted material is being used to help shape a product or service that is being offered on a commercial basis. Although there is no legal precedent either for or against this interpretation, the initial appearance of this language fits with the common sense definitions of enforcing copyright on a commercial basis. This is why the data collating approach that IBM has taken to generative AI is an important differentiator that may end up being meaningful.
  5. Don’t take money if you’re not willing to accept the consequences. This is a common non-profit mistake to accept funding and simply hope it won’t affect the research. But the moment research is primarily dependent on one single funder, there will always be compromises. Make sure those compromises are expressly delineated in advance and if the research is worth doing under those circumstances.
  6. Licensing nonprofit technologies and resources should not paralyze the core non-profit mission. Universities do this all the time! Somebody at OpenAI, both in the board and at the operational level, should be a genius at managing tech transfer and commercial utilization to help avoid conflicts between the two institutions. There is no reason that the OpenAI nonprofit should be hamstrung by the commercialization of its technology because there should be a structure in place to prevent or minimize conflicts of interest other than firing the CEO.

    Second, there are also some important business lessons here.
  7. Startups are inherently unstable. Although OpenAI is an extreme example, there are many other more prosaic examples of owners or boards who are unpredictable, uncontrollable, volatile, vindictive, or otherwise unmanageable in ways that force businesses to close up shop or to struggle operationally. This is part of the reason that half of new businesses fail within five years.
  8. Loyalty matters, even in the world of tech. It is remarkable that Sam Altman was backed by over 90% of his team on a letter saying that they would follow him to Microsoft. This includes employees who were on visas and were not independently rich, but still believed in Sam Altman more than the organization that actually signed their paychecks. Although it never hurts to also have Microsoft’s Kevin Scott and Satya Nadella in your corner and to be able to match compensation packages, this also speaks to the executive responsibility to build trust by creating a better scenario for your employees than others can provide. In this Game of Thrones, Sam Altman took down every contender to the throne in a matter of hours.
  9. Microsoft has most likely pulled off a transaction that ends up being all but an acquisition of OpenAI. It looks like Microsoft will end up with the vast majority of OpenAI’s‘s talent as well as an unlimited license to all technology developed by OpenAI. Considering that OpenAI was about to support a stock offering with an $80 billion market cap, that’s quite the bargain for Microsoft. In particular, Bret Taylor’s ascension to the board is telling as his work at Twitter was in the best interests of the shareholders of Twitter in accepting and forcing an acquisition that was well in excess of the publicly-held value of the company. Similarly, Larry Summers, as the former president of Harvard University, is experienced in balancing non-profit concerns with the extremely lucrative business of Harvard’s endowment and intellectual property. As this board is expanded to as many as nine members, expect more of a focus on OpenAI as a for-profit entity.
  10. With Microsoft bringing OpenAI closer to the fold, other big tech companies that have made recent investments in generative AI now have to bring those partners closer to the core business. Salesforce, NVIDIA, Alphabet, Amazon, Databricks, SAP, and ServiceNow have all made big investments in generative AI and need to lock down their access to generative AI models, processors, and relevant data. Everyone is betting on their AI strategy to be a growth engine over the next five years and none can afford a significant misstep.
  11. Satya Nadella’s handling of the situation shows why he is one of the greatest CEOs in business history. This weekend could have easily been an immense failure and a stock price toppling event for Microsoft. But in a clutch situation, Satya Nadella personally came in with his executive team to negotiate a landing for openAI, and to provide a scenario that would be palatable both to the market and for clients. The greatest CEOs have both the strategic skills to prepare for the future and the tactical skills to deal with immediate crisis. Nadella passes with flying colors on all accounts and proves once again that behind the velvet glove of Nadella’s humility and political savvy is an iron fist of geopolitical and financial power that is deftly wielded.
  12. Carefully analyze AI firms that may have similar charters for supporting safe AI, and potentially slowing down or stopping product development for the sake of a higher purpose. OpenAI ran into challenges in trying to interpret its charter, but the charter’s language is pretty straightforward for anyone who did their due diligence and took the language seriously. Assume that people mean what they say. Also, consider that there are other AI firms that have similar philosophies to OpenAI, such as Anthropic, which spun off of OpenAI for reasons similar to the OpenAI board reasoning of firing Sam Altman. Although it is unlikely that Anthropic (or large firms with safety-first philosophies like Alphabet and Meta’s AI teams) will fall apart similarly, the charters and missions of each organization should be taken into account in considering their potential productization of AI technologies.
  13. AI is still an emerging technology. Diversify, diversify, diversify. It is important to diversify your portfolio and make sure that you were able to duplicate experiments on multiple foundation models when possible. The marginal cost of supporting duplicate projects pales in comparison to the need to support continuity and gain greater understanding of the breath of AI output possibilities. With the variety of large language models, software vendor products, and machine learning platforms on the market, this is a good time to experiment with multiple vendors while designing process automation and language analysis use cases.
Posted on

8 Keys to Managing the Linguistic Copycats that are Large Language Models

Over the past year, Generative AI has taken the world by storm as a variety of large language models (LLMs) appeared to solve a wide variety of challenges based on basic language prompts and questions.

A partial list of market-leading LLMs currently available include:

Amazon Titan
Anthropic Claude
Cohere
Databricks Dolly
Google Bard, based on PaLM2
IBM Watsonx
Meta Llama
OpenAI’s GPT

The biggest question regarding all of these models is simple: how to get the most value out of them. And most users fail because they are unused to the most basic concept of a large language model: they are designed to be linguistic copycats.

As Andrej Karpathy of OpenAI stated earlier this year,

"The hottest new programming language is English."

And we all laughed at the concept for being clever as we started using tools like ChatGPT, but most of us did not take this seriously. If English really is being used as a programming language, what does this mean for the prompts that we use to request content and formatting?

I think we haven’t fully thought out what it means for English to be a programming language either in terms of how to “prompt” or ask the model how to do things correctly or how to think about the assumptions that an LLM has as a massive block of text that is otherwise disconnected from the real world and lacks the sensory input or broad-based access to new data that can allow it to “know” current language trends.

Here are 8 core language-based concepts to keep in mind when using LLMs or considering the use of LLMs to support business processes, automation, and relevant insights.

1) Language and linguistics tools are the relationships that define the quality of output: grammar, semantics, semiotics, taxonomies, and rhetorical flourishes. There is a big difference between asking for “write 200 words on Shakespeare” vs. “elucidate 200 words on the value of Shakespeare as a playwright, as a poet, and as a philosopher based on the perspective on Edmund Malone and the English traditions associated with blank verse and iambic pentameter as a preamble to introducing the Shakespeare Theatre Association.”

I have been a critic of the quality that LLMs provide from an output perspective, most recently in my perspective “Instant Mediocrity: A Business Guide to ChatGPT in the Enterprise.” https://amalgaminsights.com/2023/06/06/instant-mediocrity-a-business-guide-to-chatgpt-in-the-enterprise/. But I readily acknowledge that the outputs one can get from LLMs will improve. Expert context will provide better results than prompts that lack subject matter knowledge

2) Linguistic copycats are limited by the rules of language that are defined within their model. Asking linguistic copycats to provide language formats or usage that are not commonly used online or in formal writing will be a challenge. Poetic structures or textual formats referenced must reside within the knowledge of the texts that the model has seen. However, since Wikipedia is a source for most of these LLMs, a contextual foundation exists to reference many frequently used frameworks.

3) Linguistic copycats are limited by the frequency of vocabulary usage that they are trained on. It is challenging to get an LLM to use expert-level vocabulary or jargon to answer prompts because the LLM will typically settle for the most commonly used language associated with a topic rather than elevated or specific terms.

This propensity to choose the most common language associated with a topic makes it difficult for LLM-based content to sound unique or have specific rhetorical flourishes without significant work from the prompt writer.

4) Take a deep breath and work on this. Linguistic copycats respond to the scope, tone, and role mentioned in a prompt. A recent study found that, across a variety of LLM’s, the prompt that provided the best answer for solving a math problem and providing instructions was not a straightforward request such as “Let’s think step by step,” but “Take a deep breath and work on this problem step-by-step.”

Using a language-based perspective, this makes sense. The explanations of mathematical problems that include some language about relaxing or not stressing would likely be designed to be more thorough and make sure the reader was not being left behind at any step. The language used in a prompt should represent the type of response that the user is seeking.

5) Linguistic copycats only respond to the prompt and the associated prompt engineering, custom instructions, and retrieval data that they can access. It is easy to get carried away with the rapid creation of text that LLM’s provide and mistake this for something resembling consciousness, but the response being created is a combination of grammatical logic and the computational ability to take billions of parameters into account across possibly a million or more different documents. This ability to access relationships across 500 or more gigabytes of information is where LLMs do truly have an advantage over human beings.

6) Linguistic robots can only respond based on their underlying attention mechanisms that define their autocompletion and content creation responses. In other words, linguistic robots make judgment calls on which words are more important to focus on in a sentence or question and use that as the base of the reply.

For instance, in the sentence “The cat, who happens to be blue, sits in my shoe,” linguistic robots will focus on the subject “cat” as the most important part of this sentence. The cat “happens to be,” implies that this isn’t the most important trait. The cat is blue. The cat sits. The cat is in my shoe. The words include an internal rhyme and are fairly nonsensical. And then the next stage of this process is to autocomplete a response based on the context provided in the prompt.

7) Linguistic robots are limited by a token limit for inputs and outputs. Typically, a token is about four characters while the average English content word is about 6.5 characters (https://core.ac.uk/download/pdf/82753461.pdf). So, when an LLM talks about supporting 2048 tokens, that can be seen as about 1260 words, or about four pages of text, for concepts that require a lot of content. In general, think of a page of content as being about 500 tokens and a minute of discussion typically being around 200 tokens when one is trying to judge how much content is either being created or entered into an LLM.

8) Every language is dynamic and evolves over time. LLMs that provide good results today may provide significantly better or worse results tomorrow simply because language usage has changed or because there are significant changes in the sentiment of a word. For instance, the English language word “trump” in 2015 has a variety of political relationships and emotional associations that are now standard to language usage in 2023. Be aware of these changes across languages and time periods in making requests, as seemingly innocuous and commonly used words can quickly gain new meanings that may not be obvious, especially to non-native speakers.

Conclusion

The most important takeaway of the now-famous Karpathy quote is to take it seriously not only in terms of using English as a programming language to access structures and conceptual frameworks, but also to understand that there are many varied nuances built into the usage of the English language. LLM’s often incorporate these nuances even if those nuances haven’t been directly built into models, simply based on the repetition of linguistic, rhetorical, and symbolic language usage associated with specific topics.

From a practical perspective, this means that the more context and expertise provided in asking an LLM for information and expected outputs, the better the answer that will typically be provided. As one writes prompts for LLMs and seek the best possible response, Amalgam Insights recommends providing the following details in any prompt:

Tone, role, and format: This should include a sentence that shows, by example, the type of tone you want. It should explain who you are or who you are writing for. And it should provide a form or structure for the output (essay, poem, set of instructions, etc…). For example, “OK, let’s go slow and figure this out. I’m a data analyst with a lot of experience in SQL, but very little understanding of Python. Walk me through this so that I can explain this to a third grader.”

Topic, output, and length: Most prompts start with the topic or only include the topic. But it is important to also include perspective on the size of the output. Example, “I would like a step by step description of how to extract specific sections from a text file into a separate file. Each instruction should be relatively short and comprehensible to someone without formal coding experience.”

Frameworks and concepts to incorporate: This can include any commonly known process or structure that is documented, such as an Eisenhower Diagram, Porter’s Five Forces, or the Overton Window. As a simpe example, one could ask, “In describing each step, compare each step to the creation of a pizza, wherever possible.”

Combining these three sections together into a prompt should provide a response that is encouraging, relatively easy to understand, and compares the code to creating a pizza.

In adapting business processes based on LLMs to make information more readily available for employees and other stakeholders, be aware of these biases, foibles, and characteristics associated with prompts as your company explores this novel user interface and user experience.