{"id":14346,"date":"2026-08-02T13:10:46","date_gmt":"2026-08-02T13:10:46","guid":{"rendered":"https:\/\/sawahsolutions.com\/range\/attackers-are-targeting-open-source-ai-just-as-big-tech-is-embracing-it\/"},"modified":"2026-08-02T13:10:46","modified_gmt":"2026-08-02T13:10:46","slug":"attackers-are-targeting-open-source-ai-just-as-big-tech-is-embracing-it","status":"publish","type":"post","link":"https:\/\/sawahsolutions.com\/range\/attackers-are-targeting-open-source-ai-just-as-big-tech-is-embracing-it\/","title":{"rendered":"Attackers are targeting open-source AI just as Big Tech is embracing it"},"content":{"rendered":"
\n

The future of AI will be driven in significant part by startups and small players using open-source software to make their own, cheaper, more efficient versions of Claude or ChatGPT, tech leaders finally admitted this week. For companies like AWS, the challenge now is protecting those public tools from increasingly sophisticated attacks from China, Russia, North Korea, or other players.<\/p>\n

\u201cOpen-weight models\u2014AI models that anyone can download, inspect, modify, and run on their own infrastructure\u2014are an important part of that foundation because they make advanced AI more accessible, adaptable, and widely available,\u201d reads a July 24 statement by Nvidia CEO Jensen Huang, and co-signed by Amazon, Meta, Google, Microsoft, and a host of other companies.\u00a0<\/strong><\/p>\n

How open won<\/strong><\/p>\n

The statement shows a dramatic reversal by companies that made fortunes off of proprietary software and have invested billions in OpenAI, Anthropic, and other frontier AI labs whose primary business is closed AI models. Dramatic, yes. But years in the making.<\/p>\n

Satya Nadella illustrates how quickly sentiment among leading tech firms has changed. In 2024, the Microsoft CEO described proprietary models as critical to safety. Closed-source AI, he said, \u201callow[s] us to do deep end-to-end red-teaming, alignment, and safety evaluations before exposing them to the world.\u201d Microsoft had invested $13 billion in OpenAI by that point.<\/p>\n

AWS, another signatory of Huang\u2019s statement, this week completed a $50 billion investment in OpenAI; the Amazon spinoff has also put money into rival lab Anthropic. Both of those investments were bets that the intellectual property of a small number of labs was going to be better and easier to safeguard than code made, often, by volunteers.<\/p>\n

What has changed since 2024? Several things: A growing body of research showed that relatively cheap open-weight models were steadily catching up to the performance of closed ones. The American public is increasingly pessimistic about AI. The Pentagon wants AI that it can control and can operate without large, targetable data centers.\u00a0<\/p>\n

But for tech giants like Microsoft and AWS, the biggest change since 2024 is their evolution from provider of AI to provider of tools, space, and security for open-source and open-weight AI developers.<\/p>\n

During AWS\u2019 earnings call on Thursday, CEO Andy Jassy boasted of more than 10 models in Amazon\u2019s Bedrock platform, which allows users to build their own generative models.\u00a0<\/p>\n

Nadella\u00a0did the same on behalf of Microsoft this week. \u201cWe offer the broadest model catalog in the cloud, with over 11,000 models, including the latest from OpenAI, Anthropic, Mistral, xAI,\u201d and Microsoft itself.\u00a0<\/p>\n

Venture capitalist Chris Dixon in his 2025 book Read, Write, Own<\/em> describes this phenomenon as \u201ccommoditizing the complement.\u201d Big Tech\u2019s recent open-source enthusiasm is not unlike Oracle\u2019s support for the development of the open-source Linux operating system in 2006. Oracle\u2019s strategy, as Dixon describes it, was to get volunteers to make and maintain an operating system that was cheaper than Microsoft Windows. And they did.<\/p>\n

Today, Microsoft and AWS see themselves more and more not as the future\u2019s most powerful builders of AI but sellers of tools, services, computing resources, etc., to a wide ecosystem of AI builders\u2014including their own.<\/p>\n

Among those services are AI tools like Microsoft Copilot and Amazon Inspector, which help find malware and vulnerabilities in builder code, including code from open-source libraries.<\/p>\n

Poisoning the future<\/strong><\/p>\n

AWS said adversaries are increasingly turning to AI not just to find vulnerabilities in open-source code, but to poison those code libraries in ways even other AI security programs don\u2019t detect; for instance, malware that only executes when a user issues a prompt that has a typo or that only works when other code is entered into the library later.<\/p>\n

This dangerous code is often cloaked in helpful suggestions.\u00a0<\/p>\n

\u201cHow does that malicious package or software get into that open source?\u201d asked Rick Anthony, Sr., who manages Amazon Inspector. \u201cAttackers are gaining trust\u2026 They’re going out and they’re acting like real developers. You know, they’re creating packages, and these packages are doing real useful benefits.\u201d<\/p>\n

These techniques are easier to execute with AI coding agents, Anthony said. \u201cThey can sit there and have very reasonable-looking contribution histories. They can have very useful release cycles, and before you know it, these attackers look like good citizens within the open-source community.\u201d<\/p>\n

Attackers are also exploiting the fact that more and more security reviews now happen via AI agents, which have weaknesses and blind spots. \u201cWhat we’re going to see is attackers not only try to fool the humans, but try to fool the AI by giving it enough evidence to convince it that what you’re running is \u2018OK.\u2019\u201d<\/p>\n

China and Russia are in a great position to carry out such attacks because they don\u2019t face penalties for running experiments on real-world targets, AWS Chief Security Officer Stephen Schmidt said. he was He said he is \u201creally concerned\u201d about them.<\/p>\n

AWS is employing red teams running with their own AI agents to find vulnerabilities in code before adversaries can exploit them, but also to attack emerging open-weight models, hoping to discover potential adversaries’ tactics before adversaries do.<\/p>\n

But finding a hole or vulnerability is only the first part of the challenge, Schmidt said. Developing an actually useful patch takes longer. So AWS is also looking to speed up sending its fixes to the problems it encounters, and then test them again against threats they haven\u2019t yet thought of.<\/p>\n

\u201cWe test the [patches] for not only performance but also the way that they respond to certain kinds of adverse behavior, because we know that the adversaries are going to go after them as soon as we release them to the public,\u201d Schmidt said.<\/p>\n

That, too, will become increasingly difficult for more and more organizations precisely because defenders are now using AI to find more vulnerabilities. Following Anthropic\u2019s release of its powerful Mythos model to a handful of companies, the number of vulnerabilities researchers found and disclosed quickly doubled, as did the number of patches. Each new bug found is a victory, but it also increases the work to develop a good patch.<\/p>\n

\u201cWe are running this as a security industry as a sprint\u2014oh my gosh! You know this big thing called Mythos came out, and we\u2019ve got to do all this vulnerability identification. This is going to be the long haul. We’re going to be doing this forever\u201d Schmidt said.<\/p>\n

The ramifications of that for future software development, and AI model building in particular, are significant. New AI builders will have to invest in protective AI at the same rate they invest in building new tools, he said.<\/p>\n

That\u2019s one reason why not everyone is excited about the future of open-source models.<\/p>\n

Anthropic is notably absent from Huang\u2019s statement. The company\u2019s CEO issued his own statement this week, stating that he isn\u2019t for banning open-weight models outright, but he supports mandatory safety testing for all models, as well as other measures to curb China\u2019s ability to copy powerful models like Mythos.<\/p>\n

Anthropic researcher Julie Merz was more direct about the threat in a Sunday post on X: \u201cThis time next year there will be the internet hitting every rural hospital\/city council\/etc at once with crypto locker attacks,\u201d she said. \u201cI think there\u2019s a shocking lack of imagination in a lot of the CEOs\/influencers pushing open models.\u201d\n\n<\/svg><\/p>\n<\/div>\n