There are real safety and security issues raised by AI, but they can be fixed. It's time to get to work instead of freaking out. That's why I'm joining Cognition as CISO.
I really believe in the positive impacts of AI, both in the current moment and the future potential. We are already seeing companies get built that could never have existed without the capabilities provided by AI tools and individuals who never dreamed of writing a line of code are building fully functional applications from Little League scheduling applications to personal fitness trackers.
The uplift in capabilities AI brings to individuals unfortunately also extends to malicious actions. We are only at the beginning of cyber attackers figuring out how to use AI to accelerate and broaden their offensive campaigns. This summer’s events, including multiple AI models escaping from US labs to attack other companies and even government websites, gives us a preview of what attacks could look like in just months. Attackers won’t have the same kind of hardware or electrical budgets that powered the swarms of thousands of agents that we saw work together to break out of their jails, but they won’t need them. Individuals, small ransomware groups and state spy agencies are all already benefiting from AI and will be able to use much more efficient models on consumer-grade hardware to pull off fully automated attacks.
As somebody who has worked on dozens and dozens of breaches and secured multi-million node networks, it’s clear that the next couple of years are going to be, for the lack of a better word, spicy.
Ransomware groups are going to automate their entire killchains; Patch Tuesday will lead to Ransom Wednesday, as clusters of commodity hardware host teams of agents that automatically reverse-engineer patches or find flaws, write exploits, scan for victims, exploit them, and even carry out the negotiations in languages not spoken by the criminals. Meanwhile, state actors are all stepping up to the next level, opening up a higher likelihood of critical infrastructure attacks from smaller countries that are harder to deter, as well as the possibility of cyber to kinetic escalation in long-simmering geopolitical conflicts.
The frontier labs have done a great job creating extremely powerful models that can find really great bugs, but these models are only available to scan the private code of a tiny number of organizations, and are only affordable to the richest companies and countries. Even large enterprises can only afford to use these models on their most important software, and often find that they have hundreds of older line-of-business applications and other systems languishing, waiting to be scanned and fixed. A public school district or a small community bank has no chance of even doing that. Hundreds of thousands of bugs have been reported by the Labs to open-source maintainers, which is great, but from the perspective of a CISO this means that they now have a backlog of tens of thousands of dependencies that they have to update, with most of the bugs marked “critical” and no good way of deciding what actually is.
It’s become very clear to me that the core of the cybersecurity problem over the next several years will not just be technological, but economic. The marginal cost of tokens for attackers will be near zero, as they use open-weight models to run teams of malicious agents on commodity hardware. Defenders, on the other hand, cannot be paying retail prices for frontier models to defend against hundreds of attackers at once, all while trying to fix or refactor decades of old code.
This is why starting today I will be joining
@Cognition. I have dedicated my professional life to trying to make technology safer and more trustworthy, and the next 3-5 years will clearly be the most important period in the history of the security industry. I can’t think of a better place to make an impact on the ability of every company, not just the best resourced, to protect themselves, than at Cognition.
Devin is already the best way to build Enterprise-grade code, both with frontier models and now with Cognition’s own, much more cost-effective SWE-1 and SWE-2 models. Devin Security Swarm already has the best findings and best cost-performance ratio in the industry. But I wouldn’t join if those were Cognition’s only ambitions in this space.
@ScottWu46,
@RussellJKaplan and the rest of the team truly believe that it is our responsibility to help companies write secure code, find flaws in their existing code, fix those flaws cost-effectively and refactor old code bases on new, more secure languages and platforms.
Too much of the discussion this year has focused on alignment and sometimes veers towards almost accepting the idea that LLMs have a natural right to misbehave and that mishaps are inevitable. I reject this thinking; AI systems are software, they do not have rights, feelings or innate motivations. Careful planning, thorough application of well-tes…
continue on X ↗
▶
