A startup referred to as Letta has simply emerged from stealth with tech that helps AI fashions bear in mind customers and conversations. Created in UC Berkeley’s famed labs startup manufacturing unit, it additionally introduced $10 million in seed cash led by Felicis’ Astasia Myers, at a $70 million post-money valuation. 

Letta can also be backed by a who’s who of angel traders in AI, like Google’s Jeff Dean, Hugging Face’s Clem Delangue, Runway’s Cristóbal Valenzuela, and Anyscale’s Robert Nishihara, amongst others.

Based by Berkeley PhD college students Sarah Wooders and Charles Packer, this can be a extremely anticipated AI startup launch. That’s as a result of it’s a baby of Berkeley’s Sky Computing Lab and is the business entity of the favored MemGPT open supply mission.

Berkeley’s Sky Computing Lab, led by acclaimed professor and Databricks co-founder Ion Stoica, is the descendent of RISELab and AMPLab, which spawned such firms as Anyscale, Databricks, and SiFive. Sky Lab, specifically, birthed quite a few in style open supply giant language mannequin (LLM) tasks just like the Gorilla LLM, vLLM, and the LLM structured language SGLang.

“A ton of tasks in a short time, inside a yr’s time-frame, got here out of the lab. Simply folks sitting subsequent to us,” described Wooders. “So it was form of an unbelievable time.”

MemGPT is one such mission and is such a sizzling commodity that it truly went viral earlier than it even launched. 

“Somebody scooped us,” Packer instructed TechCrunch. The founders had posted a whitepaper on Thursday, October 12, 2023, and deliberate to launch a extra in-depth paper and the code to GitHub the next Monday. Some random particular person discovered the paper, posted it to Hacker Information on Sunday, and it “went viral on Hacker Information earlier than we had an opportunity to correctly launch the code, launch the paper, or, like, do a tweet thread or something like that,” he mentioned. 

The rationale for the joy was that MemGPT mitigates a pernicious drawback for LLMs: Of their native type, fashions like ChatGPT are stateless, that means they don’t retailer historic information in long-term reminiscence. That is problematic for AI apps that depend upon attending to know and be taught from a person over time — every little thing from buyer assist bots to healthcare symptom-tracking apps. MemGPT manages information and reminiscence in order that AI brokers and chatbots can bear in mind earlier customers and conversations.

The put up on the paper stayed atop Hacker Information, the favored website for programmers run by Y Combinator, for 48 hours, Packer recounted. So he spent his weekend and the following few days answering questions on the positioning whereas attempting to get the code able to be launched. As soon as the mission was out there on GitHub, a hyperlink to it went viral on Hacker Information, once more. YouTube interviews and tutorials, Medium posts, 11,000 stars and 1.2K forks on GitHub occurred shortly.

VC Felicis’ Myers found Wooders and Packer by studying about MemGPT, too, and instantly acknowledged the tech’s business prospects.

“I noticed the paper when it was launched,” she instructed TechCrunch, and she or he promptly reached out to the founders. “We had an funding theme round AI agent infrastructure and appreciated {that a} actually essential element of that was the info and reminiscence administration to make these conversational chat bots and AI brokers efficient.”

The founders nonetheless nearly traipsed round Sand Hill Highway doing Zoom calls with VCs earlier than going with the one which beloved them first.

In the meantime, Stoica brokered introductions to Dean, Nishihara and different big-name Silicon Valley angels. “A number of the professors at Berkeley, simply as a consequence of being at Berkeley, are very nicely related,” Packer recalled about how straightforward the angel investor course of was. “They’ve their eye on tasks out of this lab which can be going to be commercialized.”

Competitors and the specter of OpenAI o1

Whereas MemGPT is already out within the wild and getting used, Letta’s business variant, Letta Cloud, is just not but open for enterprise. As of Monday, Letta is accepting requests for beta customers. It is going to supply a hosted agent service that enables builders to deploy and run stateful brokers within the cloud, accessible through REST APIs, a programming interface that may preserve state. Letta Cloud will retailer the long-term information needed to take action. Letta may even supply developer instruments for constructing AI brokers.

With MemGPT, Wooders sees a big span of makes use of. “I feel the primary use case that we see is principally, extremely customized, very participating chatbots,” she says. However there are additionally cutting-edge makes use of like “a chatbot for most cancers sufferers” the place sufferers add their historical past after which share ongoing signs so the bot can be taught and supply steering over time. 

Value noting that MemGPT isn’t alone in engaged on this. LangChain might be its finest identified competitor, and it already presents business choices. The largest mannequin makers additionally supply AI agent-making instruments as nicely, like OpenAI’s Assistants API. 

And OpenAI’s new o1 mannequin could make the necessity to repair state a moot level for its customers. As it’s a multistep mannequin, it basically should preserve state to some extent with the intention to “suppose” and reality examine earlier than it replies.

However Wooders, Packer, and Myers see just a few key variations to what Letta is providing versus what 800-pound market gorilla OpenAI is doing. Letta claims it is going to work with any AI mannequin and expects its customers to make use of a lot of them: OpenAI, Anthropic, Mistral, their very own homegrown fashions. OpenAI’s tech at present solely works with itself. 

Extra importantly, Letta is utilizing open supply MemGPT and jumping firmly into the open source side of the FOSS vs. black box LLM debate, saying open supply is a better option for AI utility programmers.

“We’re positioning ourselves because the open various to OpenAI,” Packer says. “I feel it’s truly very, very arduous to construct excellent AI purposes, particularly while you care about, like hallucination, should you can’t see what’s happening underneath the hood.”