AI Apps Should Use Closed Models Until It Hurts
Harvey spends hundreds of millions of dollars a year on inference. Co-founder Gabe Pereyra says app companies should rent closed models until the bill forces them to train their own.
· 7 min read
Gabe Pereyra, co-founder and president of Harvey, interviewed by Brendan Foody (The Frontier)
Gabe Pereyra is the co-founder and president of Harvey, the legal AI company used by large law firms and in-house teams. He grew up in Cupertino, the son of two computer science PhDs, and dropped out of high school to play soccer in Brazil before turning to AI research. He worked at Google Brain, DeepMind and Meta, and worked with Yoshua Bengio and Geoffrey Hinton before starting Harvey with his former roommate Winston Weinberg in 2022. Harvey now spends in the hundreds of millions of dollars a year on inference and has started building a post-training research team.
Summary
Gabe Pereyra says he was wrong to push for custom models at Harvey in the early days, and that application companies should build a sticky product on closed models until their inference bill makes training their own an obvious call. For Harvey that point is here, with inference spend in the hundreds of millions, and it is hiring post-training researchers to cut cost and to handle 80-million-token data rooms. He sees law as harder than code, since it has no unit tests and no public data, and he expects the training signal to come from each firm's partner red lines. His bet is that law firms are the first industry to change end to end, and that the base of the pyramid fills with agents. These signals are from an interview that Brendan Foody did with Gabe Pereyra on The Frontier.
