"Turpentine VC" | Venture Capital and Investing
00:13:24 · Brendan Foody, Usman Hanif
[00:13:24] S2: Yeah. So we started out with just textual data, like just the transcript of the interview. And then we expanded to also do visual stuff. But I will say that we're just starting to dip our toes into a lot of the more complicated visual stuff just because models are only now beginning to have really precise, you know, ability to extract features from these interviews. But I find it to be such like an interesting technical problem, because it's something that humans struggle immensely with. And models are actually superhuman. Like MIT released a paper that they were able to predict the heart rate of an individual and an interview with a reasonably high degree of competence based on the recording of that interview. And I believe that there's all of these subtle nuances and features of the heart rate of the individual, the facial expressions, have the tone, all of these, you know, little nuances that actually go a long way in predicting that person's attitude, their passion, their competence, and all these different characteristics and how they'll perform for a role. And so our focus now is in digging in beyond just what they're saying to all the other characteristics of that individual and how all of these subtle things that are often overlooked can can be priced in reasonably to predict their performance in a given task or job. [00:14:47] S1: And and have you seen any promise with that, like being able to take their facial expression or, you know, their voice inflection, whatever it may be? Is that actually a pretty reasonable predictor?
00:14:58 · Brendan Foody
[00:14:58] S2: So we found that audio is certainly something that's really, really important and has way more signal than people's transcript. However, the visual evaluation is just starting to take we found some signals and visual evaluation. There's interesting things like the blurriness of someone's camera actually contains some signal. Wow. Yeah. And, and there's like little features like that. We're really just dipping our toes into the facial expression stuff. But I believe that as we get more data about this, that there's going to be a lot of signals that emerge. One thing we talked about a lot internally is that like, this fascinating problem of predicting human ability is extremely high dimensional, right? There are so many things that go into what make you good at a given task or job, everything from, you know, all the different tangentially relevant experiences to your motivation to the manager in that job, whether they're like incentivizing you and you want to do a good job. And because the problem is so high dimensional, similar to predicting the next token and a transformer model, it requires an immense amount of data to actually help you meaningfully extract the right features. And so one of the reasons that we're so excited about the business and all of the progress that we're making, is that it allows us to operate at a much larger scale with a lot more data, where we're actually able to make sense of these, you know, small features that we previously weren't able to. And so as the performance data expands, it makes all that possible.
00:16:37 · Usman Hanif, Brendan Foody
[00:16:37] S1: Yeah, absolutely. And just to probe a bit more here, just because I find this technology to be so exciting. You know, when when you're collecting this information, you know, now are you taking into account more, say, multimodal models, vision language models? Is is this where you guys are headed now if you're expanding beyond just the textual space? [00:16:56] S2: Completely. Yeah. So that's why the frontier models that can do good multimodal evaluation are extremely compelling to us. And we've been using vision models for a while and are starting to, you know, explore a lot of what's possible with audio models as well as more sophisticated features with vision models. There also are other kinds of custom models we run experiments with, like the facial expression stuff that we were previously discussing as an example of that and beyond beyond just language models. But there's there's all sorts of models that go into extracting good features. And then it's about, you know, mapping those features to the right performance metrics that we wanna optimize for.
00:16:56 · Brendan Foody, Usman Hanif
[00:16:56] S2: Completely. Yeah. So that's why the frontier models that can do good multimodal evaluation are extremely compelling to us. And we've been using vision models for a while and are starting to, you know, explore a lot of what's possible with audio models as well as more sophisticated features with vision models. There also are other kinds of custom models we run experiments with, like the facial expression stuff that we were previously discussing as an example of that and beyond beyond just language models. But there's there's all sorts of models that go into extracting good features. And then it's about, you know, mapping those features to the right performance metrics that we wanna optimize for. [00:17:38] S1: Yeah. And and I'm curious in the early days. I mean, you know, the sort of the... You know, it it makes complete sense that once you get a bit more data on one of the candidates, say they've done a couple jobs through the the platform, the reviews are pretty valuable. But when it comes to that first almost, you know, zero shot, one shot approach of sort of classifying this candidate, how did you guys sort of test this? Was it just like, pray that this works and and hopefully the the the company like it? Or how did you go about that?
00:17:38 · Usman Hanif, Brendan Foody
[00:17:38] S1: Yeah. And and I'm curious in the early days. I mean, you know, the sort of the... You know, it it makes complete sense that once you get a bit more data on one of the candidates, say they've done a couple jobs through the the platform, the reviews are pretty valuable. But when it comes to that first almost, you know, zero shot, one shot approach of sort of classifying this candidate, how did you guys sort of test this? Was it just like, pray that this works and and hopefully the the the company like it? Or how did you go about that? [00:18:10] S2: It started with a lot of doing things that don't scale and was, you know, of doing human processes and think how we can replace them, where, you know, my co founders and I have done thousands, not exaggerating ourselves. And we obviously would be able to reach the scale that the models are at now for us at hundreds of 1000s of interviews. But you gain an immense amount of appreciation for like all the different ways to optimize an interview experience, and the importance of automating all of those processes. And so it started with, you know, manually reviewing resumes, and then figuring out how we can automate at least the shortlisting, even if not the final review. And then I remember for a time, we automated conducting the interview, but not yet evaluating the interview because it would actually be way faster. We could just like watch interviews on two x speed and then, you know, and and stop an interview if someone wasn't good. And then and then pushing towards complete automation, which is which is where we're
00:19:09 · Erik Torenberg, Usman Hanif
[00:19:09] S0: at today. Hey. We'll continue our interview in a moment after a word from our sponsors. Hey, everyone. Eric here. In this environment, founders need to become profitable faster and do more with smaller teams, especially when it comes to engineering. That's why Sean Lenehan started Squad, a specialized global talent firm for top engineers that will seamlessly integrate with your org. Squad offers rigorously vetted top 1% talent that will actually work hard for you every day. Their engineers work in your time zone, follow your processes, and use your tools. Squad has front end engineers excelling in TypeScript, React, and Next. Js ready to onboard to your team today. For back end, Squad engineers are experts at Node. Js, Python, Java, and a range of other languages and frameworks. While it may cost more than the freelancer on Upwork billing you for forty hours but working only two, Squad offers premium quality at a fraction of the typical cost without the headache of assessing for skills and culture fit. Squad takes care of sourcing, legal compliance, and local HR for global talent. Increase your velocity without amping up burn. Visit chewsquad.com and mention turpentine to skip the wait list. [00:20:18] S1: I see. What would you say was the most challenging part when it came to building this technology? I mean, it could either be one of these specific features, whether it be the AI interviewer or the resume analyzer, or maybe it was just at a broader level, maybe data collection, like what was really hard to get right for this to work?