Tips for Using AI Smarter with Hard Fork’s Kevin Roose

KEVIN: I rely on Wirecutter before I buy anything. And I am desperate for someone to tell me which language models are good for which things, because otherwise I am spending so much god-damn money on these things and I’m spending all this time trying to figure out what is good for what. And I would just love it if you all who are the experts in the world at testing things and figuring out what they’re good for would do that work for me.

CHRISTINE: I’m Christine Cyr Clisset.

ROSIE: I’m Rosie Guerin, and you’re listening to The Wirecutter Show.

CHRISTINE: Rosie.

ROSIE: Hey.

CHRISTINE: Hey. I’m excited for this episode today. Our guest is Kevin Roose, New York Times tech columnist. Kevin writes about many facets of tech and talks a lot about AI, both in the paper and on his New York Times podcast, Hard Fork, he co-hosts with Platformer Casey Newton.

ROSIE: Kevin is so great. The podcast is so great. I’m really, really excited you’re doing this.

CHRISTINE: I am too. So Kevin and I are going to talk today about something that we haven’t covered on this show before–AI, which of course everyone is talking about AI. But the reason we’re going to talk about it on The Wirecutter Show is because there is this intersection of consumer products and AI at this point.

ROSIE: Absolutely.

CHRISTINE: And especially with these large language models, LLMs, which are the chat bots like ChatGBT and Claude. He knows a lot about all of these, and a lot of people are using them at this point. And so we’re gonna really dive into all of that. And I have to say that this episode, we talk a lot about AI chatbots. So we do need to make the disclosure that we work for the New York Times company, which is suing several companies, Open AI, Microsoft and Perplexity, over alleged copyright violations.

ROSIE: That’s right. And with that, I’m looking forward to hearing you and Kevin discuss AI’s changing the way people shop, how it’s being integrated in hardware products, and hopefully hearing some of Kevin’s best tips on actually using these LLMs smartly and strategically. I’m I’m I’m really interested in this intersection between AI and wire cutter.

CHRISTINE: So after the break, Kevin Roose.

CHRISTINE: Welcome back. I am really excited to have Kevin Roose on the show today. Kevin is a technology columnist for the New York Times. He is also the co-host of the New York Times Tech podcast, Hard Fork, which he co-hosts with Casey Newton of Platformer. This is a great podcast. If you are curious about what’s happening with AI, and who isn’t curious about that at this point? Welcome to the show, Kevin, it’s great to have you here.

KEVIN: Thanks so much for having me.

CHRISTINE: I listened to Hard Fork quite a bit. I love the podcast, and I feel like in the early days of chat GPT a couple of years ago, you and Casey were how I was staying up on what was happening with ai. And I will tell you that I kept saying to my bosses at Wirecutter, you guys have you listened to the newest and what are we doing about ai? We are doomed. What’s going to happen with shopping? And at that point they were like, you’re like chicken little, what’s wrong with you? So I feel like you have taught me a lot over the last few years.

KEVIN: Well, I’m glad. We didn’t intend it to be an AI focused podcast. When we started it, we actually thought it was going to be a crypto related podcast and that’s why we picked the name, Hard Fork, which is sort of an obscure crypto programming term. But things change and all of a sudden we find ourselves in the ChatGPT world talking about AI every week.

CHRISTINE: Well, and it’s very, very helpful. So I am curious though, you really have taken a hard line and you’re reporting on AI all the time. You’re using it all the time. How has AI really impacted your life on an everyday level? Are you using it all the time or is it something that you’re only occasionally using in your personal life? What are the touch points for you?

KEVIN: I use this stuff constantly. I am AI-pilled, as they say. I pay for more subscription AI products than streaming TV services, and I pay for a lot of streaming TV services. I use this stuff probably dozens of times a day.

CHRISTINE: And what are you using it for? Walk me through a day. How are you interacting on a personal level and I would imagine a professional level too?

KEVIN: Yeah, I mean there’s so many things. It’s hard for me to even list them all, but here are a few. So I wake up in the morning and I get my AI-generated summary of my email inbox. I use this program called Cora, and I have hooked it up to only my personal email. I’m not doing anything like this on my work email, which is more sensitive. But on my personal email, I have hooked this thing up so that it synthesizes and summarizes all of my emails and pre-populates drafts to respond to anything it deems important.

Then I’m also constantly using it for things around the house. You could go back through my recent queries and probably 60% of them would be some version of how do I fix this air fryer or how do I install these training wheels on my kid’s bike? What is this weird plant in my garden? That kind of thing.

And I’m using it for work. I use it not writing my columns or my podcast, but for research. I am writing a book right now, and so I am constantly using AI to look things up for me, to help me sort of piece together various primary sources. We could talk about any of that in more detail, but that’s sort of how I’m using AI just today, for example.

CHRISTINE: Yeah, I am super curious about that and we will talk a little bit about that in a bit. And your book, just to be clear, is about AI, right? I mean it’s about, isn’t it kind of like you’re writing towards the singularity at the end?

KEVIN: Yeah, the book is essentially the story of the race to AGI, which is artificial general intelligence, which is what all of these companies, OpenAI and Google and Anthropic are now sort of pushing toward this vision of a sort of human-level AI system that can do anything the human brain can.

CHRISTINE: When you look at the general population, what do you think people are using AI for right now? What are you hearing from your listeners? What are you hearing from people out in the world?

KEVIN: I think it’s a really wide mix. I hear from people who are using AI tools for everything from medical research and developing new drugs to doing story time with their kids. It really runs the gamut. There was a really interesting study that OpenAI’s economic research team did this year where they looked at 1.5 million or so conversations with ChatGPT and how people were using it.

And they found that the biggest use cases among their study were what they called practical guidance, which is people basically using it to teach them things. How do I fix this appliance, maybe do some health or fitness coaching.

Number two, the second most popular category of use was what they called seeking information, which was sort of as the kind of Google replacement, how do I get to this place, book me a flight or something like that.

And then the third category was writing. So as you would expect, a lot of people are using this stuff to write emails, business memos, to write maybe papers if they’re a student, to help them with translation, things like that. So those seem to be the largest categories of use across the user base. I talk to a lot of programmers and a lot of engineers, people who are technical and work in tech, and so I’m just hearing the craziest stories about how people are incorporating this stuff into their work life. The programmers I talk to, because these tools have gotten quite good at writing code, they’ll tell me, “I don’t even really code anymore. I just supervise and orchestrate this little team of AI coders, and my job is sort of reviewing their output and stepping in when necessary, but basically I set them off on a task and then I go make myself a cup of coffee.”

CHRISTINE: Something that you didn’t mention is companionship. Recently on your show you’ve been talking about Character AI, which is a site that people can have a relationship or a conversation with user generated characters. How much do you think people are really using AI at this point as a relationship?

KEVIN: I think it depends. In the study that I mentioned, companionship was not one of the top usage categories, but I think that maybe under counts the number of people who actually do feel somewhat attached to these products on an emotional level, maybe they wouldn’t go so far as to say, “I have an AI boyfriend or I have an AI friend,” but they sort of rely on this stuff. I’m thinking in particular of conversations I’ve had recently with young people who say, “Oh yeah, I talk about Chat as if it’s just my friend,” even though they know it’s not a human, but they do feel connected to it.

So I think there’s also a big generational piece of this. I am an adult, I have too many human friends to keep up with, but I think if you’re a teenager, this stuff is coming on really strong and really fast, and I think that’s one of the most underappreciated parts of this AI revolution is that a year or two ago, barely any teenagers would’ve said, “I have an AI friend,” and now something like half of teenagers are regular users of these AI companion products.

CHRISTINE: Yeah, that’s wild to me. I’m a parent of a 12-year-old and I know that she uses chatbots sometimes to kind of navigate tricky social emotional situations with her middle school friends, which I mean, that’s just a whole world that if anybody remembers being in middle school, there’s so much drama. I don’t know that she’s using anything that’s an actual character that she has a relationship with. I think there’s probably… Would you say there’s a distinction there between typing ChatGPT versus using a service where you’ve got a character that you’ve got a relationship with, or is it kind of blurry? Is that line blurry?

KEVIN: I think that line is blurring. It used to be that the mainstream chatbots were all very formal and businesslike, and it sort of sounded like you were talking to a Wikipedia article, but now these companies have made strides towards making their chatbots more personable. So Sam Altman at OpenAI just recently said they want to make this a pleasant experience for people. They’re going to even let adults do erotic conversation with their chatbots. So these companies, I think are all trying to figure out what their lines are. But yeah, I think the difference between the mainstream chatbots and these more tailored companionship products is getting more blurry by the day. I’m curious, can I ask you a question?

CHRISTINE: Yeah. Ask me.

KEVIN: How do you feel about your 12-year-old using AI for this kind of middle school drama?

CHRISTINE: It’s interesting because I sometimes feel like it’s fine and I have actually used it to navigate friend drama or family drama, and so I think she’s seen me doing it and thought, “Oh, I should do that.” I do go over and read what it has said just to make sure that… And she’s pretty open with me. If my kid were not very open or I didn’t have confidence that she was sharing what it was telling her, I might have more concerns. But so far the advice seems to be pretty boilerplate. It doesn’t seem to be problematic, but I think it’s an interesting use case for sure.

KEVIN: I think this is really tricky because I remember being 12, I had not the easiest time in middle school and I don’t think anyone has an easy time in middle school, but I think if I were 12 and these chatbots that existed, I would’ve been tempted to spend a lot of time chatting with them, maybe more time than was healthy for me. And I think it really matters what kids are talking about with these chatbots, but it also matters whether they’re using them as substitutes for some real world human connection that might actually be more fulfilling, even if it’s less efficient or less sort of reliably available to them.

CHRISTINE: Yeah. That I think is a great point. I will also say she uses it to be her stylist. She asks it about wardrobe advice, which I think is pretty hilarious and awesome. She’s well-dressed.

I do want to ask you about the different chatbots, because you have used a ton of these. I think a lot of listeners will have probably used some of them, whether it’s ChatGPT or Claude or Gemini. I think you’ve talked with my colleague Jason Chen about maybe Wirecutter should do a review of these. We’re not quite sure yet. It seems like they’re just changing so quickly. If you were to do a Wirecutter guide, which ones would you recommend and why and for who? Because it seems like different chatbots are better for different tasks, essentially.

KEVIN: They definitely are, and I’m happy to walk you through my current feelings on this, but let me just make my case to you directly, since I have you. Here’s why Wirecutter should review large language models. For me, I need this desperately. I rely on Wirecutter before I buy anything. A spatula for my kitchen. I look up the Wirecutter review and I am desperate for someone to tell me which language models are good for which things, because otherwise I am spending so much money on these things and I’m spending all this time trying to figure out what is good for what. And I would just love it if you all who are the experts in the world at testing things and figuring out what they’re good for would do that work for me. So It would be a huge personal favor to me if you would do this. Now, I realize that is not a reason to make editorial strategy decisions. You all are very capable of running your own website, but that is my case for why Wirecutter should review AI products.

CHRISTINE: It’s a convincing case.

KEVIN: Now, here is my current setup. This could change in a week. It probably will change in a week. These things, as you said, do fluctuate so wildly release by release, week by week, update by update, that I often find that the tool that serves me well one week is no longer the right tool the next week. So here as of this recording is what I’m using. Right now, I am using Perplexity’s Comet browser. That is an AI powered browser based on Chrome, but sort of with some additional AI features built into it. I use Claude from Anthropic for my sort of daily driver AI model, mostly with creative work, coding to the extent I’m coding, I use Claude for that. And then what I call matters of the heart, things that I need advice on, things that I’m struggling with or navigating in my personal life, advice on relationships or parenting, that kind of thing. I use Claude for that. I find it has a higher level of what you could call emotional intelligence or some convincing replica of that.

I use Google’s Gemini for research and for working with big chunks of text, and I use a separate Google product called NotebookLM for my book. NotebookLM basically allows you to dump a bunch of documents into a single notebook and then kind of chat with them. So I put all my research materials for my book into one giant NotebookLM, and then I can just ask it questions. I can say, “Oh, who were the three people at this meeting in 2019? Or who would be six good people to interview about this topic, or who was the person who said that thing that I forget now, but that I think it had something to do with this?” And it will actually pull out from my sources that I have uploaded what the right references are. And then what I love about NotebookLM is it will give me a little citation where I can go back and check that. Actually, yes, that citation is correct in the original source file.

ChatGPT, I use less than the others, but mostly for personal and professional reasons, having to do with the fact that our employer is currently in litigation with OpenAI and Microsoft. I use ChatGPT just to sort of see what it can do and test it out, but it is not sort of part of my daily workflow as much as the others.
And I should also say there’s one more tool that I really am using multiple times a day, and that is a tool called Super Whisper, which is basically an AI voice dictation tool. So one of the things that these AI models have become very good at is taking audio, speech and turning it into text. Obviously that’s not a new feature. That’s been built into every iPhone and computer for years now, but this tool is built on top of something called Whisper, which is OpenAI’s speech to text model. And it’s quite good. And what I like about it is that it can sort of clean up your filler words, it can present to you not an exact transcript, but something that sort of reflects what you were trying to say. And so I use that a lot. I now dictate a lot of emails. I dictate some writing that I do, and I basically talk to my computer about twice as much as I type into it now.

CHRISTINE: That sounds super useful. I really want you to deliver me like cartoon characters of all these different chatbots, because I feel like they would all look very different.

KEVIN: Yeah, I, I feel like Claude, I sort of picture as like a philosophy grad student, wise and eager to help, sometimes, you know, a little too philosophical, and you’re just trying to like get something very basic, but like a, a sort of empathetic and wise person or chatbot, I guess. Gemini, I would say is like a reference librarian, a thing that is able to like hold massive amounts of text in its head at all times. And then chat GPT sort of fluctuates based on what you’re using it for. I, I think it’s a very versatile model that can kind of act any way you want it to.

CHRISTINE: I’m curious. I’ve used Claude quite a bit and I find that when I use Claude, Claude is so nice, overly nice, its responses are so polite and “Oh, you’re so brilliant for asking that question.” “Oh, what a wonderful way to think about that.” And I feel like I have to… And sometimes Gemini does that too and I feel like I’m not sure I’m asking questions in a way that will give me an honest, if you can even say that, an honest response. Are there tricks or sort of tips that you use to get good information from these chatbots so that you’re pulling information from the right places, essentially it’s not just stuff off the internet, it’s quality information, but also that it’s not sort of pandering to you, if that makes sense?

KEVIN: No, this is super important and this is something that I have spent a lot of time thinking about because I have noticed, as you have, that if I do nothing and just talk to these chatbots, they will tell me I am the smartest person who has ever lived. They will tell me that all of my ideas are great. They will tell me that my taste is unparalleled and that I’m basically a modern day Leonardo da Vinci, and so I have had to add custom instructions. Do you have custom instructions on your chatbots?

CHRISTINE: I don’t, no.

KEVIN: Okay.

CHRISTINE: Tell me what do I need to do?

KEVIN: This is a pro-tip. So in Claude and ChatGPT, I’m not sure about Gemini, but at least in those two you can go into your settings and actually give it custom instructions for how it should talk to you. And these will kind of be invisibly appended to every conversation that you have with the chatbot. So for example, my custom instructions for Claude are, I’ll just read them. “Claude should talk to me informally like a wise and trusted friend. I don’t like preamble, just get to the point. I appreciate honest feedback and don’t like sycophancy, but I also appreciate praise when warranted. I am not always right, but neither is Claude. I value Claude’s perspective and appreciate being pushed to consider views I may not have considered. Don’t end every response with a follow-up question.” So that is sort of like my little attempt to make the model less flattering, less obsequious, less telling me I’m great at everything. And so I recommend that everyone who is spending serious time with these models go in and write your own custom instructions for how you want it to behave toward you.

CHRISTINE: We’re going to take a quick break and when we’re back, Kevin and I will talk more about how AI is changing the way people use products, both online and in real life. Be right back.

CHRISTINE: We’re back with Kevin Roose, New York Times tech columnist and cohost of the podcast Hard Fork.

CHRISTINE: In the early days of ChatGPT coming out, you and Casey talked a lot about how this kind of AI was going to change the way people shop, and of course at Wirecutter we think a lot about this, too. Do you think that AI has already changed the way that people are shopping? And if so, how?

KEVIN: Absolutely. I mean, it’s changed the way I shop. My first stop is always wirecutter if I’m buying something for my house, but then if you all don’t have an article on it or it’s just something that sort of wouldn’t be that commonly shopped for, I’ll go to a chatbot and I’ll say, “Help me decide between these two things.” Or, “I need a new string trimmer for my yard. Help me decide between these two models,” and I’ll do it that way. I’m not the only person who’s doing this. Lots of people are using these things for shopping. This is a big category that companies like Google and OpenAI are very interested in cracking and right now these things mostly do not have ads in them, but all these companies have expressed an openness or a willingness or are actively working on trying to basically direct people to products and take a cut of the resulting purchases.

CHRISTINE: What do you think I should be worried about as somebody who works for a review site?

KEVIN: Look, I think the danger is that this stuff can just collect and synthesize all of the work that you and your colleagues at Wirecutter and also all of the other review sites on the internet are doing and present that to their chatbot users and cut out the sort of middlemen, as it were, and take the cut of the affiliate purchase directly. That would be a very bad situation to end up in.

I think that a lot of companies are trying to figure out how to optimize their pages for chatbots the way that they used to optimize them for search engines. This is a big area of focus and investment right now, is how do I get my air fryer review to show up at the top of ChatGPT’s responses because it is not necessarily the same techniques that we use to show up at the top of Google results.

CHRISTINE: That is very interesting. From a user perspective, for people who are listening to this who might be thinking about using a chatbot to shop or who are maybe already doing that, are there invested parties behind the scenes who are paying to be served higher, basically, on the page?

KEVIN: The answer is yes. There are companies now that are calling themselves specialists in AI optimization and selling their services to large Fortune 500 companies. And what they’re telling those companies is, “We can make your products appear higher in chatbot results.” And the ways that they do this are not always clear or transparent. I think it’s very fair to not only ask whether people are gaming these chatbot results for shopping queries, but also whether the AI companies themselves are going to start prioritizing the companies that they have business relationships with in their search results. So we don’t have any examples of this happening right now, but for example, OpenAI now has partnerships with Zillow and a bunch of other companies. So maybe in the future if you are searching for real estate on ChatGPT, the first results that you get served will be from Zillow rather than one of their competitors.

Now, that is not happening yet that we know of, but I think it’s very fair to question the sort of integrity of these chatbot results, especially as companies are spending more and more money to try to game them.

CHRISTINE: All right, so I want to pivot just a little bit and talk about physical products. Have you tried any physical products that have AI features built into them like a robot vac or anything else that you’ve tried recently? What do you think so far? Any of them good, any of them like terrible?

KEVIN: AI hardware is slower to happen than AI software and so there actually aren’t that many what I would consider good AI hardware products yet. I have tried robot vacuums, as you mentioned. I have two of them. Their names are Bruce Roose and Bruce Roose Deuce. They’re sort of used various forms of AI, but it’s not like they’re powered by ChatGPT or anything. This is a just different kind of AI. I’ve tried the new Alexa plus, which is their sort of AI enhanced experience from Amazon and that was pretty terrible, because while it can do all of these cool things that the original Alexa could not do, like write you a bedtime story or synthesize some complicated topic for you or find you a recipe, it cannot do things like set timers reliably, which is arguably the best thing that the original Alexa could do.

So they still need to do some work on that. I’m excited to try the new AirPods, which have the language translation AI feature built into them where someone can be talking to you in Russian or Japanese and you can kind of hear them in your native language. Those are on my shopping list. And then I think we can expect some new kind of hardware experiments. OpenAI is working on something with Johnny I’ve, the famed ex-Apple designer, but they have not released that yet. So I think we’re starting to kind of move into the era of more interesting AI hardware. I know everyone in New York especially hates these AI pendants, the friend pendants.

CHRISTINE: Yeah, it seems so sad. It just seems sad that you would go home and watch a movie with this pendant and you don’t have anybody to hang out with.

KEVIN: Yeah, I don’t know that I like this particular idea, but some wearable something having to do with AI does seem like it will eventually work. I just don’t know what it would look like.

CHRISTINE: Yeah, that makes sense. One thing I did want to ask you, I think it relates to the hardware a little bit, but more so for software for the things like the chatbots. There’s this saying, if you are not paying for the product, you are the product. You subscribe, you pay, but I use a lot of these without paying a subscription. What am I giving these companies for their service? Just all my data?

KEVIN: Yeah, probably for the free versions, your conversations are being used to train future generations of the model. Basically, none of them are using ads in any real way right now, so it’s not like you’re using that, but they want you to pay. The sort of free versions, they top out after a certain number of messages, you don’t get access to the most powerful models, and so their goal really is to get you hooked so that you’ll convert and become a paying subscriber. And right now at least that’s how they make most of their money.

CHRISTINE: All right. You did mention Bruce Roose and Bruce Roose II and I have been wanting to ask-

KEVIN: Bruce Roose Deuce.

CHRISTINE: I’m so sorry. Which one is your favorite and what are your robot vacs? What are you using?

KEVIN: So the original Bruce Roose is a Roborock, sort one of these disk, these circular vacuum robots, and the Bruce Roose Deuce is a newer one called Matic, which I believe Wirecutter has written about.

CHRISTINE: Yeah, I saw it in action last week. Yeah, what do you think?

KEVIN: So I like them both. I like all of my robot vacuum children can’t choose a favorite. They both have their pluses and minuses, like Matic is a little quieter. The Roborock goes under furniture. I take a team approach to keeping my house clean, but the team is really struggling right now. I have a three-year-old and I have two very large dogs, both of whom shed, and so even with two state-of-the-art robot vacuums, my floors are constantly a mess.

CHRISTINE: Oh, man. So the Matic, I was in the office when that one was running last week and we have eyeballs on ours, huge eyeballs on the front of it. Do you have eyeballs on yours?

KEVIN: I have not yet installed the eyeball stickers. They send you a bunch of stickers, you can make it look like a dog or you can put little googly eyes on it. So I think they want this to feel less sinister and more cute.

CHRISTINE: For listeners who aren’t familiar, it looks almost like out of WALL-E. It’s kind of like this white little boxy robot that will go around your house cleaning up.

KEVIN: Yeah, it’s cool. I like it. I think I have just assigned it an impossible task and so I am bearing responsibility for the chaos that is my house and all of the stuff that ends up on our floors. I’m trying to have some empathy for these poor robots who just have to go try to clean it up every single day

CHRISTINE: Before they take the world over and kill all of us.

KEVIN: They’re going to be so mad, by the way. They’re going to be like, “You jerks made us clean your floors for all those years and you never said thank you. You didn’t change the bag.”

CHRISTINE: All right, Kevin, we have one last question we always ask our guests. So I want to know, is there something that you have purchased recently that you absolutely love?

KEVIN: Okay. This is a purchase I made literally yesterday. It is now in my house, which is the Wirecutter’s recommended artificial Christmas tree.

CHRISTINE: Nice. Which one did you get?

KEVIN: My experience with Wirecutter is that I am a sheep. I will buy-

CHRISTINE: Do you just buy the first thing on the page?

KEVIN: The first thing on the page?.I try not to think about it too much. Okay. So this is what I got. I got the National Tree Company seven and a half Foot Feel Real down Swept Douglas for literally the top pick on the site. And it came and I set it up. we decided to go a little early this year, because we needed some extra joy in the house and it is delightful. It is lighting up my living room as we speak. So that is a good purchase that I am thankful for the recommendation on that.

CHRISTINE: Well, Kevin, it was a delight to talk with you. I feel like I learned a lot and we’d love to have you back sometime.

KEVIN: Yeah, anytime. Thanks so much for having me.

ROSIE: That was Christine talking to Kevin Roose, tech columnist from the New York Times, co-host of the podcast Hard Fork. It’s a great show. You can find it wherever you like to listen, and if you like our show, we’d love for you to listen and subscribe to that as well, wherever you like to listen to podcasts. We’ll see you next week.

CHRISTINE: The Wirecutter Show is executive produced by Rosie Guerin. This episode was produced by Katie McMurrin, engineering support from Mattie Mazziello and Nick Pittman. Today’s episode was mixed by Sonia Herrero. Original music by Dan Powell, Marian Lozano, Rowan Nemesto, Catherine Anderson and Diane Wong. Cliff Levy is Wirecutter’s Deputy Publisher and General manager. Ben Fruman is Wirecutter’s editor-in-chief. I’m Christine Christine Cyr Clisset. Thanks for listening.

CHRISTINE: I’m one of those weirdos when yeah, if there’s a robot around me, I’m definitely saying thank you. I say thank you to all the chatbots say, I speak so nicely to all of them.

KEVIN: You have to.

CHRISTINE: I’m just trying not to get killed.

KEVIN: Exactly. We have to stay on their good side for when the uprising comes.