One Year In, Skift Plans Next Steps With Ask Skift

While the media industry worried about the impact of AI on their businesses, travel publication Skift immediately went to work to figure out how AI could help its audience. One of its first endeavors, Ask Skift, launched in May 2023 to give subscribers a chat bot-like experience trained on only Skift content and public SEC reports.

At the time, I wrote:

As a product for subscribers, this is a no-brainer. What I particularly like about this is it introduces functionality to a media company. Now you’re not just selling content; you’re selling an easy-to-access archive of topical information that surfaces content for readers to get more context. That makes the product stickier because readers not only need new information, but they also need to be able to easily access older stuff as well.

How can a long-standing news company easily surface buried information in their archive? Site searches are often ineffective, leading users to Google the domain and keyword instead. Offering users the ability to ask a question and receive trusted information from a well-trained bot would greatly improve user experience.

A year in, how well has the product been received and what do current and prospective subscribers think about it? Skift partnered with House of Kaizen to conduct a study to answer those questions. (Disclosure: I’ve done some work with House of Kaizen unrelated to this project.)

According to the survey of 344 respondents (56% subscribers, 30% non-subscribers, 5% unclear), the results are generally positive. 44% were aware of Ask Skift, with 22% saying it increased their visits to Skift.

Jason Clampet, co-founder and Chief Product Officer at Skift, told A Media Operator, “we’ve had a little over 23,000 questions asked since December when we started tracking the questions.” That suggests over 3,000 questions monthly. “Users can thumbs up, thumbs down [answers], they can also email it to themselves. And so, we see about 10% of people making a choice to give feedback or to email to themselves. A lot of people just cut and paste.”

Clampet was pleased with the engagement, especially considering a majority of people were unaware of its existence. Whether it contributed to concrete business outcomes, such as subscriber acquisition or retention, remains to be seen.

I don’t know if it’s driven subscriptions yet. I think mainly because we do a bad job selling it at this point. We do a bad job identifying that this is part of the value of your experience. I would like to think that it is just a slightly additive thing to a subscription that makes you feel like, ‘okay, I’m getting value for what I’m doing here, and I can get answers I couldn’t otherwise get.’ I think that’s the point we’re at now.

It’s a good point to be at, but Clampet admits there’s a lot of work needed to make the product more robust beyond selling it better.

Peter Figueredo, founding partner at House of Kaizen, told A Media Operator, “Skift has first mover advantage, but AI adoption is growing rapidly. To capitalize on this advantage, they must keep Ask Skift optimization efforts at the top of their priority list. Maximize the return on their investment by creating a testing and optimization flywheel fed by audience research and experimentation.””

The team identified an area for improvement in the product’s appearance based on audience data. “Four to five percent of people think that it’s a customer service thing,” Clampet said, which makes sense given its location on the lower right-hand side. To overcome this, he explained they will position it more as a research tool than a chatbot.

“We won’t use the words research (they already have a research product), but like a landing page where there’s a lot more interactive things,” Clampet said. Ironically, considering Perplexity is disregarding Robots.txt and continuing to steal publisher content, Clampet hopes to draw inspiration from Perplexity’s product layout. “They ask you specific follow-up questions that are contextual to what you asked initially,” he explained.

Another area where Skift is going to spend time exploring various other models. When it launched, Ask Skift used OpenAI’s GPT-3.5. Since then, they’ve upgraded to GPT-4 and are experimenting with other LLMs. But it’s not simply just throwing a new LLM in and hoping it works. “We made tweaks, looking at the context window, which is where you can kind of broaden or narrow the focus on the piece of content to get greater insight from it,” Clampet explained. By shrinking the context window, it could use a newer version of GPT and it would be cheaper than an older one.

But even from a cost perspective, it’s not as expensive as before. Clampet explained that supporting the thousands of monthly questions only costs a few hundred bucks.

Looking forward, Clampet sees an opportunity to move faster with AI. “The other build took about five months,” he said. “This [next iteration] is going to take about six weeks, probably, or five weeks. And it won’t be just kind of like a one to 1.5 change. It’ll be more like a one to three. It will cost less to develop and the output will be much more compelling.”

As the media industry grapples with AI, Skift’s early adoption with Ask Skift gives it a unique offering to its readers. While the team needs to make more readers aware of it, it becomes a tool to acquire and engage subscribers. And while Skift has not yet been negatively impacted by a drop in traffic due to AI (Clampet said it hasn’t been as dependent on search for a few years now), having many options in your toolbox to engage the audience will become critical for all publishers as time goes on.

But the important thing to understand about an AI tool like this is it’s only as good as its training. Skift was intentional to only use its own reporting and public SEC filings. As long as you control the inputs, the expected outputs should be stronger.