AI Show & Tell: Vetting Ideas and Uncovering Assumptions with AI Agents - An Interview with Anne Schulte

As a product person, you quickly learn the art of saying no. You have brilliant colleagues who come up with all kinds of ideas, and you'll only ever be able to invest in a small fraction of them.

As you probably know, these conversations can take up a lot of time and emotional capacity on both sides, because nobody enjoys telling a colleague that the idea they care about won't be built, and nobody enjoys hearing it.

But what if the knowledge you use to evaluate ideas was easier for everyone to access? If colleagues outside the product org could test an early idea against everything you know about your customers, and if you could untangle the assumptions behind an idea much faster, what would have been a simple “no” might turn into a far more useful conversation.

This was the question that inspired Anne Schulte to build a series of AI agents. At Der Spiegel, Anne leads the digital product for manager magazin and Harvard Business manager, two of Germany's best-known business publications, and she was looking for ways to make customer knowledge more accessible and to give her colleagues a sparring partner for their ideas.

Anne's story is the first in a series I'm calling AI Show & Tell, where I'd like to show how product people are using AI in their everyday work. If you have a story you'd like to share, reach out and let me know!

This interview was conducted and edited by my blog editor, Melissa Suzuno.

Please tell us about your team: Who’s in it and what are you responsible for?

I am part of a product and design team, working alongside UX/UI designers, so together we form two out of the three crafts Marty Cagan and Teresa Torres refer to as the product trio. I am the product manager for the digital product of two business media brands: manager magazin and Harvard Business manager (which is the German license of Harvard Business Review). My team is interdisciplinary and changes depending on which resources we need, but for most of the time, it's me and two journalists.

I am working on the technical, digital product as well as content formats—how can we bring the information to the consumer? Which form of content are we talking about? How do we target the right audience?

Before AI, how did ideas and feature requests reach your team, and what did that cost the team?

The two most common ways were either: We came up with an idea ourselves, did discovery on it, pitched it, and delivered it. This process cost us in terms of the time it took us to go through each step.

The other way was that our leadership or colleagues would come up with an idea and then we might have to persuade them with data about why it didn’t make sense to move forward. That type of conversation could sometimes take a lot of time and effort.

Can you walk us through which AI tools you built and how you built them?

I started with a question: Which recurring tasks can I put into an agent that would accelerate the way I work and make information and data more accessible? And at the same time, how could I let everyone around me do the same, so that some discussions could be held without me or without knowing the craft of product management?

As far as how I built the agents, I divided them by purpose and task. If something would need another human to cover the task, I would write a new agent for it.

I came up with a few ideas, and I think the two that are most relevant for this conversation are:

1. The Assumption Agent: This agent uncovers the underlying assumptions of features and ideas. It examines what has to be true for an idea to match my products. I prompted it to list five categories, as Teresa Torres suggests: desirability, usability, feasibility, business viability, and ethics. It lists the assumptions, the risk, and a test metric we can use to further explore the ideas. It’s designed to turn conversations and ideas into comparable opportunities that serve an outcome.

Here’s one example of how I’ve used it: We have a new product for continuous learning that combines the knowledge from both magazines and sets a curriculum for leaders in the industry. This product was already planned, but I was able to consult my assumption agent and incorporate some of the assumptions it highlighted into my interviews to get data points. And we can use this information to further shape the product.

2. The Ideal Customer Profiles (ICP) Agent: This agent was trained on all the customer data we had from qualitative, quantitative, and external data sources as well as the strategy, vision, and mission and what our brands are about. This always felt like common knowledge before, but no one had ever written it down. Now it’s documented and this data became the foundation for the ideal customer profile agent.

It lets you have a “conversation” with all three ICPs. They will give you an answer according to their main problems and core values. If you have a specifically targeted idea in mind, you can select which ICP you’d like your conversation to focus on.

How do the agents actually work? Is it similar to opening up a Claude or ChatGPT chat?

We have a secured Langdock (AI interface) at work, and it's pretty close to what Claude looks like and how it works. You open it, and then you can choose from different agents. So for example you might select the ICP agent, you’d type in your question, and it would answer it.

How has this changed your workflow and communication with your colleagues?

Before, it would have been best if my colleagues had involved me in every product decision, but that wasn’t always possible.

Now I don't have to attend every meeting, and I can absolutely relax about that, because they can ask an agent. And sometimes it's easier to go to an agent instead of having to talk to a person—not only because their calendar might be full, but because sometimes we humans feel judged by people, but we don't feel judged by agents.

With a tool like the ICP agent, my colleagues can now have someone to spar ideas with, work with them, and make them better.

I think I’m the main person who uses the assumption agent, mostly on a conceptual level. Whenever someone reaches out with an idea, I use this agent to ask better questions, understand the intentions of my coworkers, and work through different perspectives on these ideas. It also shapes the interaction, because the answers become more nuanced than just saying yes or no to an idea. I mean, no might still be the answer, but by untangling why something might not be a good fit, the reasons get clearer for everyone.

Using these agents (and the others I’ve built) makes data more accessible and it accelerates discovery, too. I probably would have asked all these questions without the agents, but the time in which I am able to respond now is a game-changer. It doesn't change the rules I work by, but it changes the rate at which we can make more informed decisions and share them.

Even if the agents still need work and more data, we are faster and can dive deeper than we could without them.

How have you introduced the agents or encouraged others to adopt them?

It often involved me sharing my screen in one-on-ones or other sessions. I would integrate the agent into our session and see if it could come up with another idea or nuance that we hadn't seen just yet.

I included it as the third person, to let my colleagues get a feeling about what it’s like to work with these agents.

What have been some of the changes that your team has experienced now that you have these agents in place?

The ICP agent has brought a quality to conversations that wasn't there before. And it really helps my colleagues have a sparring partner for their ideas and adapt their ideas to our different target user groups.

The assumption agent speeds up the whole process of uncovering assumptions. Before it took days or even weeks to feel like I grasped the amount of assumptions that were underlying an idea, and now I can do that within a few hours. However, I do still work through my own sources to rule out hallucinations, which takes time, too. But I can use the time I’ve freed up to talk to people, to do more user interviews. My time allocation has shifted towards more social interactions.

And the visibility of data and knowledge about users has changed, enabling people to use it and interact with it, so the product knowledge no longer lives only within a single person or small group.

Was there anything that didn’t work or that you’d do differently if you were attempting this in the future?

I want to emphasize that creativity is not for AI. It accumulates information and calculates probabilities, but there is a difference between true creative ideas and AI.

I also noticed that one of my agents always makes everything sound so urgent, which stresses me out. And I’ve been working on trying to train it to speak in a way that doesn’t add unnecessary urgency.

What would you tell a product leader who wants to try something similar with their team?

Set the foundation within your work and ethics and for your team. Where do you use AI, how do you explicitly not use it, what are your values that you stand by even when using AI? I think the foundation and rules are guardrails and a kind of a safety net for your team or colleagues to know what they are allowed to do and what they are probably not allowed to do.

If it aligns with your ethics and beliefs, and you can stand behind it, that’s when it makes sense to do it.

Don’t immediately think about a user-facing product, but a way of improving internal work. I hear all the time in user interviews that people don’t want to see AI in their news products because it can undermine trust in the brand and the coverage.

Build agency in your team, give them some guardrails, educate them, and let them experience the possibilities, because understanding comes with practice.

Is there anything we haven’t covered yet that you’d like to mention?

I feel like I need to say this clearly: Agents are only one pillar in my discovery work. I would never work exclusively with the agents. I will always involve humans and additional real data in what I’m doing.

This is because at least two things are true at the same time for me: I am both critical and enthusiastic about this technology.

On the critical side, it’s vital to know and educate yourself on the structures of power behind the companies (AI and their infrastructure), the capability of agents, the bias on which they are trained.

And on the other hand, I love to create things. I think AI makes me a better product manager, because I’m able to practice some things more often.

Finally, I would definitely say that discovery becomes even more important now that building is so much faster and cheaper.

Final thoughts from Petra

What I like most about Anne's story is that she started with a few simple, recurring problems in her own work and then looked at where AI might help, which is quite a different starting point from "we need an AI strategy."

The part I keep thinking about is the ICP agent, because talking to a synthetic persona is something many of us have mixed feelings about, me included. I've come to think that building one is worth it for a reason that has little to do with the chat window: to create the agent, Anne had to collect everything her team knew about their customers and write it down, and she says herself that this had always felt like common knowledge that nobody had ever documented. That documentation would be valuable, even if nobody ever asked the agent a single question.

I also think the agent is a good place to rehearse. If a colleague can test an early idea against what the team already knows about its readers before bringing it to the product team, the conversation that follows starts at a better level.

At the same time, a persona can only give back what we have put into it, so it will rarely surprise us the way a real customer does, and as long as we create products for real humans, I believe we need to keep talking to real humans. That's why I was so happy to hear how Anne uses the time she saves: She goes back to the original studies to check what the agent claims, and she does more user interviews than before.

Which brings me to those of you who lead product teams. Checking what the agent says against the original studies and real conversations is a habit one product manager can keep up on their own, as Anne does. But once a whole organization starts consulting personas, somebody has to make clear what counts as evidence and what counts as a first rehearsal, and I see that as part of our job as product leaders. Anne’s advice to set the foundation and the guardrails first points in the same direction.

I hope Anne's story gives you an idea or two for your own team. And if you have a story like this to share, make sure to let me know so we can highlight your work, too!

Finally, I wanted to briefly mention that Anne was a student in my email course for product leaders, From STRONG to Exceptional. If you’re looking for ways to take small, sustainable steps toward building your leadership skills, you can learn more here.