Ask AI Right · part 6
[Ask AI Right] The Art of Follow-Up Questions — What to Do When the First Answer Is Too Shallow
❯ cat --toc
- Plain-Language Version: Why the first answer is never the best one
- Preface
- Why the first answer is almost always too shallow
- Technique 1: Add constraints
- Technique 2: Ask for comparisons
- Technique 3: Ask AI to ask you questions
- Technique 4: Challenge the answer
- Technique 5: Iterate — the first answer is a rough draft
- Before and after: the full difference
- Someone planning their first vegetable garden
- When to stop following up
- Try this today
- One sentence
TL;DR
The first answer AI gives you is a rough draft. The quality lives in the follow-ups: add constraints, ask for comparisons, challenge the answer, and — most powerful of all — ask AI what information it needs from you. Two to four follow-up messages routinely turn a generic response into something genuinely useful.
Plain-Language Version: Why the first answer is never the best one
When you ask AI a question and get a response, that response is based on guessing what you probably mean. AI doesn't know your situation, your skill level, your constraints, or what you've already tried. It gives you the safest, most general answer it can — which usually means a shallow one.
According to research from Wharton professor Ethan Mollick, the biggest gap between casual AI users and power users isn't technical skill — it's the willingness to have a conversation instead of expecting one perfect answer. The people who get the most from AI treat it like a back-and-forth discussion, not a search engine.
This article teaches you five specific follow-up techniques. None of them require any technical knowledge. All of them work with ChatGPT, Claude, Gemini, or any other AI assistant you use.
Preface
Imagine you walk into a restaurant and tell the waiter: "Bring me something good." You'll get a dish that's fine — probably their most popular item. But it might not match your mood, your dietary needs, or what you're craving.
Now imagine you say: "I'm in the mood for something spicy, not too heavy, and I don't eat shellfish." Completely different result. Same kitchen, same chef — but a better order because you gave them something to work with.
AI works the same way. The first answer is "something good." The follow-ups are where you tell it what you actually want.
In the last article, we talked about using AI as a research assistant to find existing tools. This one is about a more fundamental skill: the back-and-forth. How to take any AI answer — whether it's research, advice, writing help, or problem-solving — and make it dramatically better with a few follow-up messages.
Why the first answer is almost always too shallow
This isn't AI being lazy. It's AI being cautious.
When you ask "How do I start a small business?" — AI doesn't know if you're a college student with $500, a retired professional with $50,000, if you're in Taipei or Toronto, if you want an online store or a physical shop, or if you've already tried and failed once.
So it gives you the answer that works for the most people: a safe, broad overview. Something like "First, identify your target market. Then write a business plan. Register your business..."
That's not wrong. It's just not useful to you specifically. And the only way AI can give you a better answer is if you tell it more — through follow-up questions.
Here are five techniques that work every time.
Technique 1: Add constraints
The simplest and most effective follow-up. Tell AI what the answer should look like.
Without constraints:
"How do I learn photography?" → Gets a 10-paragraph essay covering cameras, composition, lighting, editing, online courses, and practice tips. Technically correct, practically overwhelming.
With constraints:
"I have a smartphone, no budget, and 30 minutes a day. Give me a 2-week beginner plan for learning photography — just the first steps, nothing about gear." → Gets a focused, day-by-day plan using only a phone. Immediately actionable.
Here are constraint types you can mix and match:
- Length: "in 300 words or less" / "in 3 bullet points"
- Audience: "explain it to a 10-year-old" / "assume I'm a complete beginner"
- Scope: "only talk about [specific thing]" / "skip the basics, I already know those"
- Format: "give me a comparison table" / "just the pros and cons" / "as a checklist"
- Context: "I'm in Taiwan" / "I work from home" / "my budget is $0"
You don't need all of them. Even one constraint transforms the answer.
Technique 2: Ask for comparisons
When AI recommends something, don't just accept it. Ask it to compare.
Without comparison:
"What's a good note-taking app?" → "Notion is a popular choice. It offers flexible databases, templates, and collaboration features."
That tells you nothing about whether Notion is right for you. So follow up:
With comparison:
"Compare Notion, Apple Notes, and Obsidian for a freelancer who just wants quick notes on their phone. Pros, cons, and who each one is best for."
Now you get a decision table. You see that Apple Notes wins on speed and simplicity, Notion wins on structure, and Obsidian wins on privacy and ownership. You pick based on what matters to you — not based on what's popular.
Pro tip: You can also ask for comparisons between approaches, not just tools:
"I want to improve my English. Compare taking an online course vs using a language exchange app vs practicing with AI. Which gets results fastest for someone who can spend 20 minutes a day?"
Technique 3: Ask AI to ask you questions
This is the single most underused technique, and it's the most powerful.
Instead of trying to anticipate what AI needs to know, just ask:
"I want to plan a weekend trip. Before you give me suggestions — what information do you need from me to give a really good answer?"
AI will come back with something like:
"To give you better suggestions, I'd like to know:
- Where are you located?
- How far are you willing to travel?
- What's your budget?
- Are you traveling solo, with a partner, or with kids?
- Do you prefer nature, cities, or cultural experiences?
- Any dietary restrictions or accessibility needs?"
Answer those questions, and the next recommendation will be dramatically more specific than anything you'd have gotten from a cold start.
This works for almost anything:
- "I want to change careers. What do you need to know about me to give useful advice?"
- "I need to write a cover letter. Before you draft one, what should you know about the job and my experience?"
- "I want to pick a laptop. Ask me questions first so your recommendation actually fits."
You're essentially saying: I don't know what I don't know — help me figure out what matters.
Technique 4: Challenge the answer
AI tends to be agreeable. It gives you a solution and doesn't mention the downsides unless you ask. So ask.
After getting any recommendation or advice, try:
"What are the downsides of this approach?"
"What could go wrong if I follow this advice?"
"Is there a counter-argument or a different school of thought?"
"What would someone who disagrees with you say?"
Example:
First message:
"Should I learn Python or JavaScript first?"
AI says Python — easier syntax, great for beginners, versatile.
Challenge:
"What would someone who recommends JavaScript over Python say? What are Python's weaknesses for a beginner?"
Now you get the other side: JavaScript is more immediately visual (you see results in a browser), has a bigger job market in some regions, and Python's simplicity can actually hide important concepts you'll need later.
With both sides, you make a better decision than with either alone.
Technique 5: Iterate — the first answer is a rough draft
Think of your first message as a rough draft of your question, and AI's first answer as a rough draft of the response. Neither is the final version.
Here's what iteration looks like in practice:
Round 1:
"Help me write an email to my landlord about a broken heater."
AI writes a polite, generic email.
Round 2:
"Make it shorter — just 3 sentences. And make the tone more direct. The heater has been broken for two weeks and I've already asked once."
AI rewrites: shorter, more assertive, mentions the timeline.
Round 3:
"Good, but add that I'd like a response by Friday, and mention that the lease requires working heating."
AI adds the deadline and the lease reference.
Three rounds. Each one took 15 seconds to write. The final email is exactly right — specific to your situation, in your tone, with the right details. The first draft would have been forgettable.
This applies to everything: meal plans, travel itineraries, project proposals, study guides. The pattern is always the same — start broad, then narrow with each round.
Before and after: the full difference
Let's see the complete contrast — the same need, handled two different ways.
Someone planning their first vegetable garden
The one-message approach:
"How do I start a vegetable garden?"
AI gives a 800-word overview: choose a sunny spot, test your soil, decide between raised beds and ground planting, here are 15 vegetables to consider, watering schedule, composting basics...
Technically helpful. Practically overwhelming. You close the tab.
The follow-up approach:
"How do I start a vegetable garden?"
(Gets the same overview)
"I live in an apartment with a balcony that gets about 4 hours of sun. I've never grown anything. What are the 3 easiest things I can grow in containers?"
(Gets: cherry tomatoes, basil, lettuce. With container size and specific instructions.)
"I'm in a subtropical climate — does that change anything?"
(Gets adjusted advice: lettuce prefers cooler months, add sweet potato leaves as a better alternative.)
"What are the most common beginner mistakes for balcony gardening?"
(Gets: overwatering, too-small containers, not enough drainage holes. Specific to your situation.)
Four messages. Each one took seconds. The result is a personalized, actionable plan instead of a generic encyclopedia entry.
When to stop following up
Not every question needs four rounds. Here's a quick guide:
One message is enough when:
- You need a simple fact ("What year was [movie] released?")
- You need a quick format conversion ("Convert this to bullet points")
- The first answer nails it
Follow up when:
- The answer is too generic (add constraints)
- You're making a decision (ask for comparison)
- You're not sure what to ask (let AI ask you)
- The answer sounds too good (challenge it)
- The answer is close but not quite (iterate)
Try this today
Pick something you've been meaning to ask AI about — anything at all. Ask your first question as you normally would. Then, before accepting the answer, try just one of these follow-ups:
- Add one constraint: "Now assume I'm a beginner with no budget."
- Ask for a comparison: "How does this compare to [alternative]?"
- Flip the conversation: "What do you need to know about me to give a better answer?"
- Challenge it: "What could go wrong with this approach?"
- Iterate: "Good start. Now make it shorter/more specific/more practical."
You'll be surprised how much better the second answer is.
One sentence
The first answer AI gives you is a rough draft — the quality you're looking for lives in the follow-up questions you ask next.
Next up: AI is incredibly useful, but it has real limits. How do you know when to trust an answer and when to double-check? We'll talk about hallucinations, outdated information, and knowing AI's boundaries.
This is Part 6 of the "Ask AI Right" series. Previous: Before You Build It, Ask: Does This Already Exist?.
FAQ
- How do I get better answers from ChatGPT or Claude?
- Don't accept the first answer — treat it as a rough draft. Follow up with constraints ('explain in 500 words or less'), ask for comparisons ('compare A and B with pros and cons'), or ask AI what it needs from you ('what information would help you give me a better answer?'). Most people stop after one message. The quality lives in the follow-ups.
- What should I do when AI gives me a generic answer?
- Add constraints to force specificity. Tell AI your experience level ('assume I'm a complete beginner'), your context ('I'm a freelance designer in Taiwan'), your format preference ('give me a comparison table, not paragraphs'), or your limits ('only free tools'). Generic questions get generic answers — specific follow-ups get useful ones.
- Can I ask AI to ask me questions?
- Yes, and this is one of the most powerful techniques. Say: 'What information do you need from me to give a better answer?' AI will ask about your budget, timeline, experience level, specific use case — things you forgot to mention. This turns a one-way Q&A into a real conversation.
- How many follow-up questions should I ask AI?
- There's no fixed number, but 2-4 follow-ups usually transform a mediocre answer into a genuinely useful one. Think of it like a conversation with a knowledgeable friend: the first question opens the topic, and each follow-up narrows it to exactly what you need.