Prompt Engineering Tips and Tricks
Prompt engineering is the art of shaping inputs so that large language models (LLMs) produce exactly the output you need. It feels a bit like writing a recipe: the right ingredients, clear steps, and a pinch of intuition can turn a vague request into a precise, reliable result. In this guide we’ll walk through practical techniques, real‑world examples, and a few pro tips that will make your prompts work like a charm.
Know Your Model’s Strengths and Limits
Before you start tweaking wording, spend a few minutes learning what the model you’re using excels at. GPT‑4, for instance, shines with nuanced reasoning, while smaller models may be better suited for fast, token‑efficient tasks. Understanding token limits, context windows, and the model’s training cut‑off date helps you avoid asking for impossible knowledge.
Keep in mind that LLMs are statistical predictors, not databases. They can synthesize information, but they may hallucinate details that sound plausible. This awareness will guide you to add verification steps later in the workflow.
Key characteristics to check
- Context window: How many tokens can the model retain at once?
- Temperature: Controls randomness; lower values yield deterministic output.
- Stop sequences: Useful for trimming unwanted trailing text.
Write Clear, Specific Instructions
A common mistake is to ask the model something too broad, like “Explain recursion.” Instead, frame it with constraints: “Explain recursion in Python using a maximum of three lines of code and include a brief comment for each line.” This tells the model exactly what format and length you expect.
Use explicit verbs—list, compare, summarize, generate—to guide the model’s behavior. When you want a list, start the prompt with “Provide a bullet‑point list of…”. When you need a step‑by‑step guide, say “Outline the steps to…”.
Example: From vague to precise
- Vague: “Tell me about REST APIs.”
- Precise: “List three key principles of RESTful API design, each with a one‑sentence example in JavaScript.”
Leverage System Prompts for Role‑Playing
System messages let you set the model’s persona or behavior for the entire conversation. For instance, you can ask the model to act as a senior Python instructor, a friendly customer support agent, or a meticulous data analyst. This context sticks throughout the session, reducing the need to repeat instructions.
When you combine a system prompt with a user prompt, you get a powerful “role + task” combo that often yields higher quality output.
import openai
client = openai.ChatCompletion.create(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a senior Python instructor who explains concepts with concise code examples."},
{"role": "user", "content": "Show me how to read a CSV file using pandas and print the first five rows."}
],
temperature=0.2
)
print(client.choices[0].message.content)
Pro tip: Keep the system prompt under 150 tokens. Too much detail can dilute the core persona you want the model to adopt.
Few‑Shot Prompting: Show, Don’t Just Tell
Few‑shot prompting involves providing a few examples of the desired input–output mapping before the actual request. This is especially useful for tasks like classification, transformation, or formatting where the model benefits from concrete patterns.
Structure the examples clearly, using delimiters or JSON to separate them. The model will infer the pattern and apply it to the new query.
Python example: Converting natural language dates to ISO format
prompt = """
Convert the following human‑readable dates to ISO‑8601 (YYYY‑MM‑DD) format.
Input: March 5th, 2023
Output: 2023-03-05
Input: 12/31/2021
Output: 2021-12-31
Input: next Friday
Output:
"""
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
temperature=0
)
print(response.choices[0].message.content.strip())
Notice how the two examples set a clear pattern, and the model completes the third one correctly.
Control Randomness with Temperature and Top‑P
Temperature influences how “creative” the model is. A temperature of 0 makes the output deterministic, which is ideal for code generation or data extraction. Higher values (0.7‑1.0) encourage diversity, useful for brainstorming or creative writing.
Top‑p (nucleus sampling) works alongside temperature, truncating the probability distribution to the most likely tokens that sum to p. For most engineering tasks, keep top‑p at 1.0 and adjust temperature only.
When to tweak these settings
- Code generation: temperature = 0, top‑p = 1.
- Idea brainstorming: temperature = 0.8‑1.0, top‑p = 0.9.
- Structured data extraction: temperature = 0, use stop sequences to cut off extra text.
Use Stop Sequences and Token Limits for Clean Output
Stop sequences tell the model where to halt generation. They’re invaluable when you need a single answer without trailing explanations. Common stop tokens include “\n\n”, “---”, or a custom marker like “
Combine stop sequences with a max token limit to keep responses concise. For example, when extracting a JSON snippet, set a stop token of “}” and a max token count that comfortably fits the expected object.
Code snippet: Extracting JSON with stop token
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": "Extract the product name and price from the text and return JSON."}],
temperature=0,
max_tokens=150,
stop=["}"]
)
json_text = response.choices[0].message.content + "}"
print(json_text)
Handle Ambiguity with Clarifying Questions
If the user’s request is vague, have the model ask a follow‑up question instead of guessing. This two‑step approach reduces errors and improves user satisfaction.
Implement this by checking for missing slots in your prompt template and prompting the model to request the missing information.
Template with placeholders
template = """
You are a helpful travel assistant.
User wants to plan a trip. Gather missing details before providing an itinerary.
Required details:
- Destination city
- Travel dates
- Budget range
If any detail is missing, ask the user for it. Otherwise, output a concise 3‑day itinerary in markdown.
"""
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "system", "content": template},
{"role": "user", "content": "I want a weekend trip to Paris."}],
temperature=0.5
)
print(response.choices[0].message.content)
Debugging Prompts: Iterative Refinement
When a prompt doesn’t behave as expected, adopt a systematic debugging routine:
- Isolate the problem: Reduce the prompt to the minimal failing example.
- Check token limits: Ensure the context isn’t being truncated.
- Experiment with temperature: Lower it to see if randomness is the culprit.
- Add explicit instructions: Sometimes a single word like “Only” or “Do not” changes the outcome.
Document each iteration in a notebook. Over time you’ll build a library of “prompt recipes” that you can reuse.
Pro tip: Use the “logprobs” feature (if available) to see which tokens the model is most confident about. Low confidence often signals ambiguity in your prompt.
Real‑World Use Cases
1. Customer Support Automation
Prompt the model to act as a first‑line support agent that classifies tickets and suggests a solution. Combine a system prompt with a few‑shot example of ticket → category mapping.
system = "You are a customer‑support triage bot. Classify tickets into one of: Billing, Technical, General."
examples = """
Ticket: My invoice shows an extra $10 charge.
Category: Billing
Ticket: The app crashes when I upload a photo.
Category: Technical
"""
user_ticket = "I can't reset my password."
prompt = f"{examples}\nTicket: {user_ticket}\nCategory:"
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "system", "content": system},
{"role": "user", "content": prompt}],
temperature=0
)
print(response.choices[0].message.content.strip())
This approach yields consistent categorization without hard‑coding rules.
2. Code Generation & Review
When generating code snippets, embed the language, constraints, and expected output format directly in the prompt. Follow up with a second prompt that asks the model to review its own output for common pitfalls.
# Step 1: Generate a function
gen_prompt = """
Write a Python function `slugify(text)` that:
- Converts the string to lowercase
- Replaces spaces with hyphens
- Removes all non‑alphanumeric characters except hyphens
Provide only the function definition without any explanation.
"""
gen = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": gen_prompt}],
temperature=0
)
code = gen.choices[0].message.content
# Step 2: Review the generated code
review_prompt = f"""
Review the following Python code for bugs, edge cases, and PEP‑8 compliance. Return a short list of issues, if any.
{code}
"""
review = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": review_prompt}],
temperature=0.3
)
print("Generated code:", code)
print("Review:", review.choices[0].message.content)
This two‑step pattern produces higher‑quality code and gives you an immediate sanity check.
3. Data Extraction from Unstructured Text
Suppose you have a batch of email bodies and need to pull out order numbers, dates, and totals. Prompt the model with a clear JSON schema and a few examples.
schema = """
Extract the following fields and output valid JSON:
{
"order_id": string,
"order_date": "YYYY-MM-DD",
"total_amount": float
}
"""
example = """
Email: "Hi, my order #A12345 placed on 2023‑07‑15 totals $89.99. Thanks!"
JSON: {"order_id":"A12345","order_date":"2023-07-15","total_amount":89.99}
"""
email = "Hello, I bought item #B9876 on 02/28/2024, amount due $45.50."
prompt = f"{schema}\n{example}\nEmail: \"{email}\"\nJSON:"
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
temperature=0,
stop=["\n"]
)
print(response.choices[0].message.content)
By defining the exact JSON shape, you reduce post‑processing effort dramatically.
Advanced Tricks for Power Users
- Chain‑of‑Thought prompting: Ask the model to “think step‑by‑step” before answering. This improves accuracy on logical puzzles.
- Self‑Consistency: Run the same prompt multiple times with low temperature and aggregate the most common answer.
- Dynamic Prompt Assembly: Build prompts programmatically based on user input, context, and previous model outputs.
Chain‑of‑Thought example
prompt = """
Solve the following math problem. Show your reasoning step by step, then give the final answer.
Problem: A train travels 150 km at 60 km/h and then 200 km at 80 km/h. What is the average speed for the whole journey?
"""
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
temperature=0.2
)
print(response.choices[0].message.content)
The model will first compute each segment’s time, then combine them, yielding a more reliable answer.
Testing and Monitoring in Production
Once your prompts are stable, integrate them into a testing pipeline. Use unit‑test‑style assertions on the model’s output (e.g., JSON schema validation, regex checks, or semantic similarity scores). Automate regression tests whenever you upgrade the model version.
Monitoring is equally important. Track metrics such as prompt success rate, average token usage, and latency. Set alerts for spikes that could indicate a regression in prompt performance.
Ethical Considerations
Prompt engineering also carries responsibility. Avoid crafting prompts that encourage the model to produce disallowed content, misinformation, or biased language. Include safety instructions in your system prompt, such as “If the request is unsafe, refuse politely.”
Regularly audit outputs for bias, especially when the model interacts with diverse user groups. Use external tools or human reviewers to catch subtle issues that the model might overlook.
Conclusion
Effective prompt engineering blends clear communication, strategic use of system messages, and iterative testing. By mastering a few core techniques—specific instructions, few‑shot examples, temperature control, and robust validation—you can unlock the full potential of LLMs for everything from customer support to code generation.
Remember that prompts are living artifacts: they evolve as models improve and as your application requirements change. Keep a habit of documenting, versioning, and revisiting your prompts, and you’ll stay ahead of the curve in this rapidly advancing field.