Prompt Engineering

Learning to speak

Foreword

Not sticking to the theme this time!

Anyways I came across this free(for now?) course earlier today and decided to give it a try. Turned out to be pretty cool. Took some notes, realised that they were already in markdown, so I’ll just store them here.

You can find a more verbose me in Prelude->2023-04-28

All information here is based on the ChatGPT Prompt Engineering for Developers short course by deeplearning.ai

Introduction

Two types of LLMs

1. Base LLM

Base LLMs are trained to predict the next most likely word to occur based on large amounts of text data fed to the model.

For example, a prompt such as:

1
Once upon a time, there was a unicorn

may be completed by the LLM to become

1
Once upon a time, there was a unicorn that lived in a magical forest with all her unicorn friends.

That sounds about right.

However, a prompt such as:

1
What is the capital of France?

may become

1
2
3
4
What is the capital of France?
What is France's largest city?
What is France's population?
What is the currency of France?

As shown above, the Base LLM predicts the next lines to also be questions related to France, possibly due to the prompt closely resembling quizzes/question lists.

2. Instruction Tuned LLM

Instruction tuned LLMs are Base LLMs that are further “finetuned” with inputs and outputs of instructions being followed.

They are trained with RLHF, with a focus on being Helpful, Honest and Harmless.

The same prompt

1
What is the capital of France?

should output

1
The capital of France is Paris.

Strong note: if your chatgpt output is useless, you’re probably not being specific enough.

Tactics

Principals of prompting

  • Write clear and specific instructions.
  • Give the model time to think.

1. Use delimiters to clearly indicate distinct parts of the input

Example prompt:

1
2
3
4
5
6
7
8
9
Summarize the text delimited by triple backticks into a single sentence.

```
You should express what you want a model to do by providing instructions that are as clear and specific as you can possibly make them.

This will guide the model towards the desired output, and reduce the chances of receiving irrelevant or incorrect responses. Don't confuse writing a clear prompt with writing a short prompt.

In many cases, longer prompts provide more clarity and context for the model, which can lead to more detailed and relevant outputs.
```

Example output:

1
To guide a model towards the desired output and reduce chances of irrelevant or incorrect responses, clear and specific instructions should be provided which may be longer to provide more clarity and context for the model.

Specifying clear delimiters can also help prevent prompt injection.

2. Ask for structured input

Example prompt:

1
2
3
Generate a list of three made-up book titles along with their authors and genres.

Provide them in JSON format with the following keys: book_id, title, author, genre.

Example output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[
  {
    "book_id": 1,
    "title": "The Midnight Garden",
    "author": "Evelyn Hart",
    "genre": "Fantasy"
  },
  {
    "book_id": 2,
    "title": "Blood in the Water",
    "author": "Alexis Lee",
    "genre": "Thriller"
  },
  {
    "book_id": 3,
    "title": "The Last Starship",
    "author": "Jackson Rivers",
    "genre": "Science Fiction"
  }
]

3. Ask model to check if conditions are satisfied

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
You will be provided with text delimited by triple quotes. If it contains a sequence of instructions, re-write those instructions in the following format:

Step 1 - ...
Step 2 - ...
...
Step N - ...

If the text does not contain a sequence of instructions, then simply write "No steps provided."

"""
Making a cup of tea is easy! First, you need to get some water boiling. While that's happening, grab a cup and put a tea bag in it. Once the water is hot enough, just pour it over the tea bag. Let it sit for a bit so the tea can steep. After a few minutes, take out the tea bag. If you like, you can add some sugar or milk to taste. And that's it! You've got yourself a delicious cup of tea to enjoy.
"""

Example output:

1
2
3
4
5
6
7
Step 1 - Get some water boiling.
Step 2 - While the water is boiling, put a tea bag in a cup.
Step 3 - Pour the hot water over the tea bag.
Step 4 - Let the tea steep for a few minutes.
Step 5 - Remove the tea bag.
Step 6 - Add sugar or milk to taste.
Step 7 - Enjoy your delicious cup of tea!

4. “Few-shot” prompting

Prior to performing our task, we give the model successful examples of task completion.

Example prompt:

1
2
3
4
5
6
7
Your task is to answer in a consistent style.

<child>: Teach me about patience.

<grandparent>: The river that carves the deepest valley flows from a modest spring; the grandest symphony originates from a single note the most intricate tapestry begins with a solitary thread.

<child>: Teach me about resilience.

Example output:

1
<grandparent>: A tree that endures the strongest winds is the one that has the deepest roots; a blade that withstands the sharpest cut is the one that has been most carefully tempered.

5. Specify the steps required to complete the task

If the model is rushing to an incorrect conclusion, we should specify a chain of reasoning that the model should take before arriving at a final answer.

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Your task is to perform the following actions:

1 - Summarize the following text delimited by <> with 1 sentence.
2 - Translate the summary into French.
3 - List each name in the French summary.
4 - Output a json object that contains the following keys: french_summary, num_names.

Use the following format:
Text: <text to summarize>
Summary: <summary>
Translation: <summary translation>
Names: <list of names in Italian summary>
Output JSON: <json with summary and num_names>

Text:
<In a charming village, siblings Jack and Jill set out on a quest to fetch water from a hilltop well. As they climbed, singing joyfully, misfortune struck—Jack tripped on a stone and tumbled down the hill, with Jill following suit. Though slightly battered, the pair returned home to comforting embraces. Despite the mishap, their adventurous spirits remained undimmed, and they continued exploring with delight.>

Example output:

1
2
3
4
5
6
7
8
9
Text: <In a charming village, siblings Jack and Jill set out on a quest to fetch water from a hilltop well. As they climbed, singing joyfully, misfortune struck—Jack tripped on a stone and tumbled down the hill, with Jill following suit. Though slightly battered, the pair returned home to comforting embraces. Despite the mishap, their adventurous spirits remained undimmed, and they continued exploring with delight.>

Summary: Jack and Jill went on a quest to fetch water but suffered a mishap, returned home slightly battered, and continued exploring with undimmed adventurous spirits.

Translation: Jack et Jill sont partis en quête d'eau, mais ont subi un accident, sont rentrés chez eux légèrement blessés et ont continué à explorer avec des esprits aventureux indomptables.

Names: Jack, Jill.

Output JSON: {"french_summary": "Jack et Jill sont partis en quête d'eau, mais ont subi un accident, sont rentrés chez eux légèrement blessés et ont continué à explorer avec des esprits aventureux indomptables.", "num_names": 2}

6. Instruct the model to work out its own solution before rushing to a conclusion

This is a useful way to force the model to spent some more processing time on the task instead of “guessing” the output.

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Your task is to determine if the student's solution is correct or not.

To solve the problem do the following:
- First, work out your own solution to the problem. 
- Then compare your solution to the student's solution and evaluate if the student's solution is correct or not.

Don't decide if the student's solution is correct until you have done the problem yourself.

Use the following format:
Question:
```
<question here>
```
Student's solution:
```
<student's solution here>
```
Actual solution:
```
<steps to work out the solution and your solution here>
```
Is the student's solution the same as actual solution just calculated:
```
<yes or no>
```
Student grade:
```
<correct or incorrect>
```

Question:
```
I'm building a solar power installation and I need help working out the financials.

- Land costs $100 / square foot
- I can buy solar panels for $250 / square foot
- I negotiated a contract for maintenance that will cost me a flat $100k per year, and an additional $10 / square foot

What is the total cost for the first year of operations as a function of the number of square feet.
``` 
Student's solution:
```
Let x be the size of the installation in square feet.
Costs:
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 100x
Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000
```
Actual solution:

Example output:

1
2
3
4
5
6
7
8
9
10
Let x be the size of the installation in square feet.

Costs:
Land cost: $100x
Solar panel cost: $250x
Maintenance cost: $100,000 + $10x per square foot = $100,000 + $10x
Total cost: $100x + $250x + $100,000 + $10x = $360x + $100,000
The student's solution is incorrect because they added the maintenance cost per square foot as a fixed cost, instead of multiplying it by x. The actual solution accounts for the maintenance cost per square foot by multiplying it by x in the calculation.

Student grade: incorrect

Note: The current model is prone to hallucination, which is spilling out made up information that sounds realistic. A trick to reducing hallucinations is to ask the model to first quote or find relevant information, then answer based on the information found. That way we can trace back the model’s output to sources which we can verify.

Iterative Prompting

Prompt engineering is an iterative process and the first prompt rarely outputs the desired result. It’s important to therefore analyze the output and ammend prompt, while sticking to the tactics mentioned above.

Some ideas include:

  • Use at most 50 words.
  • Focusing on <part of text>
  • Include <additional information>
  • Include a table with <specifications>

Honestly quite subjective so not much to take note of, apart from the fact that LLMs aren’t great at counting characters due to the nature of the tokenizer, so try to limit output by words/sentences.

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Your task is to help a marketing team create a 
description for a retail website of a product based 
on a technical fact sheet.

Write a product description based on the information 
provided in the technical specifications delimited by 
triple backticks.

The description is intended for furniture retailers, 
so should be technical in nature and focus on the 
materials the product is constructed from.

At the end of the description, include every 7-character 
Product ID in the technical specification.

After the description, include a table that gives the 
product's dimensions. The table should have two columns.
In the first column include the name of the dimension. 
In the second column include the measurements in inches only.

Give the table the title 'Product Dimensions'.

Format everything as HTML that can be used in a website. 
Place the description in a <div> element.

Technical specifications: <omitted>

Summarizing and Inferring

A great use of LLMs at their current stage is to summarize content.

Most of the tips are already mentioned above in either the Tactics section or the Iterative prompting section.

One additional tip is to try using the keyword extract instead of summarize.

Example prompt:

1
2
3
4
5
6
7
Your task is to extract relevant information from a product review from an ecommerce site to give feedback to the Shipping department. 

From the review below, delimited by triple quotes extract the information relevant to shipping and delivery.

Limit to 30 words. 

Review: <omitted>

In real life this may be used by an ecommerce business owner to build a dashboard displaying summaries of reviews. LLMs can be used to identify the lexical affects of texts.

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
13
Identify the following items from the review text: 
- Sentiment (positive or negative)
- Is the reviewer expressing anger? (true or false)
- Item purchased by reviewer
- Company that made the item

The review is delimited with triple backticks.
Format your response as a JSON object with "Sentiment", "Anger", "Item" and "Brand" as the keys.
If the information isn't present, use "unknown" as the value.
Make your response as short as possible.
Format the Anger value as a boolean.

Review text: <omitted>

Transformation

LLMs are trained on large sets of data in (hopefully)varying languages, meaning that it probably has a decent grasp of common languages.

Besides using it for sentence to sentence translation, here are some more usecases.

Tone transformation

Example prompt:

1
2
Convert the following from slang to a business letter: 
'Dude, This is Joe, check out this spec on this standing lamp.'

Example output:

1
2
3
4
5
6
7
8
9
10
11
12
Dear [Recipient],

I hope this letter finds you well. My name is Joe and I am writing to bring to your attention a special specification regarding a standing lamp.

I kindly request that you take a moment to review the specifications I have provided below:

[Insert specifications here]

Thank you for your time and consideration. Please feel free to reach out to me with any questions or concerns.

Best regards,
Joe

Format conversion

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Translate the following python dictionary from JSON to an HTML table with column headers and title:

{
  "resturant employees": [
    {
      "name": "Shyam",
      "email": "shyamjaiswal@gmail.com"
    },
    {
      "name": "Bob",
      "email": "bob32@gmail.com"
    },
    {
      "name": "Jai",
      "email": "jai87@gmail.com"
    }
  ]
}

Example output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<table>
  <caption>Restaurant Employees</caption>
  <thead>
    <tr>
      <th>Name</th>
      <th>Email</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Shyam</td>
      <td>shyamjaiswal@gmail.com</td>
    </tr>
    <tr>
      <td>Bob</td>
      <td>bob32@gmail.com</td>
    </tr>
    <tr>
      <td>Jai</td>
      <td>jai87@gmail.com</td>
    </tr>
  </tbody>
</table>

Proofreading

To signal to the LLM that you want it to proofread your text, you instruct the model to ‘proofread’ or ‘proofread and correct’.

Example prompt:

1
2
3
4
5
Proofread and correct this review. Make it more compelling. 
Ensure it follows APA style guide and targets an advanced reader. 
Output in markdown format.

Text: <omitted>

Expansion

Lastly, LLMs can be used to create/extend text given a user input.

This example tasks the LLM to reply to customer review emails based on the sentiments and details in the customer’s original email.

Example prompt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
You are a customer service AI assistant.
Your task is to send an email reply to a valued customer.

Given the customer email delimited by ```, generate a reply to thank the customer for their review.

If the sentiment is positive or neutral, thank them for their review.

If the sentiment is negative, apologize and suggest that they can reach out to customer service. 

Make sure to use specific details from the review.
Write in a concise and professional tone.
Sign the email as `AI customer agent`.

Customer review: <omitted>

Conclusion

See Prelude!

扫一扫，分享到微信

微信分享二维码

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

赐我孩童般的好奇