Contents

• Prompt Engineering - A Survey (Mar 2023)
• Chain of Hindsight Aligns Language Models with Feedback (Mar 2023)
• Natural Language Decomposition and Interpretation of Complex Utterances (May 2023, added 5/23/23)
• UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation (Mar 2023)
• The Power of Scale for Parameter-Efficient Prompt Tuning (Sep 2021)
• Boosted Prompt Ensembles for Large Language Models (Apr 2023)
• Learning to Compress Prompts with Gist Tokens (Apr 2023)
• Ensemble Refinement as a better prompting strategy for complex problem-solving (May 2023, added 5/18/23)
• Text Classification via Large Language Models (May 2023, added 5/18/23)

Prompt Engineering - A Survey (Mar 2023)

(link)

This is the best overall overview article of all methods related to inferencing with LLMs. Hits on everything from chain-of-thought prompting to the Toolformer paper (using external APIs) to producing Python code to answer a question.

Chain of Hindsight Aligns Language Models with Feedback (Mar 2023)

(link)

Paper proposes a novel technique, Chain of Hindsight, that is easy to optimize and can learn from any form of feedback. We convert all types of feedback into sentences, which are then used to fine-tune the model, allowing us to take advantage of the language comprehension capabilities of language models.

• Simple idea: generate 4 outputs A, B, C, D for a particular question from an LLM.
• A human rater ranks the 4 outputs and tells the model that: “A is better than B, which is about as good as C, and both are better than D”.