In early 2000s in Slashdot comments (and perhaps even earlier on Usenet), there was a checklist that was often posted in response to any articles presenting a possible solution to fighting spam.

The equivalent in the 2020s is perhaps the article decrying that LLMs are useless for writing software. This list is an intentionally snarky update to the spam checklist which focuses on the mistakes that those articles make time after time.

# AI Article Checklist

Your article critiquing using AI tools for software development is:

[ ] ill-informed [ ] outdated [ ] biased [ ] shortsighted

and fails to persuade because of the following reasons. (One or more of the following may apply to
your particular critique, and it may have other flaws that are specific to different LLMs or
development contexts.)

- [ ] Relies on ChatGPT rather than exploring alternative, better-suited LLMs
- [ ] Misrepresents LLM capabilities by using single-shot examples
- [ ] Neglects the benefits of prompt iteration and refinement
- [ ] Disregards the value of experimenting with diverse prompt styles
- [ ] Overlooks LLMs as code review or debugging assistants
- [ ] Rephrases a problem in a way that removes all context leading to bad results
- [ ] Uses prompt techniques that intentionally hamper the LLM algorithm (like "do not provide
- [ ] Ignores the potential of ReAct, chain of thought, or other prompt-enhancing techniques
- [ ] Omits crucial experiment details, hindering reproducibility
- [ ] Unfairly demands second-order explanations for first-order errors
- [ ] Fails to recognize that LLMs can benefit from standard software development practices
  (testing, specification, etc.)
- [ ] Underestimates LLMs' potential for generating pseudocode or high-level design outlines
- [ ] Dismisses LLMs as valuable documentation or tutorial creators
- [ ] Overlooks LLMs' potential in brainstorming, creative problem-solving, or idea generation
- [ ] Underappreciates the continuous improvement of LLMs with more training data and fine-tuning
- [ ] Ignores the value of LLMs for assisting novices or non-programmers in understanding code or
  creating simple scripts
- [ ] Obsesses over LLM-generated executable code, ignoring broader software development tasks where
  LLMs could shine

Additionally, your critique may face these philosophical objections:

- [ ] Similar critiques are easy to make, yet none have conclusively refuted the value of LLMs for
  software development
- [ ] Focusing on current limitations ignores the rapid advancements in LLM capabilities
- [ ] LLMs should be seen as complementary tools, not as replacements for human developers
- [ ] Collaboration between LLMs and humans can lead to innovative solutions and improved software
- [ ] LLMs can democratize access to programming knowledge and resources for a wider audience

Finally, this is what I think about your critique:

- [ ] Sorry, but your critique doesn't hold up against the potential benefits of LLMs in software
- [ ] This is a misguided critique, and you're missing the bigger picture.
- [ ] Nice try, but you're not going to derail the progress of LLMs in revolutionizing software
- [ ] This article could have been written by an AI tool.

Creative Commons License This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.