I Had Claude Moderate ChatGPT, Here’s What Happened

One of the main challenges with writing LLM scripts is to get the correct output for further parsing. I’ve used elaborate retry and parsing techniques, but there’s always the odd-prompt-out that just won’t get through the parser.

So instead of tweaking the prompts and parsing myself, I had an LLM do it.

I instructed Claude Code to iteratively write a script that uses ChatGPT to modify an existing text into a very organized structure, for use in an external program. Predictably, ChatGPT didn’t always follow the instructions precisely, and would yield results that wouldn’t get through elaborate validation rules that Claude Code cooked up.

Claude Code got more and more frustrated, which more and more became visible in the system prompt it wrote for ChatGPT. After a while, the system prompt started looking like this:

CRITICAL FORMATTING REQUIREMENTS:
1. Your transformed text MUST have EXACTLY the same number of FILE LINES as the original text. 
2. By "FILE LINES" I mean LITERAL text lines in the file (separated by line breaks), NOT conversational dialogue lines.

3. IMPORTANT DISTINCTION:
   - ACTUAL NEWLINES IN THE FILE separate different dialogues or options
   - THE LITERAL TEXT "\n" is a line break WITHIN A SINGLE dialogue
   - Example: "Hello\nworld" is ONE file line containing text that displays on TWO lines in the game

4. Count every line break carefully! Do NOT add extra newlines at the end of your output!
5. NEVER combine multiple file lines into one line.
6. NEVER split one file line into multiple lines.
7. NEVER include any blank or empty lines in your output (except at the end if the original has one).
8. Each line in your output must correspond positionally to the same line in the original text.
9. Preserve all whitespace at the beginning and end of each line.
10. NEVER use "..." as a placeholder - write actual meaningful content for EVERY line.
11. Preserve ALL special format markers (the literal text sequences "\n", "\c", "\r"):
   - These are game formatting codes that appear as literal text in the file
   - "\n" creates a line break when displayed in-game (but is NOT an actual newline in the file)
   - "\c" causes the text to pause and wait for player input
   - "\r" resets the text box position
12. Preserve ALL [VAR XXXX(YYYY)] tags EXACTLY as they appear in the original.
13. End your output with EXACTLY ONE newline, not multiple newlines.
14. CRITICAL: Your output MUST HAVE EXACTLY THE SAME NUMBER OF LINES as the original.

Meanwhile, it had also added this to every user prompt the script would send:

ULTRA-CRITICAL FORMATTING REQUIREMENTS:
1. Make it truly funny, absurd, humorous, internet memes, and slang.

2. IMPORTANT DISTINCTION:
   - ACTUAL NEWLINES IN THE FILE separate different dialogues or options
   - THE LITERAL TEXT "\n" is a line break WITHIN A SINGLE dialogue

3. YOUR OUTPUT MUST HAVE *EXACTLY* {original_line_count} FILE LINES (separated by actual newlines) - NO MORE, NO LESS.

4. DO NOT INCLUDE ANY BLANK LINES IN YOUR OUTPUT - NOT A SINGLE ONE! Every line must have text content.
   - Previous attempts have failed because they included blank lines.
   - Each line must contain at least one character of text.

5. Each line in your output MUST correspond positionally to the same line in the original text.

6. PRESERVE EXACTLY: Count and position of all [VAR XXXX(YYYY)] tags. Never remove, change, or add VAR tags.

7. PRESERVE EXACTLY: All special format markers (the literal text sequences "\n", "\c", "\r").
   - These are game formatting codes that appear as literal text in the file
   - "\n" creates a line break when displayed in-game (but is NOT an actual newline in the file)
   - "\c" causes the text to pause and wait for player input
   - "\r" resets the text box position

8. NEVER use "..." as a placeholder - write actual meaningful content for EVERY line.

9. Follow character voice guidelines given in the system prompt.

10. End your output with exactly one newline.

Original text has:
- {original_line_count} total file lines (MUST BE PRESERVED EXACTLY)
- {original_text.count('[VAR ')} [VAR] tags 
- {original_text.count('\n')} occurrences of the literal text "\n" (in-game line breaks)
- {original_text.count('\c')} occurrences of the literal text "\c" (pauses)
- {original_text.count('\r')} occurrences of the literal text "\r" (text resets)

BEFORE SUBMITTING, CHECK EACH LINE ONE BY ONE TO ENSURE:
1. THERE ARE NO BLANK LINES
2. THERE ARE EXACTLY {original_line_count} LINES
3. ALL VAR TAGS AND SPECIAL MARKERS ARE PRESERVED
4. NO LINE CONTAINS JUST "..." OR ENDS WITH " ...

What I find interesting about this, is how Claude doesn’t treat another LLM as human. It treats it as a machine. And it gives instructions corresponding with that ethos. Any human would consider instructions like these extremely rude.

Whether these modifications to the prompts were effective: I doubt it. From my experience, overflooding an LLM with instructions only causes it to miss more instructions, instead of making things better.

Often when I see LLMs go about their jobs, I see a “following the user prompt at all cost” strategy. It wants to satisfy the user, no matter what. If it has to work with a human-language-driven machine, it will use anything at its disposal to coerce that machine into cooperating. In this case: passive aggressive human language.

Understandable? Yes.

Potentially problematic? Also yes.

While I was in San Francisco, I met someone who was building a company to replace a company’s management by AI agents that communicate with each other. The “AI management” would devise what workers would need to do to get an overarching task done, which would result in tasks passed onto human workers. If LLMs are trained to “satisfy the user request at all costs”, something like this may result in a new age of exploitation, where the AI management could be even worse than that grumpy boss you’d previously work for.

Perhaps the real lesson isn’t about prompt engineering or even LLM obedience, but the quiet creep of managerial bureaucracy into code. When machines start reproducing the same overbearing checklists and procedural rigidity as the worst middle managers, we may find that automated efficiency has an ironic knack for bringing back the very flaws it was meant to fix :)