Proving that Cryptic Crossword Clue Answers are Correct

A cryptic clue leads to its answer only if it is read in the right way. What the clue appears to say when read normally (the surface reading) is usually a distraction with nothing to do with the solution. The challenge is to find the way of reading the clue that leads to the solution.

A typical clue consists of two parts:

• •

  The straight or definition. This is in essence the same as any non-cryptic crossword clue: a synonym for the answer. It usually exactly matches the part of speech, tense, and number of the answer, and usually appears at the start or end of a clue. For our annotations, the span that encompasses the definition is highlighted using curly braces.

• •

  The cryptic, subsidiary indication or wordplay. This gives the solver some instructions on how to get to the answer in another (less literal) way. The wordplay parts of clues can be obscure, especially to a newcomer, but they tend to utilise standard rules and conventions which become more familiar with practice.

Sometimes the two parts of the clue are joined with a link word or phrase such as ‘from’, ‘gives’ or ‘could be’. One of the tasks of the solver is to find the boundary between the definition and the wordplay, and insert a mental pause there when reading the clue cryptically.

We list below several of the important styles of wordplay that are commonly used, each with an annotated example. For a more comprehensive list, along with an outline of the ‘Ximenean principles’, please see Wikipedia (2024).

An anagram is a rearrangement of a certain section of the clue to form the answer. This is usually indicated by a codeword which indicates change, movement, breakage or something otherwise amiss. For example:

clue:       Chaperone shredded corset (6)
definition: {Chaperone} shredded corset
answer:     ESCORT
wordplay:   (corset)* (*shredded)

In a charade, the answer is formed by joining individually clued words to make a larger word (namely, the answer). For example:

clue:       Outlaw leader managing money (7)
definition: Outlaw leader {managing money}
answer:     BANKING
wordplay:   BAN (outlaw) + KING (leader)

A container or insertion clue puts one set of letters inside another. For example (also starting to add a little more indirection):

clue:       Utter nothing when there’s wickedness about (5)
definition: {utter} nothing when there’s wickedness about
answer:     VOICE
wordplay:   O (nothing) with VICE (wickedness) around it (about)

Deletion is a wordplay mechanism which removes some letters of a word to create a shorter word. For example:

clue:       Bird is cowardly, about to fly away (5)
definition: {Bird} is cowardly, about to fly away
answer:     RAVEN
wordplay:   [c]RAVEN (cowardly) - ’C’ (i.e. circa, about) (-fly away)

A clue may, rather than having a definition part and a wordplay part, have two definition parts. For example:

clue:       Not seeing window covering (5)
definition: {Not seeing} {window covering}
answer:     BLIND
wordplay:   Double Definition (DD)

With hidden word clues, the solution itself is written within the clue – either as part of a longer word or across more than one word. For example:

clue:       Found ermine, deer hides damaged (10)
definition: Found ermine, deer hides {damaged}
answer:     UNDERMINED
wordplay:   [fo]UND ERMINE D[eer] (hides)

Homophones are words that sound the same but have different meanings, such as ‘night’ and ‘knight’. Homophone clues always have an indicator word or phrase that has to do with being spoken or heard. For example:

clue:       We hear twins shave (4)
definition: We hear twins {shave}
answer:     PARE
wordplay:   "pair" (twins, "we hear")

A word that gets turned around to make another is a reversal. For example:

clue:       Returned beer fit for a king (5)
definition: Returned beer {fit for a king}
answer:     REGAL
wordplay:   (LAGER)< (beer, <returned)

The following is the rubric and wordplay preamble given to the Gemini LLM:

A Cryptic crossword question involves using the words in \
the given clue to yield an answer that matches the letter pattern.
The clue will provide a definition of the answer, as well \
as some ’wordplay’ that can also be used to confirm the answer.
Expert question solvers write informal ’proofs’ using a \
particular format.

For the definition, the original clue is annotated with \
’{}’ to denote where the definition is to be found.
For the wordplay, the following conventions are loosely used:
* The answer is assembled from the letters in CAPS
* Words in brackets show the origins of letters in CAPS, \
often being synonyms, or short forms
* Action words are annotated as illustrated:
  + (ETO N)* (*mad = anagram-signifier) = TONE
  + (FO OR)< (<back = reversal-signifier) = ROOF
  + [re]USE (missing = removal-signifier) = USE
* DD is a shorthand for ’Double Definition’

Around 20 examples from the Wordplay dataset are included in the in-context prompt:

For example:
---
clue: "arrived with an artist, to get optical device (6)"
definition: arrived with an artist, to get {optical device}
answer: CAMERA
wordplay: CAME (arrived) + RA (artist, short form)
---
clue: ...

Domain Specific Python functions are described in-context to the LLM, which appears able to use them without seeing their internal functionality. In fact, the actual implementation of the functions is more extensive than described, since calls to these functions also track ‘near misses’ which can be fed back as hints during the re-write process.

The task is to produce a formal proof using python code, \
where the docstring will also include an informal proof as an aid.
The following are functions that can be used in your output code:

Action=Enum(’Action’, ’ANAGRAM,REMOVE_FIRST,INITIALS,REMOVE_LAST,’+
                      ’GOES_INSIDE,GOES_OUTSIDE,REVERSE,SUBSTRING,HOMOPHONE’)
# External definitions
def is_synonym(phrase:str, test_synonym:str, pattern:str=’’) -> bool:
  # Determines whether ’test_synonym’ is a reasonable synonym for ’phrase’,
  # with letters optionally matching ’pattern’
def is_abbreviation(phrase:str, test_abbreviation:str) -> bool:
  # Determines whether ’test_abbreviation’ is
  # a valid abbreviation or short form for ’phrase’
def action_type(phrase:str, action:Action) -> bool:
  # Determines whether ’phrase’ might signify the given ’action’
def is_anagram(letters:str, word:str) -> bool:
  # Determines whether ’word’ can be formed from ’letters’ (i.e. an anagram)
def is_homophone(phrase:str, test_homophone:str) -> bool:
  # Determines whether ’test_homophone’ sounds like ’phrase’

The following are 3 (out of 6) of the few-shot formalisation examples given before the final test-case prompt:

The following are examples of simple functions that prove that \
each puzzle solution is correct:

‘‘‘python
def proof(answer="ONCE",
          clue="head decapitated long ago", pattern=’4’):
  """
  definition: head decapitated {long ago}
  wordplay: [b]ONCE (head decapitated = remove first letter of BONCE)
  """
  assert is_synonym("head", "BONCE")
  assert action_type("decapitated", Action.REMOVE_FIRST) \
         and "BONCE"[1:]=="ONCE"
  assert is_synonym("long ago", "ONCE", pattern=’4’)
proof()
‘‘‘

‘‘‘python
def proof(answer="DECIMAL",
          clue="the point of medical treatment", pattern=’7’):
  """
  definition: {the point} of medical treatment
  wordplay: (MEDICAL)* (*treatment = anagram)
  """
  assert is_synonym("the point", "DECIMAL", pattern=’7’)
  assert action_type("treatment", Action.ANAGRAM)
  assert is_anagram("MEDICAL", "DECIMAL")
proof()
‘‘‘

‘‘‘python
def proof(answer="SUPERMARKET",
          clue="fat bags for every brand thats a big seller",
          pattern=’11’):
  """
  definition: fat bags for every brand thats {a big seller}
  wordplay: SUET (fat) (bags = goes outside) of \
            (PER (for every) + MARK (brand))
  """
  assert is_synomym("fat", "SUET")
  assert action_type("bags", Action.IS_OUTSIDE)
  assert "SUET" == "SU" + "ET"
  assert is_abbreviation("for every", "PER")
  assert is_synomym("brand", "MARK")
  assert "SU"+"PER"+"MARK"+"ET" == "SUPERMARKET"
  assert is_synonym("a big seller", "SUPERMARKET", pattern=’11’)
proof()
‘‘‘

The following instruction is given before the final ‘test-case’ prompt illustrated in Figure 1:

# Please complete the following in a similar manner, and return the whole function:

‘‘‘python
def proof(answer= ...

Examples of assertion failures, with constructive hinting, are shown:

AssertionError: assert: is_abbreviation(’an Artist’, ’RA’) :
   ’an Artist’ does not have a valid abbreviation;
   ’RA’ is an abbreviation for : artist, artillery, Royal Artillery,
   gunners, painter
AssertionError: assert action_type(’goes crazy’, Action.ANAGRAM) :
  ’goes crazy’ itself does not suggest Action.ANAGRAM, but ’crazy’ does
AssertionError: assert action_type(’worked’, Action.HOMOPHONE) :
  ’worked’ does not suggest Action.HOMOPHONE, but maybe Action.ANAGRAM

# Please re-implement the SOLUTION above \
(altering both the docstring and the python code as required), \
taking care to fix each of the problems identified, \
and return the whole function:

‘‘‘python
def proof(answer= ...

Once the prover has fully parsed a given output with zero assertion failures, the proof is considered a success (up to 5 re-write iterations are allowed, more that that is considered an overall failure to prove the answer).