AI usage guidelines

Use the following guidelines to construct effective prompts and understand the limitations of AI outputs.

Prompt design guidelines

• Define the task clearly: Specify the exact goal of the prompt in the Prompt text field. Clear instructions reduce ambiguity in AI responses.
• Avoid formatting instructions: Avoid specifying custom output formats, such as using markdown, tables and other styling, in prompts. AI Highlighter uses the system prompt to control the AI output formatting, and overriding it at the prompt level may cause inconsistencies or errors.
• Set the context explicitly: Provide enough context for the LLM to understand and complete the task. Include only relevant portions of application data and documents. Do not include confidential or sensitive information that must not be processed or stored by the LLM. Avoid sending excessive or unnecessary content, as the LLM can only process data within its context window.
• Use safe and neutral language: Avoid language that may encourage biased or unsafe AI responses.
• Specify exact dates: LLMs are trained on historical data up to a fixed cutoff date. As a result, their responses may lag behind and not reflect the current changes. To improve accuracy, design prompts with clear context and specific dates. For example, to consider the most recent information, you can use the %date-today% or %application-academicTerm-start% content marker in prompts.
• Test and iterate: Run test prompts on sample applications to verify accuracy before you apply them to real submissions.

Limitations and considerations for AI outputs

• Variation in AI responses: Running the same prompt multiple times may produce different results, even for similar inputs. LLMs generate responses probabilistically, which can lead to variations in phrases or structure. Do not expect outputs to be identical across applications.
• Potential for hallucination: AI responses may include inaccurate, outdated or invented information. Always verify data against original source materials.
• Biased responses: AI responses strongly depend on the LLM’s training data and may reflect biases. Always evaluate results for fairness and consistency.
• Incomplete results: AI responses may sometimes be missing expected fields or data. Always check results for completeness.
• Document parsing limitations: LLMs may not correctly interpret content of poorly formatted or scanned documents. Inconsistent document layouts, low resolution images or unusual font styles may result in incomplete or inaccurate data extraction.
• System errors and processing limits: Some prompts may fail due to technical issues or LLM API constraints. Regularly monitor logs and review errors.