AI Highlighter on DreamApply Knowledge Base

AI integration basics

Mon, 01 Jan 0001 00:00:00 +0000

DreamApply provides access to AI-powered features through AI Highlighter. AI Highlighter integrates DreamApply with third-party Large Language Models (LLMs) and allows Administrators to perform admission tasks with the help of AI.

AI Highlighter can assist Administrators with the following tasks:

Data summarization: Generate concise summaries of application data
Document parsing and data extraction: Identify and extract key information from application data and uploaded documents
Classification and labeling: Analyze application data to automatically categorize and label applications based on specific criteria
Validation and consistency checking: Detect discrepancies or inconsistencies in applications
Text translation: Translate application materials into the institution’s primary language
AI-driven image verification: Automatically identify and analyze uploaded images, such as passport photos

Treat AI Highlighter as a helper, not a decision-maker. Administrators themselves remain responsible for interpreting AI responses and making final decisions. Always trace the AI reasoning trail that AI Highlighter displays for each AI output and verify it against the original documents. For details, see View AI responses.

LLM-based data processing

DreamApply does not develop or host its own AI models. Instead, it integrates with third-party LLMs to process application data.

Set up AI prompts

Mon, 01 Jan 0001 00:00:00 +0000

In DreamApply, prompts are set up centrally by Administrators with high-level privileges. Admissions administrators can only run approved prompts on selected applications; they cannot create or modify prompts themselves.

Centralized prompt creation ensures flexible AI support and minimizes the risk of accidental data exposure or misuse of the AI.

Before you begin

To create prompts, your account must have the Superuser role or the ai-prompts permission.

Create prompts

To create a prompt:

Exclude data from AI analysis

Mon, 01 Jan 0001 00:00:00 +0000

DreamApply allows you to control which application data and documents are shared with the LLM for AI processing. You can exclude data using:

Appforms
Rules

Before you begin

To manage appforms, your account must have the Superuser or Courses administrator role or the appforms permission.
To set up rules, your account must have the Superuser role or the rules permissions.
To manage consents, your account must have the Superuser role or the consents permission.

Exclude appform sections

You can mark specific sections in appforms as containing sensitive data. In this case, DreamApply automatically excludes these sections from being sent to the LLM for all prompts across the system.

Run AI analysis

Mon, 01 Jan 0001 00:00:00 +0000

To analyze application data, you can run prepared AI prompts for selected applications.

AI Highlighter analyzes application data on the per-application basis. You can run a prompt in the following ways:

For individual applications
For a group of applications
Using rules

Before you begin

To analyze applications with AI, your account must have the Superuser or Admissions administrator role or the ai-inrefence, applications-view, applications-edit permissions.
To set up automatic rules, your account must have the Superuser role or the rules permissions.

Analyze individual applications

To analyze a single application:

View AI processing statistics

Mon, 01 Jan 0001 00:00:00 +0000

AI Highlighter logs all AI responses and processing activities. You can:

Track AI processing statistics
Review AI outputs
Detect errors that occurred during AI data processing

To view the AI statistics, in the main menu, under AI highlighter, select Highlights.

AI Highlighter automatically detects and logs several types of errors that may occur during AI data processing:

Missing data: The LLM was unable to generate a response due to insufficient information in the provided context materials
Prompt injection: A request was blocked because the prompt attempted to override the initial instructions (a security measure, may occur if a third-party attacker tried to inject malicious instructions into the prompt)
Confusing prompt: The instructions provided in the prompt were contradictory or unclear
Defer to human: The prompt task violated ethical policies or core AI directives; the response requires human review
Other reasons: A generic processing error occurred

To view a list of errors, use the AI highlighter > Statuses category in filters.

AI usage guidelines

Mon, 01 Jan 0001 00:00:00 +0000

Use the following guidelines to construct effective prompts and understand the limitations of AI outputs.

Prompt design guidelines

Define the task clearly: Specify the exact goal of the prompt in the Prompt text field. Clear instructions reduce ambiguity in AI responses.
Avoid formatting instructions: Avoid specifying custom output formats, such as using markdown, tables and other styling, in prompts. AI Highlighter uses the system prompt to control the AI output formatting, and overriding it at the prompt level may cause inconsistencies or errors.
Set the context explicitly: Provide enough context for the LLM to understand and complete the task. Include only relevant portions of application data and documents. Do not include confidential or sensitive information that must not be processed or stored by the LLM. Avoid sending excessive or unnecessary content, as the LLM can only process data within its context window.
Use safe and neutral language: Avoid language that may encourage biased or unsafe AI responses.
Specify exact dates: LLMs are trained on historical data up to a fixed cutoff date. As a result, their responses may lag behind and not reflect the current changes. To improve accuracy, design prompts with clear context and specific dates. For example, to consider the most recent information, you can use the %date-today% or %application-academicTerm-start% content marker in prompts.
Test and iterate: Run test prompts on sample applications to verify accuracy before you apply them to real submissions.

Limitations and considerations for AI outputs

Variation in AI responses: Running the same prompt multiple times may produce different results, even for similar inputs. LLMs generate responses probabilistically, which can lead to variations in phrases or structure. Do not expect outputs to be identical across applications.
Potential for hallucination: AI responses may include inaccurate, outdated or invented information. Always verify data against original source materials.
Biased responses: AI responses strongly depend on the LLM’s training data and may reflect biases. Always evaluate results for fairness and consistency.
Incomplete results: AI responses may sometimes be missing expected fields or data. Always check results for completeness.
Document parsing limitations: LLMs may not correctly interpret content of poorly formatted or scanned documents. Inconsistent document layouts, low resolution images or unusual font styles may result in incomplete or inaccurate data extraction.
System errors and processing limits: Some prompts may fail due to technical issues or LLM API constraints. Regularly monitor logs and review errors.

AI outputs are advisory. Always review the results and validate all extracted data before making any final decisions.

Prompt library

Mon, 01 Jan 0001 00:00:00 +0000

Browse our library of AI Highlighter prompts maintained by the DreamApply Configuration Team.

Admissions Faculty Finance Compliance Cross-functional

Admissions

Application Completeness Check

Quick triage: is the application ready for review?

Prompt settings:

Output: Label
Temp: 0.15
Values: Complete Incomplete Partially complete other

View all prompt details

Prompt text:

Check whether this application is complete.

Verify:
1. Required form sections are filled.
2. Required uploads are present: %application-tasks-status(*)%
3. Motivation letter has substantive content: %application-motivation%

Context materials:

All sections, All tasks