critic_agent

langroid/agent/special/lance_rag/critic_agent.py

QueryPlanCritic is a ChatAgent that is created with a specific document schema.

Its role is to provide feedback on a Query Plan, which consists of: - filter condition if needed (or empty string if no filter is needed) - query - a possibly rephrased query that can be used to match the content field - dataframe_calc - a Pandas-dataframe calculation/aggregation string, possibly empty - original_query - the original query for reference - result - the answer received from an assistant that used this QUERY PLAN.

This agent has access to two tools: - QueryPlanTool: The handler method for this tool re-writes the query plan in plain text (non-JSON) so the LLM can provide its feedback using the QueryPlanFeedbackTool. - QueryPlanFeedbackTool: LLM uses this tool to provide feedback on the Query Plan

QueryPlanCritic(cfg)

Bases: ChatAgent

Critic for LanceQueryPlanAgent, provides feedback on query plan + answer.

Source code in langroid/agent/special/lance_rag/critic_agent.py

def __init__(self, cfg: LanceQueryPlanAgentConfig):
    super().__init__(cfg)
    self.config = cfg
    self.enable_message(QueryPlanAnswerTool, use=False, handle=True)
    self.enable_message(QueryPlanFeedbackTool, use=True, handle=True)
    self.enable_message(AgentDoneTool, use=False, handle=True)

query_plan_answer(msg)

Present query plan + answer in plain text (not JSON) so LLM can give feedback

Source code in langroid/agent/special/lance_rag/critic_agent.py

def query_plan_answer(self, msg: QueryPlanAnswerTool) -> str:
    """Present query plan + answer in plain text (not JSON)
    so LLM can give feedback"""
    self.expecting_feedback_tool = True
    return plain_text_query_plan(msg)

query_plan_feedback(msg)

Format Valid so return to Query Planner

Source code in langroid/agent/special/lance_rag/critic_agent.py

def query_plan_feedback(self, msg: QueryPlanFeedbackTool) -> AgentDoneTool:
    """Format Valid so return to Query Planner"""
    self.expecting_feedback_tool = False
    # indicate this task is Done, and return the tool as result
    return AgentDoneTool(tools=[msg])

handle_message_fallback(msg)

Remind the LLM to use QueryPlanFeedbackTool since it forgot

Source code in langroid/agent/special/lance_rag/critic_agent.py

def handle_message_fallback(
    self, msg: str | ChatDocument
) -> str | ChatDocument | None:
    """Remind the LLM to use QueryPlanFeedbackTool since it forgot"""
    if self.expecting_feedback_tool:
        return """
        You forgot to use the `query_plan_feedback` tool/function.
        Re-try your response using the `query_plan_feedback` tool/function,
        remember to provide feedback in the `feedback` field,
        and if any fix is suggested, provide it in the `suggested_fix` field.
        """
    return None