Skip to content

critic_agent

langroid/agent/special/lance_rag/critic_agent.py

QueryPlanCritic is a ChatAgent that is created with a specific document schema.

Its role is to provide feedback on a Query Plan, which consists of: - filter condition if needed (or empty string if no filter is needed) - query - a possibly rephrased query that can be used to match the content field - dataframe_calc - a Pandas-dataframe calculation/aggregation string, possibly empty - original_query - the original query for reference - result - the answer received from an assistant that used this QUERY PLAN.

This agent has access to two tools: - QueryPlanTool: The handler method for this tool re-writes the query plan in plain text (non-JSON) so the LLM can provide its feedback using the QueryPlanFeedbackTool. - QueryPlanFeedbackTool: LLM uses this tool to provide feedback on the Query Plan

QueryPlanCritic(cfg)

Bases: ChatAgent

Critic for LanceQueryPlanAgent, provides feedback on query plan + answer.

Source code in langroid/agent/special/lance_rag/critic_agent.py 
111
112
113
114
115
def __init__(self, cfg: LanceQueryPlanAgentConfig):
    super().__init__(cfg)
    self.config = cfg
    self.enable_message(QueryPlanAnswerTool, use=False, handle=True)
    self.enable_message(QueryPlanFeedbackTool, use=True, handle=True)

query_plan_answer(msg)

Present query plan + answer in plain text (not JSON) so LLM can give feedback

Source code in langroid/agent/special/lance_rag/critic_agent.py 
117
118
119
120
def query_plan_answer(self, msg: QueryPlanAnswerTool) -> str:
    """Present query plan + answer in plain text (not JSON)
    so LLM can give feedback"""
    return plain_text_query_plan(msg)

query_plan_feedback(msg)

Format Valid so return to Query Planner

Source code in langroid/agent/special/lance_rag/critic_agent.py 
122
123
124
def query_plan_feedback(self, msg: QueryPlanFeedbackTool) -> str:
    """Format Valid so return to Query Planner"""
    return DONE + " " + PASS  # return to Query Planner

handle_message_fallback(msg)

Create QueryPlanFeedbackTool since LLM forgot

Source code in langroid/agent/special/lance_rag/critic_agent.py 
126
127
128
129
130
131
132
133
134
135
136
def handle_message_fallback(
    self, msg: str | ChatDocument
) -> str | ChatDocument | None:
    """Create QueryPlanFeedbackTool since LLM forgot"""
    if isinstance(msg, ChatDocument) and msg.metadata.sender == Entity.LLM:
        # our LLM forgot to use the QueryPlanFeedbackTool
        feedback = QueryPlanFeedbackTool(feedback=msg.content)
        msg.tool_messages = [feedback]
        msg.content = DONE
        return msg
    return None