HUSKY: A Unified, Open-Source Language Agent for Complex Multi-Step Reasoning Across Domains

HUSKY, the open-source language agent developed by researchers from the University of Washington, Meta AI, and the Allen Institute for AI, uses a two-stage process to solve complex reasoning tasks4.

1. Generating the Next Action: In the first stage, HUSKY uses an action generator to determine the next step in solving a given task and the appropriate tool needed for that step1. This action generator is trained to predict high-level steps and the associated tools for solving tasks across different domains. The tools integrated into HUSKY's action space include code, math, search, and commonsense reasoning.

2. Executing the Action: In the second stage, HUSKY executes the predicted action using expert models. These expert models are trained to handle specific tasks such as generating code, performing mathematical reasoning, and crafting search queries. HUSKY continues this process of predicting and executing actions until it reaches a terminal state, i2.e., it arrives at a final solution.

This two-stage process allows HUSKY to handle diverse tasks efficiently, including numerical, tabular, and knowledge-based reasoning tasks13. It's also evaluated on mixed-tool tasks, where it consistently outperforms other agents, including REACT, CHAMELEON, and proprietary models like GPT-4.

HUSKY is a holistic, open-source language agent that distinguishes itself from other language agents in several ways. Firstly, it is designed to handle a diverse set of complex tasks involving numerical, tabular, and knowledge-based reasoning. Unlike most existing agents, which are based on proprietary models or designed to target specific tasks, HUSKY operates over a unified action space. This unified approach allows HUSKY to handle diverse challenges, such as numerical, tabular, and knowledge-based reasoning, whereas specialized agents might focus on specific domains like coding.

Secondly, HUSKY iterates between generating actions to solve tasks and executing these actions using expert models, constantly updating its solution state. This iterative process has proven to be a key point of distinction, enabling HUSKY to outperform previous agents across 14 datasets used for evaluation.

Another significant feature of HUSKY is its capability to manage mixed-tool reasoning. It excels in tasks that require retrieving missing knowledge and performing numerical calculations, achieving performance on par with, or exceeding, state-of-the-art models like GPT-4.

Finally, HUSKY is an open-source agent, making it more accessible and scalable compared to proprietary models. Despite using smaller 7B models, HUSKY matches or even exceeds the performance of larger, cutting-edge models on various benchmarks, showcasing the efficacy of its holistic approach in addressing complex reasoning problems.