Mixture of Agents | Substrate

Learn how to implement Mixture of Agents

Language models can perform better on tasks when given the opportunity to reflect on a proposed response. "Mixture of Agents" is a pattern (coined by Together AI (opens in a new tab)) in which multiple different LLMs propose responses, and subsequent LLM steps evaluate and synthesize those responses into a single improved response.

This pattern is simple to implement with Substrate, automatically parallelized, and easy to extend.

TypeScript

Python


const substrate = new Substrate({ apiKey: SUBSTRATE_API_KEY });
const prompt = "write me a summary of don quixote";
const mist = new ComputeText({ prompt: prompt, model: "Mistral7BInstruct" });
const llama = new ComputeText({ prompt: prompt, model: "Llama3Instruct8B" });
const mixt = new ComputeText({ prompt: prompt, model: "Mixtral8x7BInstruct" });
const reasoning = new ComputeText({
  prompt: sb.interpolate`Reason about the strengths and weaknesses of each response. Explain which elements from each response are superior.
PROMPT: ${prompt}
CANDIDATE RESPONSES:
1) ${mist.future.text}
2) ${llama.future.text}
3) ${mixt.future.text}`,
});
const answer = new ComputeText({
  prompt: sb.interpolate`Come up with one detailed, comprehensive, unified response using the best parts of the candidate responses, based on the evaluation. Return only the response, do not reveal the process (do not say candidate response or evaluation).
PROMPT: ${prompt}
CANDIDATE RESPONSES:
1) ${mist.future.text}
2) ${llama.future.text}
3) ${mixt.future.text}
EVALUATION: ${reasoning.future.text}`,
});
const res = await substrate.run(answer);
console.log(res.get(answer).text);

Transcribe speech API design