Brief in progress — review the script and demo before recording.
This video continues our explanation of core agentic mechanics, focusing on how agents interact with the outside world through native tools, CLIs, and MCP servers. It should cover discovery, configuration, best practices, and common issues.
- Purpose: why might agents perform worse when using an API directly than when using a CLI or MCP server? What can make a CLI a poor fit for production? What makes a good MCP server, and what are the core components of MCP? We should also explore MCP as a capability layer for agents, not just a way to integrate external services.
- Design: how models see tools, how the harness presents them, and what makes a good tool name, description, schema, and result. How does an API designed for programmers differ from one designed for models? What are augmented tools: tools paired with skills or instructions?
- Evaluations: the role of the harness in evaluations, how model choice affects tool use, and which metrics help us evaluate tools within an agentic workflow.
- Practice: the biggest challenges in designing, deploying, maintaining, and extending CLI and MCP integrations. How can we prevent agents from misusing tools, protect privacy, and retain control when agents have access to multiple systems?
We may want to showcase scenarios such as:
- Generating an example MCP server from scratch.
- Showcasing evaluations from Goose or running example evaluations in Langfuse, Promptfoo, or a similar tool.