Latest in AI
LangSmith Tool Gateway: build a small kit before giving agents more access
Start with a read-only LangSmith Tool Gateway kit, choose the right caller identity, and separate tool discovery, enforced policies and audit records.
An agent finding the right tool is only half the integration. The other half is knowing whose account it will use, whether the call is allowed and where the record will appear.
LangChain launched LangSmith Tool Gateway in public beta on November 19. It puts a managed MCP endpoint between clients and tool servers, handling credentials and tool access. The launch also directs teams to contact LangChain for access to deny policies and audit logs, so those controls should not be assumed to come with every setup. Launch announcement.
My first deployment would be a documentation-search kit, not an agent connected to every business system. It is a small enough workflow to inspect before introducing private accounts or write operations.
Check the prerequisites first
The quickstart requires LangSmith Plus or Enterprise. Workspace Admins can create shared servers and kits; Workspace Editors can create personal ones. Server and kit creation require a user credential such as a personal access token, not a service key.
For the public-docs pilot, add https://docs.langchain.com/mcp as a Public server. It needs no upstream sign-in. In Tools > Tool kits, create a kit containing search_docs_by_lang_chain, then copy its MCP URL into a client supporting Streamable HTTP and custom headers. The client supplies the LangSmith key in X-Api-Key. Quickstart.
Choose a personal kit while exploring alone. For a shared pilot, give the kit an owner and a name that describes its job. “Documentation lookup” is easier to review than “everything our agents might need.” Keep credentials out of the repository and shared onboarding screenshots.
Inspect discovery before expanding the kit
Ask the client to find one specific documentation page and inspect which tool it called. Check that the tool list matches the intended search-only setup, that the response is useful and that an upstream failure is visible rather than silently replaced with an answer from memory.
I would record those checks in the rollout ticket. This creates a baseline for later changes: adding a second server should solve a named problem, not merely make the tool list longer.
When the pilot is finished, avoid deleting a shared server as routine cleanup. The quickstart warns that deleting a server removes its connected accounts and affects everyone using it. Remove only resources whose owners agree they are disposable.
Choose user identity or agent identity deliberately
A personal access token calls tools as its owner. Multi-user backends have separate patterns using a service key with an end-user identity. Deployed agents can instead use a key bound to an agent environment, or a service key with a selected profile for unattended work.
Each request acts as one identity. Combining profile and end-user identity selectors is rejected with 400; a service key also cannot perform the management operations needed to create servers or kits. Identity documentation.
Before connecting an internal issue tracker, write down which account should own the access. A developer's successful local request is not proof that a scheduled agent has the right production identity. Keep the interactive-user and unattended-job paths separate in your configuration and review checklist.
A kit is not an access-control boundary
Kits curate the tools an agent sees. Policies enforce what it may invoke. The policy documentation says calls are allowed when no policy matches, and a block takes precedence over an approval requirement.
Policies apply across kit and single-server endpoints. A blocked tool can still appear in a tool list. Approval also means the caller confirms through a compatible MCP client, not that a request goes to a separate administrator. A client unable to display the approval form gets 403. Policy rules.
That distinction matters when moving beyond the search pilot. Do not call a production integration “read-only” simply because its visible kit contains read tools. Confirm the effective upstream permissions and available gateway policies. For unattended work, decide what should be blocked rather than relying on a person being present to approve it.
Separate tracing from the audit log
Gateway traces contain call metadata, not tool arguments or results. Workspace-wide recording is enabled under Tools > Usage. To nest gateway calls under an existing agent run, send that run's tracing headers on each call.
The Usage view shows the 100 latest calls from the last seven days; the full tracing view provides further browsing. Organization audit logs are Enterprise-only and record tool calls and configuration changes, but not requests that merely list tools. Tracing and audit documentation.
Do not turn that metadata-only guarantee into a claim about every log in your application. Review what the agent, client and upstream service record independently. Before widening access, verify one expected success and one intentionally blocked sandbox call, then check identity and outcome in the records.
I'm Paras. I work at the intersection of AI, engineering and distribution.
I help AI teams turn infrastructure details into clear technical writing and practical product stories.