メモ
Computer use is in パブリック プレビュー and subject to change.
About computer use
Copilot can interact with desktop applications on your behalf by reading accessible application content and visual context, clicking controls, entering and editing text, pressing keys, scrolling, dragging, and navigating workflows across applications.
Computer use is available in GitHub Copilot CLI and GitHub Copilot app on macOS and Windows.
Computer use expands the tasks Copilot can help automate, including workflows in legacy and GUI-only software that do not provide an API, command-line interface, or MCP integration.
Computer use can help you complete tasks such as:
- Reviewing and summarizing information in a legacy desktop application.
- Updating content in a presentation.
- Entering or updating information in GUI-only software.
- Moving information between applications as part of a multi-step workflow.
ヒント
Computer use is designed for tasks that require interaction with a visual interface. If an API, MCP server, terminal command, filesystem tool, or dedicated browser tool can complete the task directly, that tool typically provides more structured information and predictable results.
Capabilities
When enabled, computer use allows Copilot to:
- Read accessible application content and visual context through your operating system's accessibility tree or screenshots when visual context is needed.
- Click controls.
- Enter and edit text.
- Press keys, scroll, and drag items.
- Navigate workflows across applications.
The agent selects these tools as it works through your prompt. You can review tool activity in the session.
Controlling access for computer use
Computer use is disabled by default. You must enable it before an agent can use its tools.
Computer use follows the tool permission settings for the Copilot surface where your session is running. These settings determine whether Copilot asks for approval before controlling a desktop application. You can change the tool permission settings for each surface. If prompted to approve access to an application, you can grant access for the current session, save the approval for future sessions, or deny access.
If you choose Always allow for a specific application, Copilot stores the decision locally. The decision applies to both GitHub Copilot CLI and GitHub Copilot app on the same computer. In the app, you can review the list of always allowed applications and remove individual approvals. Removing an application deletes its saved approval for future sessions in both the app and CLI. It does not revoke access already granted in a running session.
Permission rules that deny a tool take precedence over automatic or saved approvals.
Enterprise administrators can disable computer use through managed settings. Enabling computer use locally does not override an enterprise policy. For configuration details, see エンタープライズ管理設定.
You can interrupt an active operation if computer use starts acting unexpectedly. In GitHub Copilot CLI, press Esc twice. In GitHub Copilot app, click Stop or press Esc.
On macOS, computer use guides you through granting:
- Accessibility permission to interact with application controls.
- Screen Recording permission to inspect application windows when visual context is needed.
Limitations and risks
Computer use interprets interfaces that can change between application versions, operating systems, and window states. It can select the wrong control, enter text in the wrong location, or have difficulty with non-standard or dynamic controls and complex workflows. Changes in timing or window state can produce different results, cause computer use to repeat an action, or prevent it from continuing.
警告
Computer use can automate interactions across desktop applications, but it also introduces security risks. Ambiguous instructions or unexpected on-screen content may cause unintended actions that affect your device, data, or connected accounts, including access to personal, financial, or enterprise systems. Computer use is not a substitute for human judgment. Review the target application, requested permissions, and result, particularly before allowing actions that modify data or affect other people.
Application windows may display sensitive information, including information about other people. Only use computer use with applications and tasks whose visible content you are comfortable providing as context to Copilot.
If you choose Always allow for an application, later computer-use actions can control it without asking again. Avoid choosing Always allow for applications that contain sensitive information or support high-impact actions.
For comprehensive information about responsible use, see アプリケーション カード: GitHub Copilot エージェント.
Next steps
To enable and use computer use, see: