Microsoft’s Copilot Studio has introduced a groundbreaking ‘computer use’ feature that enables AI agents to interact with websites and desktop applications like humans—clicking, typing and navigating GUIs—paving the way for smarter, more intuitive enterprise automation. (Source: Image by RR)

Microsoft Copilot Studio Agents Adapt to Real-Time UI Changes with Ease

Microsoft Copilot Studio continues to lead enterprise AI innovation by integrating cutting-edge capabilities into a platform designed to solve complex business challenges. Building on recent updates like deep reasoning, model context protocol (MCP) support, and the general availability of agent flows, Microsoft has now introduced “computer use”—a major upgrade available through an early access research preview. This new feature, according to a story at microsoft.com, allows AI agents to interact directly with websites and desktop applications, essentially enabling them to perform tasks in any system with a graphical user interface (GUI), even when APIs are unavailable. This marks a significant step forward in making Copilot Studio agents more capable and autonomous.

The “computer use” feature empowers agents to emulate human interactions with on-screen elements by clicking, typing and navigating menus across web browsers and desktop applications. It automatically adapts to changes in the interface, using built-in reasoning to continue tasks without manual adjustments. Notably, this feature operates within Microsoft’s secure cloud infrastructure, ensuring data privacy and compliance. Organizations benefit from reduced maintenance and deployment costs since the infrastructure is fully managed by Microsoft. The agents function across major browsers such as Edge, Chrome and Firefox, creating seamless cross-platform support.

This advancement unlocks a wide range of automation and agentic scenarios for various industries. For instance, agents can automate data entry from multiple sources into centralized systems, reducing time and errors. Marketing teams can streamline online market research, gathering insights without manual input. Finance departments can automate invoice processing by extracting and entering data into accounting systems, significantly improving efficiency. These use cases demonstrate how Copilot Studio’s “computer use” capability brings tangible value across enterprise workflows by reducing manual tasks and increasing operational speed.

Copilot Studio’s intelligent automation capabilities also reimagine traditional robotic process automation (RPA). Unlike legacy RPA tools that struggle with dynamic or changing UIs, Copilot Studio adapts in real time and allows users to prompt and guide agents using natural language. Makers can monitor AI actions through side-by-side video playback, view detailed histories of interactions and understand how decisions are made—enhancing transparency and control. With these advancements, Copilot Studio transforms automation from a technical function into a strategic business enabler. Microsoft invites interested users to sign up for early access to “computer use” and be part of shaping the next wave of intelligent enterprise solutions.

read more at microsoft.com