{"id":180466,"date":"2026-07-03T18:35:44","date_gmt":"2026-07-03T18:35:44","guid":{"rendered":"http:\/\/ktromedia.com\/?p=180466"},"modified":"2026-07-03T18:35:44","modified_gmt":"2026-07-03T18:35:44","slug":"building-browser-using-ai-agents-in-python","status":"publish","type":"post","link":"https:\/\/ktromedia.com\/?p=180466","title":{"rendered":"Building Browser-Using AI Agents in Python"},"content":{"rendered":"<div id=\"\">\n<p>In this article, you will learn how to build AI agents that can browse and interact with real websites using Playwright, browser-use, and LangGraph.<\/p>\n<p>Topics we will cover include:<\/p>\n<ul>\n<li>Why Playwright is the right foundation for browser automation in 2026, and how it differs from Selenium.<\/li>\n<li>How to scrape dynamic, JavaScript-rendered pages and complete multi-step forms reliably.<\/li>\n<li>How to wire browser actions into LangGraph and browser-use agents, handle anti-bot detection, manage waiting and session persistence, and deploy the result in Docker.<\/li>\n<\/ul>\n<div style=\"width: 810px\" class=\"wp-caption aligncenter\"><\/p>\n<p class=\"wp-caption-text\">Building Browser-Using AI Agents in Python<\/p>\n<\/div>\n<h2>Introduction<\/h2>\n<p>Most AI agent tutorials start with an API. They show you how to call OpenWeather, hit the Stripe endpoint, pull data from GitHub. That is a fine starting point until you try to build something real and realize that the task you actually need done does not have an API.<\/p>\n<p>Think about what humans do with browsers every day: filing government forms, reading competitor pricing, extracting research from sites that guard their data behind JavaScript rendering, logging into portals that have never heard of OAuth. There are roughly 1.1 billion websites on the internet. A vanishingly small fraction of them have public APIs. The rest only speak browser.<\/p>\n<p>An agent that is limited to API calls handles maybe 5% of the tasks a human worker does daily. Give that agent a browser, and the coverage approaches everything. That is the gap this article closes.<\/p>\n<p>The <a href=\"https:\/\/www.ringly.io\/blog\/ai-agent-statistics-2026\" target=\"_blank\">global AI agents market<\/a> stands at \\$10.91 billion in 2026 and is projected to reach \\$50.31 billion by 2030, with browser-capable agents at the center of that growth. <a href=\"https:\/\/velofill.com\/articles\/agentic-ai-browsers-2026-market-landscape\/\" target=\"_blank\">27.7% of enterprises<\/a> are already running agentic browsers in production, up from virtually none two years prior. The tooling has matured fast, and the patterns are settled enough to teach properly.<\/p>\n<p>By the end of this article, you will have a working browser agent that navigates real websites, fills forms, extracts structured data, and connects to an LLM that decides what to do next, all in Python.<\/p>\n<h2>Why Playwright, Not Selenium<\/h2>\n<p>If you built browser automation five years ago, you built it with Selenium. Selenium is still widely deployed, still works, and is not going anywhere. But for any new project in 2026, Playwright is the default. The reasons are practical, not theoretical.<\/p>\n<p>Selenium communicates with the browser by sending individual HTTP requests to a WebDriver. Every action, click, type, scroll, is a separate request. Playwright uses a persistent WebSocket connection for the entire session. Commands flow through that channel with no per-action round-trip cost. Independent benchmarks consistently show Playwright running 30-50% faster than Selenium at the test-suite level and averaging ~290ms per action versus Selenium\u2019s ~536ms. For a browser agent that might execute hundreds of actions, that gap compounds.<\/p>\n<p>Playwright also bundles its own browser binaries. When you install it, you get pre-configured versions of Chromium, Firefox, and WebKit that are guaranteed to work with your Playwright version. No driver version mismatches, no broken CI pipelines because someone updated Chrome. It has built-in auto-waiting before it clicks an element; it verifies the element is visible, enabled, and not animating. You do not have to write <strong>time.sleep(2)<\/strong> and hope for the best.<\/p>\n<p>For AI agents specifically, Playwright fires real mouse and keyboard events that mirror how humans interact with browsers. Sites designed to detect automation look for synthetic DOM clicks. Playwright\u2019s interaction model is harder to distinguish from genuine human input.<\/p>\n<p>There is also the <strong>browser-use<\/strong> library, which sits one level higher. Browser-use is a Python library that gives an LLM a working browser. Under the hood, it uses Playwright to drive the browser, but the LLM reads the page state and decides what to click, type, and extract, no CSS selectors required. You give it a task in plain English, and it figures out the rest. We will cover both raw Playwright and <strong>browser-use<\/strong> in this article, because they serve different needs: Playwright when you want precise, predictable control; <strong>browser-use<\/strong> when you want the agent to handle navigation decisions autonomously.<\/p>\n<h2>Setting Up the Environment<\/h2>\n<p>You need Python 3.10 or higher, an OpenAI API key, and about five minutes.<\/p>\n<p><strong>Step 1: Create a virtual environment<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5d6498594457\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\npython -m venv browser_agent_env&#13;<br \/>\n&#13;<br \/>\n# macOS \/ Linux&#13;<br \/>\nsource browser_agent_env\/bin\/activate&#13;<br \/>\n&#13;<br \/>\n# Windows&#13;<br \/>\nbrowser_agent_env\\Scripts\\activate<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-v\">python<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">m<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">venv <\/span><span class=\"crayon-v\">browser_agent<\/span><span class=\"crayon-sy\">_<\/span>env<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># macOS \/ Linux<\/span><\/p>\n<p><span class=\"crayon-e\">source <\/span><span class=\"crayon-v\">browser_agent_env<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-v\">bin<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-i\">activate<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Windows<\/span><\/p>\n<p><span class=\"crayon-v\">browser_agent_env<\/span><span class=\"crayon-sy\">\\<\/span><span class=\"crayon-v\">Scripts<\/span><span class=\"crayon-sy\">\\<\/span><span class=\"crayon-v\">activate<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>Step 2: Install dependencies<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5e0818948603\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\npip install playwright \\&#13;<br \/>\n            browser-use \\&#13;<br \/>\n            langchain \\&#13;<br \/>\n            langchain-openai \\&#13;<br \/>\n            langgraph \\&#13;<br \/>\n            langchain-community \\&#13;<br \/>\n            python-dotenv<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-e\">pip <\/span><span class=\"crayon-e\">install <\/span><span class=\"crayon-i\">playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-st\">use<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">langchain<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">langchain<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">openai<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">langgraph<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">langchain<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">community<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">python<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">dotenv<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>Step 3: Install the browser binaries<\/strong><br \/>This is the step most people miss. Playwright needs to download Chromium, Firefox, and WebKit separately from the Python package. Run this once after installing:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5e4549041021\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\nplaywright install chromium<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-e\">playwright <\/span><span class=\"crayon-e\">install <\/span><span class=\"crayon-v\">chromium<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>If you want all three browser engines: <strong>playwright install<\/strong>. Chromium alone is sufficient for most agent work and is smaller to download.<\/p>\n<p><strong>Step 4: Store your API key<\/strong><br \/>Create a <strong>.env<\/strong> file in your project directory:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5e7111033822\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\nOPENAI_API_KEY=your_openai_api_key_here<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-v\">OPENAI_API_KEY<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">your_openai_api_key_here<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>Add <strong>.env<\/strong> to your <strong>.gitignore<\/strong> immediately. Do not commit API keys.<\/p>\n<p><strong>Step 5: Verify everything works<\/strong><br \/>Here is a first script that navigates to a URL, reads the heading, and saves a screenshot. Use <a href=\"http:\/\/example.com\" target=\"_blank\">example.com<\/a>, a publicly available test domain maintained by <strong>IANA<\/strong> that will not block you.<\/p>\n<p>How to run: Save as <strong>first_run.py<\/strong> and run python <strong>first_run.py<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5ea158106395\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# first_run.py&#13;<br \/>\n# Navigate to a URL, take a screenshot, and extract the page title.&#13;<br \/>\n# Prerequisites: pip install playwright &amp;&amp; playwright install chromium&#13;<br \/>\n# How to run: python first_run.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nasync def main():&#13;<br \/>\n    async with async_playwright() as p:&#13;<br \/>\n        # Launch Chromium in headless mode (no visible browser window).&#13;<br \/>\n        # Set headless=False if you want to watch it run during development.&#13;<br \/>\n        browser = await p.chromium.launch(headless=True)&#13;<br \/>\n&#13;<br \/>\n        # A browser context is like a fresh browser profile.&#13;<br \/>\n        # It isolates cookies, storage, and cache from other contexts.&#13;<br \/>\n        context = await browser.new_context(&#13;<br \/>\n            viewport={&#8220;width&#8221;: 1280, &#8220;height&#8221;: 720},&#13;<br \/>\n            user_agent=(&#13;<br \/>\n                &#8220;Mozilla\/5.0 (Windows NT 10.0; Win64; x64) &#8220;&#13;<br \/>\n                &#8220;AppleWebKit\/537.36 (KHTML, like Gecko) &#8220;&#13;<br \/>\n                &#8220;Chrome\/120.0.0.0 Safari\/537.36&#8243;&#13;<br \/>\n            )&#13;<br \/>\n        )&#13;<br \/>\n&#13;<br \/>\n        page = await context.new_page()&#13;<br \/>\n&#13;<br \/>\n        # Navigate to the URL and wait until the network is idle.&#13;<br \/>\n        # &#8220;networkidle&#8221; means no open network connections for 500ms.&#13;<br \/>\n        # For faster pages, &#8220;domcontentloaded&#8221; is sufficient.&#13;<br \/>\n        await page.goto(&#8220;https:\/\/example.com&#8221;, wait_until=&#8221;networkidle&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # Extract the page title&#13;<br \/>\n        title = await page.title()&#13;<br \/>\n        print(f&#8221;Page title: {title}&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # Extract the text content of the h1 heading&#13;<br \/>\n        h1 = await page.text_content(&#8220;h1&#8243;)&#13;<br \/>\n        print(f&#8221;H1 heading: {h1}&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # Take a full-page screenshot and save it to disk&#13;<br \/>\n        await page.screenshot(path=&#8221;screenshot.png&#8221;, full_page=True)&#13;<br \/>\n        print(&#8220;Screenshot saved to screenshot.png&#8221;)&#13;<br \/>\n&#13;<br \/>\n        await browser.close()&#13;<br \/>\n&#13;<br \/>\nasyncio.run(main())<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># first_run.py<\/span><\/p>\n<p><span class=\"crayon-p\"># Navigate to a URL, take a screenshot, and extract the page title.<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install playwright &amp;&amp; playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python first_run.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">with <\/span><span class=\"crayon-e\">async_playwright<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Launch Chromium in headless mode (no visible browser window).<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Set headless=False if you want to watch it run during development.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">await<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># A browser context is like a fresh browser profile.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># It isolates cookies, storage, and cache from other contexts.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">viewport<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;width&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1280<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;height&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">720<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">user_agent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Mozilla\/5.0 (Windows NT 10.0; Win64; x64) &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;AppleWebKit\/537.36 (KHTML, like Gecko) &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Chrome\/120.0.0.0 Safari\/537.36&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Navigate to the URL and wait until the network is idle.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># &#8220;networkidle&#8221; means no open network connections for 500ms.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># For faster pages, &#8220;domcontentloaded&#8221; is sufficient.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-st\">goto<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;https:\/\/example.com&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wait_until<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;networkidle&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Extract the page title<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">title<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">title<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Page title: {title}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Extract the text content of the h1 heading<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">h1<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">text_content<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;h1&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;H1 heading: {h1}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Take a full-page screenshot and save it to disk<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">screenshot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">path<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;screenshot.png&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">full_page<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;Screenshot saved to screenshot.png&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> async_playwright() is the entry point for the entire Playwright session. The <strong>browser_context<\/strong> is equivalent to opening a fresh incognito window; cookies, local storage, and cache are isolated from everything else. <strong>wait_until=\u201dnetworkidle\u201d<\/strong> tells Playwright to wait until the page has finished all its network activity before your code continues, which is the safest wait strategy for dynamic pages.<\/p>\n<p>If this runs and saves a screenshot, your environment is working correctly.<\/p>\n<h2>Web Navigation and Scraping<\/h2>\n<p>The reason you need Playwright instead of <strong>requests + BeautifulSoup<\/strong> is JavaScript rendering. Modern websites deliver a skeleton of HTML and then build the actual content dynamically after the page loads: React, Vue, Angular, Next.js. A plain HTTP request fetches the skeleton. Playwright runs a real browser, so it sees exactly what a human sees after all JavaScript has executed.<\/p>\n<p>The target below is <a href=\"http:\/\/books.toscrape.com\/\" target=\"_blank\">books.toscrape.com<\/a>, a legal scraping sandbox built for practice. It paginates results, uses dynamic class names for ratings, and closely mirrors the structure of real e-commerce product pages.<\/p>\n<p>How to run: Save as <strong>scrape_books.py<\/strong> and run python <strong>scrape_books.py<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5ec067080982\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<div class=\"urvanov-syntax-highlighter-plain-wrap\"><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# scrape_books.py&#13;<br \/>\n# Scrape book titles, prices, and ratings from books.toscrape.com&#13;<br \/>\n# This is a legal scraping sandbox site built for practice.&#13;<br \/>\n# Prerequisites: pip install playwright &amp;&amp; playwright install chromium&#13;<br \/>\n# How to run: python scrape_books.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nimport json&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nasync def scrape_books(max_pages: int = 3) -&gt; list[dict]:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Scrape book listings from books.toscrape.com across multiple pages.&#13;<br \/>\n    Returns a list of dicts with title, price, rating, and page number.&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    results = []&#13;<br \/>\n&#13;<br \/>\n    async with async_playwright() as p:&#13;<br \/>\n        browser = await p.chromium.launch(headless=True)&#13;<br \/>\n        context = await browser.new_context(viewport={&#8220;width&#8221;: 1280, &#8220;height&#8221;: 720})&#13;<br \/>\n        page = await context.new_page()&#13;<br \/>\n&#13;<br \/>\n        for page_num in range(1, max_pages + 1):&#13;<br \/>\n            url = f&#8221;https:\/\/books.toscrape.com\/catalogue\/page-{page_num}.html&#8221;&#13;<br \/>\n            print(f&#8221;Scraping page {page_num}: {url}&#8221;)&#13;<br \/>\n&#13;<br \/>\n            await page.goto(url, wait_until=&#8221;domcontentloaded&#8221;)&#13;<br \/>\n&#13;<br \/>\n            # Wait for the product cards to be visible before extracting.&#13;<br \/>\n            # This is critical on JavaScript-heavy pages where content loads after the HTML.&#13;<br \/>\n            # timeout=10000 means wait up to 10 seconds before raising an error.&#13;<br \/>\n            await page.wait_for_selector(&#8220;article.product_pod&#8221;, timeout=10000)&#13;<br \/>\n&#13;<br \/>\n            # Get all book cards on the current page&#13;<br \/>\n            books = await page.query_selector_all(&#8220;article.product_pod&#8221;)&#13;<br \/>\n&#13;<br \/>\n            for book in books:&#13;<br \/>\n                # Extract title from the <a> tag&#8217;s title attribute&#13;<br \/>\n                title_el = await book.query_selector(&#8220;h3 a&#8221;)&#13;<br \/>\n                title = await title_el.get_attribute(&#8220;title&#8221;) if title_el else &#8220;N\/A&#8221;&#13;<br \/>\n&#13;<br \/>\n                # Extract price text&#13;<br \/>\n                price_el = await book.query_selector(&#8220;.price_color&#8221;)&#13;<br \/>\n                price = await price_el.inner_text() if price_el else &#8220;N\/A&#8221;&#13;<br \/>\n&#13;<br \/>\n                # Extract star rating from the CSS class name.&#13;<br \/>\n                # e.g. <\/p>\n<p class=\"star-rating Three\"> \u2192 &#8220;Three&#8221;&#13;<br \/>\n                rating_el = await book.query_selector(&#8220;p.star-rating&#8221;)&#13;<br \/>\n                rating_class = await rating_el.get_attribute(&#8220;class&#8221;) if rating_el else &#8220;&#8221;&#13;<br \/>\n                rating = rating_class.replace(&#8220;star-rating&#8221;, &#8220;&#8221;).strip()&#13;<br \/>\n&#13;<br \/>\n                results.append({&#13;<br \/>\n                    &#8220;title&#8221;: title,&#13;<br \/>\n                    &#8220;price&#8221;: price,&#13;<br \/>\n                    &#8220;rating&#8221;: rating,&#13;<br \/>\n                    &#8220;page&#8221;: page_num&#13;<br \/>\n                })&#13;<br \/>\n&#13;<br \/>\n            print(f&#8221;  Extracted {len(books)} books from page {page_num}&#8221;)&#13;<br \/>\n&#13;<br \/>\n        await browser.close()&#13;<br \/>\n&#13;<br \/>\n    return results&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def main():&#13;<br \/>\n    books = await scrape_books(max_pages=2)&#13;<br \/>\n    print(f&#8221;\\nTotal books scraped: {len(books)}&#8221;)&#13;<br \/>\n    print(json.dumps(books[:3], indent=2))&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasyncio.run(main())<\/p>\n<p><\/a><\/textarea><\/div>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>68<\/p>\n<p>69<\/p>\n<p>70<\/p>\n<p>71<\/p>\n<p>72<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># scrape_books.py<\/span><\/p>\n<p><span class=\"crayon-p\"># Scrape book titles, prices, and ratings from books.toscrape.com<\/span><\/p>\n<p><span class=\"crayon-p\"># This is a legal scraping sandbox site built for practice.<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install playwright &amp;&amp; playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python scrape_books.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">json<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">scrape_books<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">max_pages<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">list<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">dict<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Scrape book listings from books.toscrape.com across multiple pages.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Returns a list of dicts with title, price, rating, and page number.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">results<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">with <\/span><span class=\"crayon-e\">async_playwright<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">await<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">viewport<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;width&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1280<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;height&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">720<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">page_num <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">range<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">max_pages<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">+<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;https:\/\/books.toscrape.com\/catalogue\/page-{page_num}.html&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Scraping page {page_num}: {url}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-st\">goto<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wait_until<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;domcontentloaded&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Wait for the product cards to be visible before extracting.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># This is critical on JavaScript-heavy pages where content loads after the HTML.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># timeout=10000 means wait up to 10 seconds before raising an error.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;article.product_pod&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10000<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Get all book cards on the current page<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">books<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector_all<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;article.product_pod&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">book <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">books<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">title_el<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">book<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;h3 a&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">title<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">title_el<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_attribute<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;title&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">title_el <\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;N\/A&#8221;<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Extract price text<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">price_el<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">book<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.price_color&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">price<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">price_el<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">inner_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">price_el <\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;N\/A&#8221;<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Extract star rating from the CSS class name.<\/span><\/p>\n<div class=\"crayon-line\" id=\"urvanov-syntax-highlighter-6a3d6b51aa5ec067080982-47\"><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># e.g. <\/p>\n<p class=\"star-rating Three\"> \u2192 &#8220;Three&#8221;<\/p>\n<p><\/span><\/div>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rating_el<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">book<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;p.star-rating&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rating_class<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">rating_el<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">get_attribute<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;class&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">rating_el <\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">rating<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">rating_class<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">replace<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;star-rating&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">strip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">results<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">{<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;title&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">title<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;price&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">price<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;rating&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">rating<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;page&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">_<\/span>num<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8221;\u00a0\u00a0Extracted {len(books)} books from page {page_num}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">results<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">books<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">scrape_books<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">max_pages<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;\\nTotal books scraped: {len(books)}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">json<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dumps<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">books<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">indent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> <strong>wait_for_selector()<\/strong> is the key call here. Instead of sleeping for a fixed time and hoping the content has loaded, it watches the DOM and proceeds the moment the target element appears, or raises a <strong>TimeoutError<\/strong> if it does not appear within the timeout window. That is the right behavior: fail fast and explicitly rather than silently extracting from an empty page.<\/p>\n<p>The rating extraction deserves attention. The star rating is encoded as a CSS class (<strong>star-rating Three<\/strong>), not a number. The code strips \u201cstar-rating\u201d from the class string to get the text value. This is the kind of thing you only know by inspecting the actual HTML. When you hand this task to a raw LLM with no browser, it has no way to know what the class structure looks like. With Playwright, you can inspect it directly and extract it exactly.<\/p>\n<h2>Form Completion and Multi-Step Flows<\/h2>\n<p>Filling forms is where browser agents earn their keep and where most automation scripts fail. The reason is that web forms are not just inputs and buttons. They fire <strong>focus, input, change<\/strong>, and <strong>blur<\/strong> events in sequence. JavaScript validation listens for those events. If you inject a value into an input field by directly setting <strong>value<\/strong> in the DOM (as older automation tools often do), the validation listeners never fire and the form breaks.<\/p>\n<p>Playwright\u2019s <strong>fill()<\/strong> and <strong>click()<\/strong> methods fire real browser events in the right order, which is why they work on form validation that would block lower-level approaches.<\/p>\n<p>The target below is <a href=\"http:\/\/the-internet.herokuapp.com\/login\" target=\"_blank\">the-internet.herokuapp.com\/login<\/a>, a public test site maintained specifically for automation practice. It accepts <strong>tomsmith \/ SuperSecretPassword!<\/strong> as valid credentials and returns clear success\/failure messages.<\/p>\n<p>How to run: Save as <strong>form_submit.py<\/strong> and run python <strong>form_submit.py<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5f0016812360\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# form_submit.py&#13;<br \/>\n# Complete and submit a multi-field login form on a public demo site.&#13;<br \/>\n# Target: https:\/\/the-internet.herokuapp.com\/login (public test site)&#13;<br \/>\n# Prerequisites: pip install playwright &amp;&amp; playwright install chromium&#13;<br \/>\n# How to run: python form_submit.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nasync def login_and_verify(username: str, password: str) -&gt; dict:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Attempt to log in to a demo site and return whether it succeeded.&#13;<br \/>\n    Handles: input filling, button clicking, and result verification.&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    async with async_playwright() as p:&#13;<br \/>\n        browser = await p.chromium.launch(headless=True)&#13;<br \/>\n        context = await browser.new_context()&#13;<br \/>\n        page = await context.new_page()&#13;<br \/>\n&#13;<br \/>\n        await page.goto(&#8220;https:\/\/the-internet.herokuapp.com\/login&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # Wait for the form to be visible before interacting.&#13;<br \/>\n        # state=&#8221;visible&#8221; is the default but makes the intent explicit.&#13;<br \/>\n        await page.wait_for_selector(&#8220;#username&#8221;, state=&#8221;visible&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # fill() clears the field first, then types the value.&#13;<br \/>\n        # It fires the focus, input, and change events in order.&#13;<br \/>\n        await page.fill(&#8220;#username&#8221;, username)&#13;<br \/>\n        await page.fill(&#8220;#password&#8221;, password)&#13;<br \/>\n&#13;<br \/>\n        # click() fires real mouse events &#8212; mousedown, mouseup, click.&#13;<br \/>\n        # This triggers JavaScript listeners that a plain DOM click misses.&#13;<br \/>\n        await page.click(&#8220;button[type=&#8221;submit&#8221;]&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # Wait for the page to settle after form submission&#13;<br \/>\n        await page.wait_for_load_state(&#8220;networkidle&#8221;)&#13;<br \/>\n&#13;<br \/>\n        # Check which result element appeared&#13;<br \/>\n        success_el = await page.query_selector(&#8220;.flash.success&#8221;)&#13;<br \/>\n        error_el = await page.query_selector(&#8220;.flash.error&#8221;)&#13;<br \/>\n&#13;<br \/>\n        if success_el:&#13;<br \/>\n            message = await success_el.inner_text()&#13;<br \/>\n            result = {&#8220;success&#8221;: True, &#8220;message&#8221;: message.strip()}&#13;<br \/>\n        elif error_el:&#13;<br \/>\n            message = await error_el.inner_text()&#13;<br \/>\n            result = {&#8220;success&#8221;: False, &#8220;message&#8221;: message.strip()}&#13;<br \/>\n        else:&#13;<br \/>\n            result = {&#8220;success&#8221;: False, &#8220;message&#8221;: &#8220;Unknown result&#8221;}&#13;<br \/>\n&#13;<br \/>\n        await browser.close()&#13;<br \/>\n        return result&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def main():&#13;<br \/>\n    # Valid credentials for the demo site&#13;<br \/>\n    result = await login_and_verify(&#8220;tomsmith&#8221;, &#8220;SuperSecretPassword!&#8221;)&#13;<br \/>\n    print(f&#8221;Valid login:   {result}&#8221;)&#13;<br \/>\n&#13;<br \/>\n    # Invalid credentials to verify error handling&#13;<br \/>\n    result_fail = await login_and_verify(&#8220;wronguser&#8221;, &#8220;wrongpass&#8221;)&#13;<br \/>\n    print(f&#8221;Invalid login: {result_fail}&#8221;)&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasyncio.run(main())<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># form_submit.py<\/span><\/p>\n<p><span class=\"crayon-p\"># Complete and submit a multi-field login form on a public demo site.<\/span><\/p>\n<p><span class=\"crayon-p\"># Target: https:\/\/the-internet.herokuapp.com\/login (public test site)<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install playwright &amp;&amp; playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python form_submit.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">login_and_verify<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">username<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">password<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dict<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Attempt to log in to a demo site and return whether it succeeded.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Handles: input filling, button clicking, and result verification.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">with <\/span><span class=\"crayon-e\">async_playwright<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">await<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-st\">goto<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;https:\/\/the-internet.herokuapp.com\/login&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Wait for the form to be visible before interacting.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># state=&#8221;visible&#8221; is the default but makes the intent explicit.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#username&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">state<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;visible&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># fill() clears the field first, then types the value.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># It fires the focus, input, and change events in order.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fill<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#username&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">username<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fill<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#password&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">password<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># click() fires real mouse events &#8212; mousedown, mouseup, click.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># This triggers JavaScript listeners that a plain DOM click misses.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">click<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;button[type=&#8221;submit&#8221;]&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Wait for the page to settle after form submission<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_load_state<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;networkidle&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Check which result element appeared<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">success_el<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.flash.success&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">error_el<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.flash.error&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">success_el<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">message<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">success_el<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">inner_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;success&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;message&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">message<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">strip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">elif <\/span><span class=\"crayon-v\">error_el<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">message<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">error_el<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">inner_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;success&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">False<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;message&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">message<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">strip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;success&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">False<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;message&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Unknown result&#8221;<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">result<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Valid credentials for the demo site<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">login_and_verify<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;tomsmith&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;SuperSecretPassword!&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Valid login:\u00a0\u00a0 {result}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Invalid credentials to verify error handling<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result_fail<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">login_and_verify<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;wronguser&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;wrongpass&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Invalid login: {result_fail}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> The pattern here, <strong>fill() \u2192 click() \u2192 wait_for_load_state() \u2192<\/strong> check for result element, is the template for almost any form interaction. The <strong>wait_for_load_state(\u201cnetworkidle\u201d)<\/strong> after the submit is important: without it, you query the DOM before the page has updated and get the pre-submission state, not the result.<\/p>\n<p>For more complex forms with file uploads, dropdowns, and checkboxes:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa5fd480309442\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# File upload&#13;<br \/>\nawait page.set_input_files(&#8220;#file-upload&#8221;, &#8220;\/path\/to\/document.pdf&#8221;)&#13;<br \/>\n&#13;<br \/>\n# Select dropdown by visible label text&#13;<br \/>\nawait page.select_option(&#8220;#country-select&#8221;, label=&#8221;Nigeria&#8221;)&#13;<br \/>\n&#13;<br \/>\n# Check a checkbox&#13;<br \/>\nawait page.check(&#8220;#agree-terms&#8221;)&#13;<br \/>\n&#13;<br \/>\n# Handle a modal dialog (confirm\/alert)&#13;<br \/>\npage.on(&#8220;dialog&#8221;, lambda dialog: asyncio.ensure_future(dialog.accept()))<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># File upload<\/span><\/p>\n<p><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">set_input_files<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#file-upload&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;\/path\/to\/document.pdf&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Select dropdown by visible label text<\/span><\/p>\n<p><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">select_option<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#country-select&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">label<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;Nigeria&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Check a checkbox<\/span><\/p>\n<p><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">check<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#agree-terms&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Handle a modal dialog (confirm\/alert)<\/span><\/p>\n<p><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">on<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;dialog&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">lambda <\/span><span class=\"crayon-v\">dialog<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">ensure_future<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">dialog<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">accept<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<h2>Tool Orchestration with LangChain and LangGraph<\/h2>\n<p>Raw Playwright scripts are powerful but fixed. They do exactly what you coded, no more. The moment a page changes its structure, or the task requires a decision the script did not anticipate, it breaks.<\/p>\n<p>Connecting Playwright to an LLM changes this. Browser actions become tools the agent can call when it decides they are needed. The agent reads the task, reasons about what to do, calls a tool, reads the result, and decides what to do next. That loop handles variation that a fixed script cannot.<\/p>\n<p>This is the bridge from \u201cbrowser automation script\u201d to \u201c<strong>AI agent<\/strong>.\u201d<\/p>\n<p>How to run: Save as <strong>agent_tools.py<\/strong>, ensure <strong>OPENAI_API_KEY<\/strong> is in your <strong>.env<\/strong>, then run python <strong>agent_tools.py<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa600303903714\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# agent_tools.py&#13;<br \/>\n# LangGraph agent with three browser tools: navigate_and_extract, fill_and_submit_form, take_screenshot&#13;<br \/>\n# Prerequisites: pip install playwright langchain langchain-openai langgraph python-dotenv&#13;<br \/>\n#                playwright install chromium&#13;<br \/>\n# How to run: python agent_tools.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nimport os&#13;<br \/>\nfrom dotenv import load_dotenv&#13;<br \/>\nfrom langchain_openai import ChatOpenAI&#13;<br \/>\nfrom langchain.tools import tool&#13;<br \/>\nfrom langchain_core.messages import HumanMessage&#13;<br \/>\nfrom langgraph.prebuilt import create_react_agent&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nload_dotenv()&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 SHARED BROWSER STATE \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n# We keep a single browser instance alive for the agent&#8217;s lifetime.&#13;<br \/>\n# Creating and destroying a browser on every tool call is slow and wasteful.&#13;<br \/>\n_browser = None&#13;<br \/>\n_page = None&#13;<br \/>\n_playwright = None&#13;<br \/>\n&#13;<br \/>\nasync def get_page():&#13;<br \/>\n    &#8220;&#8221;&#8221;Return the shared page, launching the browser if needed.&#8221;&#8221;&#8221;&#13;<br \/>\n    global _browser, _page, _playwright&#13;<br \/>\n    if _browser is None:&#13;<br \/>\n        _playwright = await async_playwright().start()&#13;<br \/>\n        _browser = await _playwright.chromium.launch(headless=True)&#13;<br \/>\n        context = await _browser.new_context(viewport={&#8220;width&#8221;: 1280, &#8220;height&#8221;: 720})&#13;<br \/>\n        _page = await context.new_page()&#13;<br \/>\n    return _page&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def close_browser():&#13;<br \/>\n    &#8220;&#8221;&#8221;Clean up browser resources when the agent session ends.&#8221;&#8221;&#8221;&#13;<br \/>\n    global _browser, _page, _playwright&#13;<br \/>\n    if _browser:&#13;<br \/>\n        await _browser.close()&#13;<br \/>\n        await _playwright.stop()&#13;<br \/>\n        _browser = None&#13;<br \/>\n        _page = None&#13;<br \/>\n        _playwright = None&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 BROWSER TOOLS \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n# Note: these are async tools (async def). LangChain&#8217;s @tool decorator supports&#13;<br \/>\n# async functions directly, and the agent must be invoked with ainvoke() so that&#13;<br \/>\n# tool calls run on the same event loop instead of trying to start a second one.&#13;<br \/>\n&#13;<br \/>\n@tool&#13;<br \/>\nasync def navigate_and_extract(url: str) -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Navigate to a URL and return the visible text content of the page.&#13;<br \/>\n    Use this to visit websites and read their content.&#13;<br \/>\n    Input: a full URL string including https:\/\/ (e.g., &#8216;https:\/\/example.com&#8217;).&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    page = await get_page()&#13;<br \/>\n    await page.goto(url, wait_until=&#8221;domcontentloaded&#8221;, timeout=15000)&#13;<br \/>\n    await page.wait_for_load_state(&#8220;networkidle&#8221;)&#13;<br \/>\n    content = await page.inner_text(&#8220;body&#8221;)&#13;<br \/>\n    # Truncate to avoid flooding the LLM context window&#13;<br \/>\n    return content[:3000] if len(content) &gt; 3000 else content&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n@tool&#13;<br \/>\nasync def fill_and_submit_form(selector_value_pairs: str) -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Fill form fields and submit a form on the currently loaded page.&#13;<br \/>\n    Input: a comma-separated string of &#8216;selector:value&#8217; pairs ending with &#8216;submit:button_selector&#8217;.&#13;<br \/>\n    Example: &#8216;#email:user@example.com,#password:secret,submit:button[type=submit]&#8217;&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    page = await get_page()&#13;<br \/>\n    try:&#13;<br \/>\n        pairs = selector_value_pairs.split(&#8220;,&#8221;)&#13;<br \/>\n        submit_selector = None&#13;<br \/>\n&#13;<br \/>\n        for pair in pairs:&#13;<br \/>\n            key, val = pair.split(&#8220;:&#8221;, 1)&#13;<br \/>\n            key = key.strip()&#13;<br \/>\n            val = val.strip()&#13;<br \/>\n            if key == &#8220;submit&#8221;:&#13;<br \/>\n                submit_selector = val&#13;<br \/>\n            else:&#13;<br \/>\n                await page.fill(key, val)&#13;<br \/>\n&#13;<br \/>\n        if submit_selector:&#13;<br \/>\n            await page.click(submit_selector)&#13;<br \/>\n            await page.wait_for_load_state(&#8220;networkidle&#8221;)&#13;<br \/>\n&#13;<br \/>\n        return f&#8221;Form submitted. Current URL: {page.url}&#8221;&#13;<br \/>\n    except Exception as e:&#13;<br \/>\n        return f&#8221;Form interaction failed: {str(e)}&#8221;&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n@tool&#13;<br \/>\nasync def take_screenshot(filename: str) -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Take a screenshot of the current browser page and save it to a file.&#13;<br \/>\n    Use this to visually verify the current state of the page.&#13;<br \/>\n    Input: filename string (e.g., &#8216;result.png&#8217;).&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    page = await get_page()&#13;<br \/>\n    await page.screenshot(path=filename, full_page=False)&#13;<br \/>\n    return f&#8221;Screenshot saved to {filename}&#8221;&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 AGENT SETUP \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n&#13;<br \/>\nllm = ChatOpenAI(&#13;<br \/>\n    model=&#8221;gpt-4o&#8221;,&#13;<br \/>\n    temperature=0,&#13;<br \/>\n    api_key=os.getenv(&#8220;OPENAI_API_KEY&#8221;)&#13;<br \/>\n)&#13;<br \/>\n&#13;<br \/>\ntools = [navigate_and_extract, fill_and_submit_form, take_screenshot]&#13;<br \/>\n&#13;<br \/>\n# create_react_agent wires together the LLM, the tools, and the ReAct reasoning loop.&#13;<br \/>\n# The agent decides which tool to call, calls it, reads the result, and continues.&#13;<br \/>\nagent = create_react_agent(llm, tools)&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 DEMO \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n&#13;<br \/>\nasync def main():&#13;<br \/>\n    result = await agent.ainvoke({&#13;<br \/>\n        &#8220;messages&#8221;: [HumanMessage(&#13;<br \/>\n            content=(&#13;<br \/>\n                &#8220;Go to https:\/\/example.com, read the page content, &#8220;&#13;<br \/>\n                &#8220;then take a screenshot called example.png&#8221;&#13;<br \/>\n            )&#13;<br \/>\n        )]&#13;<br \/>\n    })&#13;<br \/>\n    print(result[&#8220;messages&#8221;][-1].content)&#13;<br \/>\n    await close_browser()&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasyncio.run(main())<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>68<\/p>\n<p>69<\/p>\n<p>70<\/p>\n<p>71<\/p>\n<p>72<\/p>\n<p>73<\/p>\n<p>74<\/p>\n<p>75<\/p>\n<p>76<\/p>\n<p>77<\/p>\n<p>78<\/p>\n<p>79<\/p>\n<p>80<\/p>\n<p>81<\/p>\n<p>82<\/p>\n<p>83<\/p>\n<p>84<\/p>\n<p>85<\/p>\n<p>86<\/p>\n<p>87<\/p>\n<p>88<\/p>\n<p>89<\/p>\n<p>90<\/p>\n<p>91<\/p>\n<p>92<\/p>\n<p>93<\/p>\n<p>94<\/p>\n<p>95<\/p>\n<p>96<\/p>\n<p>97<\/p>\n<p>98<\/p>\n<p>99<\/p>\n<p>100<\/p>\n<p>101<\/p>\n<p>102<\/p>\n<p>103<\/p>\n<p>104<\/p>\n<p>105<\/p>\n<p>106<\/p>\n<p>107<\/p>\n<p>108<\/p>\n<p>109<\/p>\n<p>110<\/p>\n<p>111<\/p>\n<p>112<\/p>\n<p>113<\/p>\n<p>114<\/p>\n<p>115<\/p>\n<p>116<\/p>\n<p>117<\/p>\n<p>118<\/p>\n<p>119<\/p>\n<p>120<\/p>\n<p>121<\/p>\n<p>122<\/p>\n<p>123<\/p>\n<p>124<\/p>\n<p>125<\/p>\n<p>126<\/p>\n<p>127<\/p>\n<p>128<\/p>\n<p>129<\/p>\n<p>130<\/p>\n<p>131<\/p>\n<p>132<\/p>\n<p>133<\/p>\n<p>134<\/p>\n<p>135<\/p>\n<p>136<\/p>\n<p>137<\/p>\n<p>138<\/p>\n<p>139<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># agent_tools.py<\/span><\/p>\n<p><span class=\"crayon-p\"># LangGraph agent with three browser tools: navigate_and_extract, fill_and_submit_form, take_screenshot<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install playwright langchain langchain-openai langgraph python-dotenv<\/span><\/p>\n<p><span class=\"crayon-p\">#\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python agent_tools.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">os<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">dotenv <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">load_dotenv<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">langchain_openai <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">ChatOpenAI<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">langchain<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">tools <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">langchain_core<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">messages <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">HumanMessage<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">langgraph<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">prebuilt <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">create_react_agent<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">load_dotenv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 SHARED BROWSER STATE \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p><span class=\"crayon-p\"># We keep a single browser instance alive for the agent&#8217;s lifetime.<\/span><\/p>\n<p><span class=\"crayon-p\"># Creating and destroying a browser on every tool call is slow and wasteful.<\/span><\/p>\n<p><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-v\">_page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Return the shared page, launching the browser if needed.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-m\">global<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_page<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_playwright<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_browser <\/span><span class=\"crayon-st\">is<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">None<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">async_playwright<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">start<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">viewport<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;width&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1280<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;height&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">720<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_page<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">close_browser<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Clean up browser resources when the agent session ends.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-m\">global<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_page<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_playwright<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">stop<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">None<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 BROWSER TOOLS \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p><span class=\"crayon-p\"># Note: these are async tools (async def). LangChain&#8217;s @tool decorator supports<\/span><\/p>\n<p><span class=\"crayon-p\"># async functions directly, and the agent must be invoked with ainvoke() so that<\/span><\/p>\n<p><span class=\"crayon-p\"># tool calls run on the same event loop instead of trying to start a second one.<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-sy\">@<\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">navigate_and_extract<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Navigate to a URL and return the visible text content of the page.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Use this to visit websites and read their content.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Input: a full URL string including https:\/\/ (e.g., &#8216;https:\/\/example.com&#8217;).<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-st\">goto<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wait_until<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;domcontentloaded&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">15000<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_load_state<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;networkidle&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">inner_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;body&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Truncate to avoid flooding the LLM context window<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-cn\">3000<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">len<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3000<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">content<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-sy\">@<\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">fill_and_submit_form<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">selector_value_pairs<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Fill form fields and submit a form on the currently loaded page.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Input: a comma-separated string of &#8216;selector:value&#8217; pairs ending with &#8216;submit:button_selector&#8217;.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Example: &#8216;#email:user@example.com,#password:secret,submit:button[type=submit]&#8217;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">try<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">pairs<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">selector_value_pairs<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">split<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;,&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">submit_selector<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">pair <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pairs<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">val<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">pair<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">split<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;:&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">strip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">val<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">val<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">strip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;submit&#8221;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">submit_selector<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">val<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">fill<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">key<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">val<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">submit_selector<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">click<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">submit_selector<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_load_state<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;networkidle&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Form submitted. Current URL: {page.url}&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">except <\/span><span class=\"crayon-e\">Exception <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">e<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Form interaction failed: {str(e)}&#8221;<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-sy\">@<\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">take_screenshot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Take a screenshot of the current browser page and save it to a file.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Use this to visually verify the current state of the page.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Input: filename string (e.g., &#8216;result.png&#8217;).<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">screenshot<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">path<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">filename<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">full_page<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">False<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Screenshot saved to {filename}&#8221;<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 AGENT SETUP \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">llm<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">ChatOpenAI<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;gpt-4o&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">temperature<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">api_key<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">getenv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;OPENAI_API_KEY&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">tools<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">navigate_and_extract<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">fill_and_submit_form<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">take_screenshot<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># create_react_agent wires together the LLM, the tools, and the ReAct reasoning loop.<\/span><\/p>\n<p><span class=\"crayon-p\"># The agent decides which tool to call, calls it, reads the result, and continues.<\/span><\/p>\n<p><span class=\"crayon-v\">agent<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_react_agent<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">llm<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">tools<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 DEMO \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">agent<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">ainvoke<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">{<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;messages&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-e\">HumanMessage<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Go to https:\/\/example.com, read the page content, &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;then take a screenshot called example.png&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;messages&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">close_browser<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> The three <strong>@tool<\/strong>-decorated functions are registered with the agent. Each docstring is what the LLM reads to understand what the tool does and when to use it. Write them like job descriptions, not code comments. The shared <strong>_browser<\/strong> and <strong>_page<\/strong> globals mean the browser stays open across multiple tool calls, which is essential for tasks that span several pages in the same session. Because the tools are defined with <strong>async def<\/strong>, the agent is invoked with <strong>ainvoke()<\/strong> rather than <strong>invoke()<\/strong>, so the tool calls run on the same event loop that <strong>main()<\/strong> is already using.<\/p>\n<div style=\"width: 810px\" class=\"wp-caption aligncenter\"><a href=\"http:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_508_Building-Browser-Using-AI-Agents-in-Python.png\" target=\"_blank\"><img fetchpriority=\"high\" decoding=\"async\" src=\"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_508_Building-Browser-Using-AI-Agents-in-Python.png\" alt=\"A vertical flow diagram showing how a task request flows through the agent\" width=\"800\" height=\"706\"\/><\/a><\/p>\n<p class=\"wp-caption-text\">A vertical flow diagram showing how a task request flows through the agent (<a href=\"http:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_508_Building-Browser-Using-AI-Agents-in-Python.png\" target=\"_blank\">click to enlarge<\/a>)<br \/>Image by Editor<\/p>\n<\/div>\n<p>The key design decision in this snippet is the shared browser instance. If each tool call launched and closed its own browser, you would lose all session state between calls, such as cookies, navigation history, and any form state the agent had already built up. Keeping the browser alive for the full agent session preserves that context.<\/p>\n<h2>Using browser-use for High-Level Agent Tasks<\/h2>\n<p>Raw Playwright with <strong>@tool<\/strong> functions gives you precise control. The trade-off is that you are still writing selectors, still thinking about page structure, still handling every edge case manually. If the site changes its HTML, your selectors break.<\/p>\n<p><a href=\"https:\/\/github.com\/browser-use\/browser-use\" target=\"_blank\">browser-use<\/a> takes a different approach. Instead of writing selectors, you give the agent a task in plain English. <strong>browser-use<\/strong> uses Playwright under the hood, but the LLM reads the current page state on each step and decides what to do next: which element to click, what to type, and when the task is complete. The page structure is not hardcoded into your code. The agent figures it out at runtime.<\/p>\n<p>browser-use is a Python library that gives an LLM a working browser. The LLM reads each page and decides what to click, type, and extract. This makes it resilient to site changes that would break a selector-based script.<\/p>\n<p>When to use <strong>browser-use<\/strong> over raw Playwright:<\/p>\n<ol>\n<li>If the task is exploratory and the page structure is unpredictable, use <strong>browser-use<\/strong>.<\/li>\n<li>If you are running a fixed, repeatable workflow where every selector is known and stable, raw Playwright is more reliable and cheaper per run.<\/li>\n<li>A <strong>browser-use<\/strong> agent makes multiple LLM calls per task step; a scripted Playwright run makes none.<\/li>\n<\/ol>\n<p><strong>How to run<\/strong>: Save as <strong>browser_use_agent.py<\/strong>, ensure <strong>OPENAI_API_KEY<\/strong> is in your <strong>.env<\/strong>, then run <strong>python browser_use_agent.py<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa604755413748\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# browser_use_agent.py&#13;<br \/>\n# A browser-use agent that accepts a natural language task and completes it&#13;<br \/>\n# without any CSS selectors or hardcoded page structure.&#13;<br \/>\n# Prerequisites: pip install browser-use playwright python-dotenv&#13;<br \/>\n#                playwright install chromium&#13;<br \/>\n# How to run: python browser_use_agent.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nimport os&#13;<br \/>\nfrom dotenv import load_dotenv&#13;<br \/>\nfrom langchain_openai import ChatOpenAI&#13;<br \/>\nfrom browser_use import Agent&#13;<br \/>\n&#13;<br \/>\nload_dotenv()&#13;<br \/>\n&#13;<br \/>\nasync def run_browser_task(task: str) -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Hand a natural language task to a browser-use agent.&#13;<br \/>\n    The agent handles navigation, clicks, and extraction without selectors.&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    # temperature=0 keeps decisions deterministic and reduces hallucinated actions&#13;<br \/>\n    llm = ChatOpenAI(&#13;<br \/>\n        model=&#8221;gpt-4o&#8221;,&#13;<br \/>\n        temperature=0,&#13;<br \/>\n        api_key=os.getenv(&#8220;OPENAI_API_KEY&#8221;)&#13;<br \/>\n    )&#13;<br \/>\n&#13;<br \/>\n    # Agent wraps the browser, the LLM, and the task loop together.&#13;<br \/>\n    # max_actions_per_step limits how many actions the agent takes before&#13;<br \/>\n    # re-reading the page &#8212; prevents runaway loops on complex pages.&#13;<br \/>\n    agent = Agent(&#13;<br \/>\n        task=task,&#13;<br \/>\n        llm=llm,&#13;<br \/>\n        max_actions_per_step=5&#13;<br \/>\n    )&#13;<br \/>\n&#13;<br \/>\n    # run() executes the full task loop:&#13;<br \/>\n    # read page \u2192 decide action \u2192 take action \u2192 read updated page \u2192 repeat&#13;<br \/>\n    result = await agent.run()&#13;<br \/>\n&#13;<br \/>\n    # final_result() returns the agent&#8217;s extracted content or conclusion&#13;<br \/>\n    return result.final_result() or &#8220;Task completed with no extracted output.&#8221;&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def main():&#13;<br \/>\n    task = (&#13;<br \/>\n        &#8220;Go to https:\/\/books.toscrape.com and find the 3 most expensive books &#8220;&#13;<br \/>\n        &#8220;on the first page. Return their titles and prices.&#8221;&#13;<br \/>\n    )&#13;<br \/>\n    print(f&#8221;Task: {task}\\n&#8221;)&#13;<br \/>\n    output = await run_browser_task(task)&#13;<br \/>\n    print(f&#8221;Result:\\n{output}&#8221;)&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasyncio.run(main())<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># browser_use_agent.py<\/span><\/p>\n<p><span class=\"crayon-p\"># A browser-use agent that accepts a natural language task and completes it<\/span><\/p>\n<p><span class=\"crayon-p\"># without any CSS selectors or hardcoded page structure.<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install browser-use playwright python-dotenv<\/span><\/p>\n<p><span class=\"crayon-p\">#\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python browser_use_agent.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">os<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">dotenv <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">load_dotenv<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">langchain_openai <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">ChatOpenAI<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">browser_use <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">Agent<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">load_dotenv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">run_browser_task<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">task<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Hand a natural language task to a browser-use agent.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0The agent handles navigation, clicks, and extraction without selectors.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># temperature=0 keeps decisions deterministic and reduces hallucinated actions<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">llm<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">ChatOpenAI<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;gpt-4o&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">temperature<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">api_key<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">getenv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;OPENAI_API_KEY&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Agent wraps the browser, the LLM, and the task loop together.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># max_actions_per_step limits how many actions the agent takes before<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># re-reading the page &#8212; prevents runaway loops on complex pages.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">agent<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Agent<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">task<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">task<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">llm<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">llm<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">max_actions_per_step<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">5<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># run() executes the full task loop:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># read page \u2192 decide action \u2192 take action \u2192 read updated page \u2192 repeat<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">agent<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># final_result() returns the agent&#8217;s extracted content or conclusion<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">final_result<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">or<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;Task completed with no extracted output.&#8221;<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">task<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Go to https:\/\/books.toscrape.com and find the 3 most expensive books &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;on the first page. Return their titles and prices.&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Task: {task}\\n&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">output<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">run_browser_task<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">task<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Result:\\n{output}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">main<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does<\/strong>: The entire task, navigating to the site, reading the page, identifying the three highest prices, and extracting them, is handled by the agent without a single CSS selector in your code. If <strong>books.toscrape.com<\/strong> redesigns its price display tomorrow, the script still works. With a selector-based scraper, it would break silently.<\/p>\n<p>The <strong>max_actions_per_step=5<\/strong> parameter is worth explaining. On each step, the agent reads the page and can decide to take up to five actions (click, type, scroll, navigate) before re-reading the page. Keeping this low forces the agent to check its work more frequently, which catches mistakes earlier.<\/p>\n<h2>Handling the Hard Parts<\/h2>\n<p>Three things break most browser agents in production. Each has a solution, but none of them is obvious until you have already been burned.<\/p>\n<p><strong>1. Anti-Bot Detection<\/strong><br \/>Websites that do not want to be automated detect automation in several ways, such as checking the <strong>navigator.webdriver<\/strong> property (which Playwright sets to true by default), looking for headless browser fingerprints in the JavaScript environment, and analyzing interaction patterns that are too fast or too uniform to be human.<\/p>\n<p>The most important mitigation is removing the webdriver flag. Beyond that, a realistic user agent string, a standard viewport size, and a realistic locale and timezone cover most detection methods short of sophisticated fingerprint analysis.<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa60d495304044\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# hard_parts.py &#8212; Part 1: Anti-bot stealth launch&#13;<br \/>\n# Prerequisites: pip install playwright &amp;&amp; playwright install chromium&#13;<br \/>\n# How to run: python hard_parts.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nimport json&#13;<br \/>\nfrom pathlib import Path&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nasync def launch_stealth_browser(playwright):&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Launch a browser context that looks more like a real human session.&#13;<br \/>\n    Covers: realistic viewport, user-agent, locale, timezone, webdriver flag.&#13;<br \/>\n    Note: For serious anti-bot targets, consider a paid service like Browserbase.&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    browser = await playwright.chromium.launch(&#13;<br \/>\n        headless=True,&#13;<br \/>\n        args=[&#13;<br \/>\n            &#8220;&#8211;disable-blink-features=AutomationControlled&#8221;,  # Hides webdriver detection&#13;<br \/>\n            &#8220;&#8211;no-sandbox&#8221;,&#13;<br \/>\n            &#8220;&#8211;disable-dev-shm-usage&#8221;,&#13;<br \/>\n        ]&#13;<br \/>\n    )&#13;<br \/>\n&#13;<br \/>\n    context = await browser.new_context(&#13;<br \/>\n        viewport={&#8220;width&#8221;: 1366, &#8220;height&#8221;: 768},   # Common desktop resolution&#13;<br \/>\n        user_agent=(&#13;<br \/>\n            &#8220;Mozilla\/5.0 (Windows NT 10.0; Win64; x64) &#8220;&#13;<br \/>\n            &#8220;AppleWebKit\/537.36 (KHTML, like Gecko) &#8220;&#13;<br \/>\n            &#8220;Chrome\/124.0.0.0 Safari\/537.36&#8243;&#13;<br \/>\n        ),&#13;<br \/>\n        locale=&#8221;en-US&#8221;,&#13;<br \/>\n        timezone_id=&#8221;America\/New_York&#8221;,&#13;<br \/>\n        java_script_enabled=True,&#13;<br \/>\n    )&#13;<br \/>\n&#13;<br \/>\n    # Remove the &#8216;webdriver&#8217; property that Playwright injects by default.&#13;<br \/>\n    # Bot detection systems check for this in the browser&#8217;s JS environment.&#13;<br \/>\n    await context.add_init_script(&#13;<br \/>\n        &#8220;Object.defineProperty(navigator, &#8216;webdriver&#8217;, {get: () =&gt; undefined})&#8221;&#13;<br \/>\n    )&#13;<br \/>\n&#13;<br \/>\n    return browser, context<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># hard_parts.py &#8212; Part 1: Anti-bot stealth launch<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install playwright &amp;&amp; playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python hard_parts.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">json<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">pathlib <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">Path<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">launch_stealth_browser<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Launch a browser context that looks more like a real human session.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Covers: realistic viewport, user-agent, locale, timezone, webdriver flag.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Note: For serious anti-bot targets, consider a paid service like Browserbase.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">args<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8211;disable-blink-features=AutomationControlled&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\">\u00a0\u00a0<\/span><span class=\"crayon-p\"># Hides webdriver detection<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8211;no-sandbox&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8211;disable-dev-shm-usage&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">viewport<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;width&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1366<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;height&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">768<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># Common desktop resolution<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">user_agent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Mozilla\/5.0 (Windows NT 10.0; Win64; x64) &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;AppleWebKit\/537.36 (KHTML, like Gecko) &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Chrome\/124.0.0.0 Safari\/537.36&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">locale<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;en-US&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">timezone_id<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;America\/New_York&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">java_script_enabled<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Remove the &#8216;webdriver&#8217; property that Playwright injects by default.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Bot detection systems check for this in the browser&#8217;s JS environment.<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add_init_script<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Object.defineProperty(navigator, &#8216;webdriver&#8217;, {get: () =&gt; undefined})&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">context<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does<\/strong>: The <strong>add_init_script()<\/strong> call runs before any page JavaScript executes, which means the <strong>navigator.webdriver<\/strong> override is in place before the site\u2019s detection code can check for it. The <strong>\u2013disable-blink-features=AutomationControlled<\/strong> launch argument removes a separate automation flag at the browser engine level. Together, these two changes handle the most common detection methods.<\/p>\n<p>For sites with aggressive fingerprinting and CAPTCHA systems, these mitigations will not be enough. Services like <a href=\"https:\/\/www.browserbase.com\/\" target=\"_blank\">Browserbase<\/a>, <a href=\"http:\/\/www.spidra.io\" target=\"_blank\">Spidra<\/a> and <a href=\"https:\/\/brightdata.com\/products\/scraping-browser\" target=\"_blank\">Brightdata\u2019s Scraping Browser<\/a> handle CAPTCHA solving, residential IP rotation, and browser fingerprint management as managed infrastructure.<\/p>\n<p><strong>2. Smart Waiting<\/strong><\/p>\n<p>The second failure mode is timing. The reflex is to add <strong>time.sleep()<\/strong> calls and increase them when things break. This is wrong in both directions: too short on slow connections, too long on fast ones, and completely opaque when debugging.<\/p>\n<p>Playwright has four proper wait strategies. Use the one that matches what you are actually waiting for:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa610897137021\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# Part 2: Smart waiting strategies (add to your scraper or agent tools)&#13;<br \/>\n&#13;<br \/>\nasync def smart_wait_examples(page):&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Four ways to wait for the right page state, without arbitrary sleeps.&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    # STRATEGY 1: Wait for a specific element to appear in the DOM&#13;<br \/>\n    # Use when you know exactly what element signals content has loaded&#13;<br \/>\n    await page.wait_for_selector(&#8220;.product-list&#8221;, state=&#8221;visible&#8221;, timeout=10000)&#13;<br \/>\n&#13;<br \/>\n    # STRATEGY 2: Wait for a specific API response&#13;<br \/>\n    # Use when the content comes from an XHR\/fetch call you can identify&#13;<br \/>\n    async with page.expect_response(&#13;<br \/>\n        lambda r: &#8220;\/api\/products&#8221; in r.url and r.status == 200&#13;<br \/>\n    ) as response_info:&#13;<br \/>\n        await page.click(&#8220;#load-more&#8221;)&#13;<br \/>\n    response = await response_info.value&#13;<br \/>\n    print(f&#8221;API responded: {response.status}&#8221;)&#13;<br \/>\n&#13;<br \/>\n    # STRATEGY 3: Wait for the URL to change after form submission&#13;<br \/>\n    # Use when a successful submit redirects to a new page&#13;<br \/>\n    await page.wait_for_url(&#8220;**\/dashboard**&#8221;, timeout=10000)&#13;<br \/>\n&#13;<br \/>\n    # STRATEGY 4: Wait for a JavaScript variable to be set&#13;<br \/>\n    # Use when no visual element reliably signals the ready state&#13;<br \/>\n    await page.wait_for_function(&#13;<br \/>\n        &#8220;() =&gt; window.__dataLoaded === true&#8221;,&#13;<br \/>\n        timeout=10000&#13;<br \/>\n    )<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># Part 2: Smart waiting strategies (add to your scraper or agent tools)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">smart_wait_examples<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Four ways to wait for the right page state, without arbitrary sleeps.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># STRATEGY 1: Wait for a specific element to appear in the DOM<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Use when you know exactly what element signals content has loaded<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;.product-list&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">state<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;visible&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10000<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># STRATEGY 2: Wait for a specific API response<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Use when the content comes from an XHR\/fetch call you can identify<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">with <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">expect_response<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-i\">lambda<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">r<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;\/api\/products&#8221;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">r<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">url <\/span><span class=\"crayon-st\">and<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">r<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">status<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">200<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">response_info<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">click<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;#load-more&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">response<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">response_info<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">value<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;API responded: {response.status}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># STRATEGY 3: Wait for the URL to change after form submission<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Use when a successful submit redirects to a new page<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_url<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;**\/dashboard**&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10000<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># STRATEGY 4: Wait for a JavaScript variable to be set<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Use when no visual element reliably signals the ready state<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_function<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;() =&gt; window.__dataLoaded === true&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">10000<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> Each strategy is tied to a specific observable event rather than an arbitrary time delay. <strong>wait_for_selector<\/strong> watches the DOM. <strong>expect_response<\/strong> hooks into the network layer. <strong>wait_for_url<\/strong> monitors navigation. <strong>wait_for_function<\/strong> evaluates JavaScript in the browser context. Use whichever one most directly signals \u201cthe thing I need is now ready.\u201d<\/p>\n<p><strong>3. Session and Cookie Persistence<\/strong><br \/>The third failure mode is losing session state. If your agent logs into a site during step one and then the browser context is destroyed, step two has no authentication. Recreating the login on every run is slow and can trigger rate limiting or lockout.<\/p>\n<p>The solution is saving cookies to disk after login and loading them at the start of every subsequent run:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa616550746946\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# Part 3: Session persistence across runs&#13;<br \/>\n&#13;<br \/>\nCOOKIES_FILE = Path(&#8220;session_cookies.json&#8221;)&#13;<br \/>\n&#13;<br \/>\nasync def save_session(context) -&gt; None:&#13;<br \/>\n    &#8220;&#8221;&#8221;Save browser cookies to disk after a successful login.&#8221;&#8221;&#8221;&#13;<br \/>\n    cookies = await context.cookies()&#13;<br \/>\n    COOKIES_FILE.write_text(json.dumps(cookies, indent=2))&#13;<br \/>\n    print(f&#8221;Session saved: {len(cookies)} cookies written.&#8221;)&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def load_session(context) -&gt; bool:&#13;<br \/>\n    &#8220;&#8221;&#8221;Load saved cookies before navigating. Returns True if session was found.&#8221;&#8221;&#8221;&#13;<br \/>\n    if not COOKIES_FILE.exists():&#13;<br \/>\n        print(&#8220;No saved session. Fresh login required.&#8221;)&#13;<br \/>\n        return False&#13;<br \/>\n    cookies = json.loads(COOKIES_FILE.read_text())&#13;<br \/>\n    await context.add_cookies(cookies)&#13;<br \/>\n    print(f&#8221;Session restored: {len(cookies)} cookies loaded.&#8221;)&#13;<br \/>\n    return True<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># Part 3: Session persistence across runs<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">COOKIES_FILE<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">Path<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;session_cookies.json&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">save_session<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">None<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Save browser cookies to disk after a successful login.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">cookies<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">cookies<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">COOKIES_FILE<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">write_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">json<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">dumps<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">cookies<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">indent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">2<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Session saved: {len(cookies)} cookies written.&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">load_session<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">bool<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Load saved cookies before navigating. Returns True if session was found.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">not<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">COOKIES_FILE<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">exists<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;No saved session. Fresh login required.&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">False<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">cookies<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">json<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">loads<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">COOKIES_FILE<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">read_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add_cookies<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">cookies<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Session restored: {len(cookies)} cookies loaded.&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">True<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> <strong>context.cookies()<\/strong> returns all cookies for the current browser context, including session tokens and authentication cookies. Writing them to JSON and reloading them on the next run means the browser starts in an authenticated state. Note that sessions expire; add a check that falls back to a fresh login if the saved session returns a redirect to the login page.<\/p>\n<h2>Deploying Browser Agents<\/h2>\n<p>Getting a browser agent working locally is one thing. Running it reliably in a cloud environment is another.<\/p>\n<p>The main difference between a Python script that works on your laptop and one that fails in CI is system dependencies. Playwright\u2019s Chromium browser requires a set of shared libraries that are present on most developer machines but absent from minimal cloud images. The cleanest solution is Docker.<\/p>\n<p><strong>Dockerfile<\/strong> \u2014 build a container that ships everything Playwright needs:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa61b381747250\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# Dockerfile for headless Playwright-based browser agent&#13;<br \/>\n# Build: docker build -t browser-agent .&#13;<br \/>\n# Run:   docker run &#8211;rm -e OPENAI_API_KEY=your_key browser-agent&#13;<br \/>\n&#13;<br \/>\nFROM python:3.11-slim&#13;<br \/>\n&#13;<br \/>\n# Install system dependencies required by Chromium&#13;<br \/>\nRUN apt-get update &amp;&amp; apt-get install -y \\&#13;<br \/>\n    libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 \\&#13;<br \/>\n    libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \\&#13;<br \/>\n    libxrandr2 libgbm1 libasound2 libpangocairo-1.0-0 \\&#13;<br \/>\n    libpango-1.0-0 libcairo2 libx11-6 libxext6 libxfixes3 \\&#13;<br \/>\n    fonts-liberation wget ca-certificates \\&#13;<br \/>\n    &amp;&amp; rm -rf \/var\/lib\/apt\/lists\/*&#13;<br \/>\n&#13;<br \/>\nWORKDIR \/app&#13;<br \/>\n&#13;<br \/>\n# Install Python dependencies first (cached layer &#8212; only rebuilds on requirements change)&#13;<br \/>\nCOPY requirements.txt .&#13;<br \/>\nRUN pip install &#8211;no-cache-dir -r requirements.txt&#13;<br \/>\n&#13;<br \/>\n# Install Playwright browser binaries into the image&#13;<br \/>\nRUN playwright install chromium&#13;<br \/>\nRUN playwright install-deps chromium&#13;<br \/>\n&#13;<br \/>\n# Copy application code last (changes here don&#8217;t invalidate the pip\/playwright layers)&#13;<br \/>\nCOPY . .&#13;<br \/>\n&#13;<br \/>\nCMD [&#8220;python&#8221;, &#8220;agent_tools.py&#8221;]&#13;<br \/>\n&#13;<br \/>\nrequirements.txt:&#13;<br \/>\nplaywright&#13;<br \/>\nbrowser-use&#13;<br \/>\nlangchain&#13;<br \/>\nlangchain-openai&#13;<br \/>\nlanggraph&#13;<br \/>\npython-dotenv<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># Dockerfile for headless Playwright-based browser agent<\/span><\/p>\n<p><span class=\"crayon-p\"># Build: docker build -t browser-agent .<\/span><\/p>\n<p><span class=\"crayon-p\"># Run:\u00a0\u00a0 docker run &#8211;rm -e OPENAI_API_KEY=your_key browser-agent<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">FROM <\/span><span class=\"crayon-v\">python<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-cn\">3.11<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">slim<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Install system dependencies required by Chromium<\/span><\/p>\n<p><span class=\"crayon-e\">RUN <\/span><span class=\"crayon-v\">apt<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-e\">get <\/span><span class=\"crayon-v\">update<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&amp;&amp;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">apt<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-e\">get <\/span><span class=\"crayon-v\">install<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">y<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">libnss3 <\/span><span class=\"crayon-v\">libatk1<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">libatk<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">bridge2<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">libcups2<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">libdrm2 <\/span><span class=\"crayon-e\">libxkbcommon0 <\/span><span class=\"crayon-e\">libxcomposite1 <\/span><span class=\"crayon-i\">libxdamage1<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">libxrandr2 <\/span><span class=\"crayon-e\">libgbm1 <\/span><span class=\"crayon-e\">libasound2 <\/span><span class=\"crayon-v\">libpangocairo<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1.0<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">libpango<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1.0<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">libcairo2 <\/span><span class=\"crayon-v\">libx11<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">6<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">libxext6 <\/span><span class=\"crayon-i\">libxfixes3<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">fonts<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-e\">liberation <\/span><span class=\"crayon-e\">wget <\/span><span class=\"crayon-v\">ca<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">certificates<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">\\<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-o\">&amp;&amp;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">rm<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">rf<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-t\">var<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-v\">lib<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-v\">apt<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-v\">lists<\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-o\">*<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">WORKDIR<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">\/<\/span><span class=\"crayon-i\">app<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Install Python dependencies first (cached layer &#8212; only rebuilds on requirements change)<\/span><\/p>\n<p><span class=\"crayon-e\">COPY <\/span><span class=\"crayon-v\">requirements<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">txt<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">.<\/span><\/p>\n<p><span class=\"crayon-e\">RUN <\/span><span class=\"crayon-e\">pip <\/span><span class=\"crayon-v\">install<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&#8212;<\/span><span class=\"crayon-v\">no<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">cache<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">dir<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-i\">r<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">requirements<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">txt<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Install Playwright browser binaries into the image<\/span><\/p>\n<p><span class=\"crayon-e\">RUN <\/span><span class=\"crayon-e\">playwright <\/span><span class=\"crayon-e\">install <\/span><span class=\"crayon-e\">chromium<\/span><\/p>\n<p><span class=\"crayon-e\">RUN <\/span><span class=\"crayon-e\">playwright <\/span><span class=\"crayon-v\">install<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-e\">deps <\/span><span class=\"crayon-i\">chromium<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># Copy application code last (changes here don&#8217;t invalidate the pip\/playwright layers)<\/span><\/p>\n<p><span class=\"crayon-i\">COPY<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">.<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-i\">CMD<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;python&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;agent_tools.py&#8221;<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">requirements<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">txt<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-e\">playwright<\/span><\/p>\n<p><span class=\"crayon-v\">browser<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-st\">use<\/span><\/p>\n<p><span class=\"crayon-e\">langchain<\/span><\/p>\n<p><span class=\"crayon-v\">langchain<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-e\">openai<\/span><\/p>\n<p><span class=\"crayon-e\">langgraph<\/span><\/p>\n<p><span class=\"crayon-v\">python<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-v\">dotenv<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p>For concurrent workloads running multiple browser sessions in parallel, use Playwright\u2019s async API with <strong>asyncio.gather()<\/strong>:<\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa61e740061990\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# Parallel scraping with semaphore rate limiting&#13;<br \/>\n# Runs up to 3 browser sessions simultaneously&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nasync def scrape_url(browser, url: str, semaphore: asyncio.Semaphore) -&gt; dict:&#13;<br \/>\n    &#8220;&#8221;&#8221;Scrape a single URL, respecting the concurrency semaphore.&#8221;&#8221;&#8221;&#13;<br \/>\n    async with semaphore:&#13;<br \/>\n        context = await browser.new_context()&#13;<br \/>\n        page = await context.new_page()&#13;<br \/>\n        await page.goto(url, wait_until=&#8221;domcontentloaded&#8221;)&#13;<br \/>\n        title = await page.title()&#13;<br \/>\n        await context.close()   # Close context (not browser) to release resources&#13;<br \/>\n        return {&#8220;url&#8221;: url, &#8220;title&#8221;: title}&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def scrape_parallel(urls: list[str], max_concurrent: int = 3) -&gt; list[dict]:&#13;<br \/>\n    &#8220;&#8221;&#8221;Scrape a list of URLs in parallel, capped at max_concurrent sessions.&#8221;&#8221;&#8221;&#13;<br \/>\n    semaphore = asyncio.Semaphore(max_concurrent)  # Cap concurrent sessions&#13;<br \/>\n&#13;<br \/>\n    async with async_playwright() as p:&#13;<br \/>\n        # One browser shared across all contexts &#8212; much cheaper than one browser per URL&#13;<br \/>\n        browser = await p.chromium.launch(headless=True)&#13;<br \/>\n        tasks = [scrape_url(browser, url, semaphore) for url in urls]&#13;<br \/>\n        results = await asyncio.gather(*tasks)&#13;<br \/>\n        await browser.close()&#13;<br \/>\n&#13;<br \/>\n    return list(results)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># Parallel scraping with semaphore rate limiting<\/span><\/p>\n<p><span class=\"crayon-p\"># Runs up to 3 browser sessions simultaneously<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">scrape_url<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">semaphore<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">Semaphore<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">dict<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Scrape a single URL, respecting the concurrency semaphore.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">with <\/span><span class=\"crayon-v\">semaphore<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-st\">goto<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wait_until<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;domcontentloaded&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">title<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">title<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0 <\/span><span class=\"crayon-p\"># Close context (not browser) to release resources<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;url&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;title&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">title<\/span><span class=\"crayon-sy\">}<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">scrape_parallel<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">urls<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">list<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">max_concurrent<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-t\">int<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">3<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">list<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">dict<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Scrape a list of URLs in parallel, capped at max_concurrent sessions.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">semaphore<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">Semaphore<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">max_concurrent<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\">\u00a0\u00a0<\/span><span class=\"crayon-p\"># Cap concurrent sessions<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">with <\/span><span class=\"crayon-e\">async_playwright<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># One browser shared across all contexts &#8212; much cheaper than one browser per URL<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">await<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">p<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">tasks<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-e\">scrape_url<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">semaphore<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">url <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">urls<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">results<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">gather<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-o\">*<\/span><span class=\"crayon-v\">tasks<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">list<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">results<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does<\/strong>: The <strong>asyncio.Semaphore(max_concurrent)<\/strong> caps how many browser contexts run at the same time. Without it, launching 50 concurrent browser contexts will exhaust memory. One browser process is shared across all contexts; a context is cheap; a full browser instance is not.<\/p>\n<p>On the managed infrastructure side, <a href=\"https:\/\/aws.amazon.com\/nova\/act\/\" target=\"_blank\">Amazon Nova Act<\/a> launched in March 2025 as a dedicated SDK for building browser agents on AWS, integrating natively with Playwright for browser control. <a href=\"https:\/\/playwright.dev\/python\/\" target=\"_blank\">Playwright\u2019s own MCP server<\/a> gives AI assistants full browser control through the Model Context Protocol, using structured accessibility snapshots rather than screenshots, which means token costs stay low while the agent\u2019s understanding of the page stays high.<\/p>\n<h2>Putting It All Together<\/h2>\n<p>Here is a complete end-to-end agent that takes a research question, navigates to a public data source, extracts structured results, and returns a clean summary. It uses the browser tools from Section 5 orchestrated by a LangGraph agent.<\/p>\n<p><strong>How to run:<\/strong> Save as <strong>reference_agent.py<\/strong>, ensure <strong>OPENAI_API_KEY<\/strong> is in your <strong>.env<\/strong>, and run python <strong>reference_agent.py<\/strong><\/p>\n<div id=\"urvanov-syntax-highlighter-6a3d6b51aa621288452372\" class=\"urvanov-syntax-highlighter-syntax crayon-theme-classic urvanov-syntax-highlighter-font-monaco urvanov-syntax-highlighter-os-pc print-yes notranslate\" data-settings=\" touchscreen minimize scroll-mouseover disable-anim\" style=\" margin-top: 12px; margin-bottom: 12px; font-size: 12px !important; line-height: 15px !important;\">\n<p><textarea wrap=\"soft\" class=\"urvanov-syntax-highlighter-plain print-no\" data-settings=\"dblclick\" style=\"-moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4; font-size: 12px !important; line-height: 15px !important;\"><br \/>\n# reference_agent.py&#13;<br \/>\n# Full browser-using AI agent: navigates, extracts, summarizes.&#13;<br \/>\n# Target: books.toscrape.com (public scraping sandbox)&#13;<br \/>\n# Prerequisites: pip install playwright langchain langchain-openai langgraph python-dotenv&#13;<br \/>\n#                playwright install chromium&#13;<br \/>\n# How to run: python reference_agent.py&#13;<br \/>\n&#13;<br \/>\nimport asyncio&#13;<br \/>\nimport os&#13;<br \/>\nfrom dotenv import load_dotenv&#13;<br \/>\nfrom langchain_openai import ChatOpenAI&#13;<br \/>\nfrom langchain.tools import tool&#13;<br \/>\nfrom langchain_core.messages import HumanMessage, SystemMessage&#13;<br \/>\nfrom langgraph.prebuilt import create_react_agent&#13;<br \/>\nfrom playwright.async_api import async_playwright&#13;<br \/>\n&#13;<br \/>\nload_dotenv()&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 BROWSER STATE \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n_browser = None&#13;<br \/>\n_context = None&#13;<br \/>\n_page = None&#13;<br \/>\n_playwright = None&#13;<br \/>\n&#13;<br \/>\nasync def get_page():&#13;<br \/>\n    global _browser, _context, _page, _playwright&#13;<br \/>\n    if _browser is None:&#13;<br \/>\n        _playwright = await async_playwright().start()&#13;<br \/>\n        _browser = await _playwright.chromium.launch(headless=True)&#13;<br \/>\n        _context = await _browser.new_context(&#13;<br \/>\n            viewport={&#8220;width&#8221;: 1280, &#8220;height&#8221;: 720},&#13;<br \/>\n            user_agent=(&#13;<br \/>\n                &#8220;Mozilla\/5.0 (Windows NT 10.0; Win64; x64) &#8220;&#13;<br \/>\n                &#8220;AppleWebKit\/537.36 (KHTML, like Gecko) &#8220;&#13;<br \/>\n                &#8220;Chrome\/120.0.0.0 Safari\/537.36&#8243;&#13;<br \/>\n            )&#13;<br \/>\n        )&#13;<br \/>\n        # Remove webdriver fingerprint&#13;<br \/>\n        await _context.add_init_script(&#13;<br \/>\n            &#8220;Object.defineProperty(navigator, &#8216;webdriver&#8217;, {get: () =&gt; undefined})&#8221;&#13;<br \/>\n        )&#13;<br \/>\n        _page = await _context.new_page()&#13;<br \/>\n    return _page&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def teardown():&#13;<br \/>\n    global _browser, _playwright&#13;<br \/>\n    if _browser:&#13;<br \/>\n        await _browser.close()&#13;<br \/>\n        await _playwright.stop()&#13;<br \/>\n        _browser = None&#13;<br \/>\n        _playwright = None&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 TOOLS \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n&#13;<br \/>\n@tool&#13;<br \/>\nasync def navigate(url: str) -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Navigate the browser to a URL and return the page&#8217;s text content.&#13;<br \/>\n    Use when you need to open a website or move to a new page.&#13;<br \/>\n    Input: full URL with https:\/\/ prefix.&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    page = await get_page()&#13;<br \/>\n    await page.goto(url, wait_until=&#8221;domcontentloaded&#8221;, timeout=20000)&#13;<br \/>\n    await page.wait_for_load_state(&#8220;networkidle&#8221;)&#13;<br \/>\n    content = await page.inner_text(&#8220;body&#8221;)&#13;<br \/>\n    return content[:4000]&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n@tool&#13;<br \/>\nasync def extract_structured(css_selector: str) -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    Extract text from all elements matching a CSS selector on the current page.&#13;<br \/>\n    Use when you need to pull specific elements from the loaded page.&#13;<br \/>\n    Input: valid CSS selector string (e.g., &#8216;h3 a&#8217;, &#8216;.price_color&#8217;, &#8216;article.product_pod&#8217;).&#13;<br \/>\n    &#8220;&#8221;&#8221;&#13;<br \/>\n    page = await get_page()&#13;<br \/>\n    try:&#13;<br \/>\n        await page.wait_for_selector(css_selector, timeout=5000)&#13;<br \/>\n        elements = await page.query_selector_all(css_selector)&#13;<br \/>\n        texts = []&#13;<br \/>\n        for el in elements[:20]:  # Cap at 20 elements to keep output manageable&#13;<br \/>\n            text = await el.inner_text()&#13;<br \/>\n            texts.append(text.strip())&#13;<br \/>\n        return &#8220;\\n&#8221;.join(texts) if texts else &#8220;No elements found.&#8221;&#13;<br \/>\n    except Exception as e:&#13;<br \/>\n        return f&#8221;Extraction failed: {str(e)}&#8221;&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n@tool&#13;<br \/>\nasync def get_current_url() -&gt; str:&#13;<br \/>\n    &#8220;&#8221;&#8221;Return the URL the browser is currently on. No input required.&#8221;&#8221;&#8221;&#13;<br \/>\n    page = await get_page()&#13;<br \/>\n    return page.url&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 AGENT \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n&#13;<br \/>\nllm = ChatOpenAI(&#13;<br \/>\n    model=&#8221;gpt-4o&#8221;,&#13;<br \/>\n    temperature=0,&#13;<br \/>\n    api_key=os.getenv(&#8220;OPENAI_API_KEY&#8221;)&#13;<br \/>\n)&#13;<br \/>\n&#13;<br \/>\ntools = [navigate, extract_structured, get_current_url]&#13;<br \/>\nagent = create_react_agent(llm, tools)&#13;<br \/>\n&#13;<br \/>\nSYSTEM = (&#13;<br \/>\n    &#8220;You are a browser-based research agent. You have access to a real browser. &#8220;&#13;<br \/>\n    &#8220;Use navigate() to open pages, extract_structured() to pull specific elements, &#8220;&#13;<br \/>\n    &#8220;and get_current_url() to check where you are. &#8220;&#13;<br \/>\n    &#8220;Always navigate first, then extract. Be concise in your final answer.&#8221;&#13;<br \/>\n)&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\nasync def run_agent(query: str) -&gt; str:&#13;<br \/>\n    result = await agent.ainvoke({&#13;<br \/>\n        &#8220;messages&#8221;: [&#13;<br \/>\n            SystemMessage(content=SYSTEM),&#13;<br \/>\n            HumanMessage(content=query)&#13;<br \/>\n        ]&#13;<br \/>\n    })&#13;<br \/>\n    await teardown()&#13;<br \/>\n    return result[&#8220;messages&#8221;][-1].content&#13;<br \/>\n&#13;<br \/>\n&#13;<br \/>\n# \u2500\u2500 DEMO \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500&#13;<br \/>\n&#13;<br \/>\nif __name__ == &#8220;__main__&#8221;:&#13;<br \/>\n    query = (&#13;<br \/>\n        &#8220;Go to https:\/\/books.toscrape.com and extract the titles and prices &#8220;&#13;<br \/>\n        &#8220;of the first 5 books listed. Return them as a structured list.&#8221;&#13;<br \/>\n    )&#13;<br \/>\n    print(f&#8221;Query: {query}\\n&#8221;)&#13;<br \/>\n    answer = asyncio.run(run_agent(query))&#13;<br \/>\n    print(f&#8221;Answer:\\n{answer}&#8221;)<\/textarea><\/p>\n<div class=\"urvanov-syntax-highlighter-main\" style=\"\">\n<table class=\"crayon-table\">\n<tr class=\"urvanov-syntax-highlighter-row\">\n<td class=\"crayon-nums \" data-settings=\"show\">\n<div class=\"urvanov-syntax-highlighter-nums-content\" style=\"font-size: 12px !important; line-height: 15px !important;\">\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>68<\/p>\n<p>69<\/p>\n<p>70<\/p>\n<p>71<\/p>\n<p>72<\/p>\n<p>73<\/p>\n<p>74<\/p>\n<p>75<\/p>\n<p>76<\/p>\n<p>77<\/p>\n<p>78<\/p>\n<p>79<\/p>\n<p>80<\/p>\n<p>81<\/p>\n<p>82<\/p>\n<p>83<\/p>\n<p>84<\/p>\n<p>85<\/p>\n<p>86<\/p>\n<p>87<\/p>\n<p>88<\/p>\n<p>89<\/p>\n<p>90<\/p>\n<p>91<\/p>\n<p>92<\/p>\n<p>93<\/p>\n<p>94<\/p>\n<p>95<\/p>\n<p>96<\/p>\n<p>97<\/p>\n<p>98<\/p>\n<p>99<\/p>\n<p>100<\/p>\n<p>101<\/p>\n<p>102<\/p>\n<p>103<\/p>\n<p>104<\/p>\n<p>105<\/p>\n<p>106<\/p>\n<p>107<\/p>\n<p>108<\/p>\n<p>109<\/p>\n<p>110<\/p>\n<p>111<\/p>\n<p>112<\/p>\n<p>113<\/p>\n<p>114<\/p>\n<p>115<\/p>\n<p>116<\/p>\n<p>117<\/p>\n<p>118<\/p>\n<p>119<\/p>\n<p>120<\/p>\n<p>121<\/p>\n<p>122<\/p>\n<p>123<\/p>\n<p>124<\/p>\n<p>125<\/p>\n<p>126<\/p>\n<p>127<\/p>\n<p>128<\/p>\n<p>129<\/p>\n<p>130<\/p>\n<p>131<\/p>\n<p>132<\/p>\n<p>133<\/p>\n<p>134<\/p>\n<p>135<\/p>\n<p>136<\/p>\n<p>137<\/p>\n<\/div>\n<\/td>\n<td class=\"urvanov-syntax-highlighter-code\">\n<div class=\"crayon-pre\" style=\"font-size: 12px !important; line-height: 15px !important; -moz-tab-size:4; -o-tab-size:4; -webkit-tab-size:4; tab-size:4;\">\n<p><span class=\"crayon-p\"># reference_agent.py<\/span><\/p>\n<p><span class=\"crayon-p\"># Full browser-using AI agent: navigates, extracts, summarizes.<\/span><\/p>\n<p><span class=\"crayon-p\"># Target: books.toscrape.com (public scraping sandbox)<\/span><\/p>\n<p><span class=\"crayon-p\"># Prerequisites: pip install playwright langchain langchain-openai langgraph python-dotenv<\/span><\/p>\n<p><span class=\"crayon-p\">#\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0playwright install chromium<\/span><\/p>\n<p><span class=\"crayon-p\"># How to run: python reference_agent.py<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">asyncio<\/span><\/p>\n<p><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">os<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">dotenv <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">load_dotenv<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-e\">langchain_openai <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">ChatOpenAI<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">langchain<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">tools <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">langchain_core<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">messages <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-v\">HumanMessage<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">SystemMessage<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">langgraph<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">prebuilt <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">create_react_agent<\/span><\/p>\n<p><span class=\"crayon-e\">from <\/span><span class=\"crayon-v\">playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">async_api <\/span><span class=\"crayon-e\">import <\/span><span class=\"crayon-e\">async_playwright<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">load_dotenv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 BROWSER STATE \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-v\">_context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-v\">_page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-m\">global<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_context<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_page<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_playwright<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_browser <\/span><span class=\"crayon-st\">is<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">None<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">async_playwright<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">start<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-v\">chromium<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">launch<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">headless<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-t\">True<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_context<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_context<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">viewport<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">{<\/span><span class=\"crayon-s\">&#8220;width&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">1280<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;height&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-cn\">720<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">user_agent<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Mozilla\/5.0 (Windows NT 10.0; Win64; x64) &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;AppleWebKit\/537.36 (KHTML, like Gecko) &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Chrome\/120.0.0.0 Safari\/537.36&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-p\"># Remove webdriver fingerprint<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">add_init_script<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Object.defineProperty(navigator, &#8216;webdriver&#8217;, {get: () =&gt; undefined})&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_context<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">new_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_page<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">teardown<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-m\">global<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">_playwright<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">close<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">stop<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_browser<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">None<\/span><\/p>\n<p><span class=\"crayon-e\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">_playwright<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">None<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 TOOLS \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-sy\">@<\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">navigate<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Navigate the browser to a URL and return the page&#8217;s text content.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Use when you need to open a website or move to a new page.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Input: full URL with https:\/\/ prefix.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-st\">goto<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">url<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">wait_until<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;domcontentloaded&#8221;<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">20000<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_load_state<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;networkidle&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">inner_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;body&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-cn\">4000<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-sy\">@<\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">extract_structured<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">css_selector<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Extract text from all elements matching a CSS selector on the current page.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Use when you need to pull specific elements from the loaded page.<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0Input: valid CSS selector string (e.g., &#8216;h3 a&#8217;, &#8216;.price_color&#8217;, &#8216;article.product_pod&#8217;).<\/span><\/p>\n<p><span class=\"crayon-s\">\u00a0\u00a0\u00a0\u00a0&#8220;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">try<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">wait_for_selector<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">css_selector<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">timeout<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">5000<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">elements<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">query_selector_all<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">css_selector<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">texts<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">for<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">el <\/span><span class=\"crayon-st\">in<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">elements<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-cn\">20<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\">\u00a0\u00a0<\/span><span class=\"crayon-p\"># Cap at 20 elements to keep output manageable<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">text<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">el<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">inner_text<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">texts<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">append<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">text<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">strip<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;\\n&#8221;<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">join<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">texts<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">texts <\/span><span class=\"crayon-st\">else<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;No elements found.&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">except <\/span><span class=\"crayon-e\">Exception <\/span><span class=\"crayon-st\">as<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">e<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Extraction failed: {str(e)}&#8221;<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-sy\">@<\/span><span class=\"crayon-e\">tool<\/span><\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">get_current_url<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><span class=\"crayon-s\">&#8220;Return the URL the browser is currently on. No input required.&#8221;<\/span><span class=\"crayon-s\">&#8220;&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">get_page<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">page<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">url<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 AGENT \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">llm<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">ChatOpenAI<\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">model<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-s\">&#8220;gpt-4o&#8221;<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">temperature<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-cn\">0<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">api_key<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">os<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">getenv<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-s\">&#8220;OPENAI_API_KEY&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">tools<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-v\">navigate<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">extract_structured<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">get_current_url<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-v\">agent<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">create_react_agent<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">llm<\/span><span class=\"crayon-sy\">,<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">tools<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-v\">SYSTEM<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;You are a browser-based research agent. You have access to a real browser. &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Use navigate() to open pages, extract_structured() to pull specific elements, &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;and get_current_url() to check where you are. &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Always navigate first, then extract. Be concise in your final answer.&#8221;<\/span><\/p>\n<p><span class=\"crayon-sy\">)<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-e\">async <\/span><span class=\"crayon-e\">def <\/span><span class=\"crayon-e\">run_agent<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">query<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">-&gt;<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">str<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-v\">agent<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">ainvoke<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">{<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;messages&#8221;<\/span><span class=\"crayon-o\">:<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">[<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">SystemMessage<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">SYSTEM<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">,<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">HumanMessage<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">content<\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-v\">query<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">]<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">}<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">await <\/span><span class=\"crayon-e\">teardown<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-st\">return<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">result<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-s\">&#8220;messages&#8221;<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">[<\/span><span class=\"crayon-o\">&#8211;<\/span><span class=\"crayon-cn\">1<\/span><span class=\"crayon-sy\">]<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-i\">content<\/span><\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-p\"># \u2500\u2500 DEMO \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500<\/span><\/p>\n<p>\u00a0<\/p>\n<p><span class=\"crayon-st\">if<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">__name__<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">==<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-s\">&#8220;__main__&#8221;<\/span><span class=\"crayon-o\">:<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">query<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-sy\">(<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;Go to https:\/\/books.toscrape.com and extract the titles and prices &#8220;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-s\">&#8220;of the first 5 books listed. Return them as a structured list.&#8221;<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Query: {query}\\n&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-v\">answer<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-o\">=<\/span><span class=\"crayon-h\"> <\/span><span class=\"crayon-v\">asyncio<\/span><span class=\"crayon-sy\">.<\/span><span class=\"crayon-e\">run<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-e\">run_agent<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-v\">query<\/span><span class=\"crayon-sy\">)<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<p><span class=\"crayon-h\">\u00a0\u00a0\u00a0\u00a0<\/span><span class=\"crayon-e\">print<\/span><span class=\"crayon-sy\">(<\/span><span class=\"crayon-i\">f<\/span><span class=\"crayon-s\">&#8220;Answer:\\n{answer}&#8221;<\/span><span class=\"crayon-sy\">)<\/span><\/p>\n<\/div>\n<\/td>\n<\/tr>\n<\/table><\/div>\n<\/p><\/div>\n<p><strong>What this does:<\/strong> This agent has three clean tools: navigate, <strong>extract_structured<\/strong>, and <strong>get_current_url<\/strong>, plus a system prompt that tells it exactly when to use each one. The agent calls navigate to load the page, <strong>extract_structured<\/strong> to pull the book titles and prices by CSS selector, and synthesizes a structured list in the final answer. The <strong>teardown()<\/strong> call after the agent finishes closes the browser cleanly so no zombie Chromium processes are left running.<\/p>\n<h2>Conclusion<\/h2>\n<p>The browser is not a specialized tool for automation engineers. It is the universal interface for the web, and the web is where most of the world\u2019s actual work gets done. An AI agent that can use a browser does not need a partner team maintaining API integrations. It can reach anything a human can reach.<\/p>\n<p>What makes this practical now, not just theoretically interesting, is the maturity of the tooling. Playwright handles the hard parts of browser interaction. browser-use removes the need to write selectors for exploratory tasks. LangGraph gives the LLM clean tool hooks and a reasoning loop that handles variable page structures. The patterns in this article are not demos. They are the same patterns 51% of enterprises now running AI agents in production are building on.<\/p>\n<p>Start with the scraping example. Get it running against a site you actually need data from. Add the agent layer when you need decisions the script cannot anticipate. Add browser-use when the page structure is too dynamic for selectors. Deploy in Docker when you need it running somewhere other than your laptop.<\/p>\n<p>The hard part is not the code. It is knowing which tool to reach for at each layer. Hopefully this article made that clearer.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>In this article, you will learn how to build AI agents that can browse and interact with real websites using Playwright, browser-use, and LangGraph. Topics we will cover include: Why Playwright is the right foundation for browser automation in 2026, and how it differs from Selenium. How to scrape dynamic, JavaScript-rendered pages and complete multi-step<\/p>\n","protected":false},"author":1,"featured_media":180467,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[42],"tags":[],"class_list":{"0":"post-180466","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-ai"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Building Browser-Using AI Agents in Python - Ktromedia<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/ktromedia.com\/?p=180466\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building Browser-Using AI Agents in Python - Ktromedia\" \/>\n<meta property=\"og:description\" content=\"In this article, you will learn how to build AI agents that can browse and interact with real websites using Playwright, browser-use, and LangGraph. Topics we will cover include: Why Playwright is the right foundation for browser automation in 2026, and how it differs from Selenium. How to scrape dynamic, JavaScript-rendered pages and complete multi-step\" \/>\n<meta property=\"og:url\" content=\"http:\/\/ktromedia.com\/?p=180466\" \/>\n<meta property=\"og:site_name\" content=\"Ktromedia\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/KTROMedia\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-07-03T18:35:44+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"680\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"KTRO TEAM\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"KTRO TEAM\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"45 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"http:\/\/ktromedia.com\/?p=180466#article\",\"isPartOf\":{\"@id\":\"http:\/\/ktromedia.com\/?p=180466\"},\"author\":{\"name\":\"KTRO TEAM\",\"@id\":\"https:\/\/ktromedia.com\/#\/schema\/person\/612bf2fbac107722ea365932cdd35f5b\"},\"headline\":\"Building Browser-Using AI Agents in Python\",\"datePublished\":\"2026-07-03T18:35:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"http:\/\/ktromedia.com\/?p=180466\"},\"wordCount\":9172,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/ktromedia.com\/#organization\"},\"image\":{\"@id\":\"http:\/\/ktromedia.com\/?p=180466#primaryimage\"},\"thumbnailUrl\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png\",\"articleSection\":[\"\u4eba\u5de5\u667a\u80fd\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"http:\/\/ktromedia.com\/?p=180466#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"http:\/\/ktromedia.com\/?p=180466\",\"url\":\"http:\/\/ktromedia.com\/?p=180466\",\"name\":\"Building Browser-Using AI Agents in Python - Ktromedia\",\"isPartOf\":{\"@id\":\"https:\/\/ktromedia.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"http:\/\/ktromedia.com\/?p=180466#primaryimage\"},\"image\":{\"@id\":\"http:\/\/ktromedia.com\/?p=180466#primaryimage\"},\"thumbnailUrl\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png\",\"datePublished\":\"2026-07-03T18:35:44+00:00\",\"breadcrumb\":{\"@id\":\"http:\/\/ktromedia.com\/?p=180466#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/ktromedia.com\/?p=180466\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/ktromedia.com\/?p=180466#primaryimage\",\"url\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png\",\"contentUrl\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png\",\"width\":1024,\"height\":680},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/ktromedia.com\/?p=180466#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/ktromedia.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Building Browser-Using AI Agents in Python\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/ktromedia.com\/#website\",\"url\":\"https:\/\/ktromedia.com\/\",\"name\":\"Ktromedia\",\"description\":\"KTRO MEDIA Crypto News\",\"publisher\":{\"@id\":\"https:\/\/ktromedia.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/ktromedia.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/ktromedia.com\/#organization\",\"name\":\"Ktromedia\",\"url\":\"https:\/\/ktromedia.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/ktromedia.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/11\/ktroicon.png\",\"contentUrl\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/11\/ktroicon.png\",\"width\":250,\"height\":250,\"caption\":\"Ktromedia\"},\"image\":{\"@id\":\"https:\/\/ktromedia.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/KTROMedia\/\",\"https:\/\/www.linkedin.com\/company\/ktro-media\/\",\"https:\/\/t.me\/ktrogroup\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/ktromedia.com\/#\/schema\/person\/612bf2fbac107722ea365932cdd35f5b\",\"name\":\"KTRO TEAM\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/ktromedia.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/10\/cropped-Untitled-design-7-1-150x150.png\",\"contentUrl\":\"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/10\/cropped-Untitled-design-7-1-150x150.png\",\"caption\":\"KTRO TEAM\"},\"description\":\"KTRO MEDIA \u662f\u4e00\u5bb6\u5168\u7403\u6027\u7684\u534e\u6587WEB3\u5a92\u4f53\u516c\u53f8\u3002\u6211\u4eec\u81f4\u529b\u4e8e\u4e3a\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u9886\u57df\u63d0\u4f9b\u6700\u65b0\u7684\u65b0\u95fb\u3001\u89c1\u89e3\u548c\u8d8b\u52bf\u5206\u6790\u3002\u6211\u4eec\u7684\u5b97\u65e8\u662f\u4e3a\u5168\u7403\u7528\u6237\u63d0\u4f9b\u9ad8\u8d28\u91cf\u3001\u5168\u9762\u7684\u8d44\u8baf\u670d\u52a1\uff0c\u8ba9\u4ed6\u4eec\u66f4\u597d\u5730\u4e86\u89e3\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u884c\u4e1a\u7684\u6700\u65b0\u52a8\u6001\u3002\u6211\u4eec\u4e5f\u5e0c\u671b\u80fd\u5e2e\u5230\u66f4\u591a\u4f18\u79c0\u7684WEB3\u4ea7\u54c1\u627e\u5230\u66f4\u591a\u66f4\u597d\u7684\u8d44\u6e90\u597d\u8ba9\u8fd9\u9886\u57df\u53d8\u5f97\u66f4\u6210\u719f\u3002 \u6211\u4eec\u7684\u62a5\u9053\u8303\u56f4\u6db5\u76d6\u4e86\u533a\u5757\u94fe\u3001\u52a0\u5bc6\u8d27\u5e01\u3001\u667a\u80fd\u5408\u7ea6\u3001DeFi\u3001NFT \u548c Web3 \u751f\u6001\u7cfb\u7edf\u7b49\u9886\u57df\u3002\u6211\u4eec\u7684\u62a5\u9053\u4e0d\u4ec5\u6765\u81ea\u884c\u4e1a\u5185\u7684\u4e13\u5bb6\uff0c\u5148\u950b\u8005\u4e5f\u5305\u62ec\u4e86\u6211\u4eec\u81ea\u5df1\u7684\u5206\u6790\u548c\u89c2\u70b9\u3002\u6211\u4eec\u5728\u5404\u4e2a\u56fd\u5bb6\u548c\u5730\u533a\u90fd\u8bbe\u6709\u56e2\u961f\uff0c\u4e3a\u8bfb\u8005\u63d0\u4f9b\u672c\u5730\u5316\u7684\u62a5\u9053\u548c\u5206\u6790\u3002 \u9664\u4e86\u65b0\u95fb\u62a5\u9053\uff0c\u6211\u4eec\u8fd8\u63d0\u4f9b\u5e02\u573a\u7814\u7a76\u548c\u54a8\u8be2\u670d\u52a1\u3002\u6211\u4eec\u7684\u4e13\u4e1a\u56e2\u961f\u53ef\u4ee5\u4e3a\u60a8\u63d0\u4f9b\u6709\u5173\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u884c\u4e1a\u7684\u6df1\u5165\u5206\u6790\u548c\u5e02\u573a\u8d8b\u52bf\uff0c\u5e2e\u52a9\u60a8\u505a\u51fa\u66f4\u660e\u667a\u7684\u6295\u8d44\u51b3\u7b56\u3002 \u6211\u4eec\u7684\u4f7f\u547d\u662f\u6210\u4e3a\u5168\u7403\u534e\u6587\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u884c\u4e1a\u6700\u53d7\u4fe1\u8d56\u7684\u4fe1\u606f\u6765\u6e90\u4e4b\u4e00\u3002\u6211\u4eec\u5c06\u7ee7\u7eed\u4e0d\u65ad\u52aa\u529b\uff0c\u4e3a\u8bfb\u8005\u63d0\u4f9b\u6700\u65b0\u3001\u6700\u5168\u9762\u3001\u6700\u53ef\u9760\u7684\u4fe1\u606f\u670d\u52a1\u3002\",\"sameAs\":[\"https:\/\/ktromedia.com\"],\"url\":\"https:\/\/ktromedia.com\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Building Browser-Using AI Agents in Python - Ktromedia","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/ktromedia.com\/?p=180466","og_locale":"en_US","og_type":"article","og_title":"Building Browser-Using AI Agents in Python - Ktromedia","og_description":"In this article, you will learn how to build AI agents that can browse and interact with real websites using Playwright, browser-use, and LangGraph. Topics we will cover include: Why Playwright is the right foundation for browser automation in 2026, and how it differs from Selenium. How to scrape dynamic, JavaScript-rendered pages and complete multi-step","og_url":"http:\/\/ktromedia.com\/?p=180466","og_site_name":"Ktromedia","article_publisher":"https:\/\/www.facebook.com\/KTROMedia\/","article_published_time":"2026-07-03T18:35:44+00:00","og_image":[{"width":1024,"height":680,"url":"http:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png","type":"image\/png"}],"author":"KTRO TEAM","twitter_card":"summary_large_image","twitter_misc":{"Written by":"KTRO TEAM","Est. reading time":"45 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"http:\/\/ktromedia.com\/?p=180466#article","isPartOf":{"@id":"http:\/\/ktromedia.com\/?p=180466"},"author":{"name":"KTRO TEAM","@id":"https:\/\/ktromedia.com\/#\/schema\/person\/612bf2fbac107722ea365932cdd35f5b"},"headline":"Building Browser-Using AI Agents in Python","datePublished":"2026-07-03T18:35:44+00:00","mainEntityOfPage":{"@id":"http:\/\/ktromedia.com\/?p=180466"},"wordCount":9172,"commentCount":0,"publisher":{"@id":"https:\/\/ktromedia.com\/#organization"},"image":{"@id":"http:\/\/ktromedia.com\/?p=180466#primaryimage"},"thumbnailUrl":"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png","articleSection":["\u4eba\u5de5\u667a\u80fd"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["http:\/\/ktromedia.com\/?p=180466#respond"]}]},{"@type":"WebPage","@id":"http:\/\/ktromedia.com\/?p=180466","url":"http:\/\/ktromedia.com\/?p=180466","name":"Building Browser-Using AI Agents in Python - Ktromedia","isPartOf":{"@id":"https:\/\/ktromedia.com\/#website"},"primaryImageOfPage":{"@id":"http:\/\/ktromedia.com\/?p=180466#primaryimage"},"image":{"@id":"http:\/\/ktromedia.com\/?p=180466#primaryimage"},"thumbnailUrl":"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png","datePublished":"2026-07-03T18:35:44+00:00","breadcrumb":{"@id":"http:\/\/ktromedia.com\/?p=180466#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/ktromedia.com\/?p=180466"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/ktromedia.com\/?p=180466#primaryimage","url":"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png","contentUrl":"https:\/\/ktromedia.com\/wp-content\/uploads\/2026\/07\/1783103742_Building-Browser-Using-AI-Agents-in-Python.png","width":1024,"height":680},{"@type":"BreadcrumbList","@id":"http:\/\/ktromedia.com\/?p=180466#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/ktromedia.com\/"},{"@type":"ListItem","position":2,"name":"Building Browser-Using AI Agents in Python"}]},{"@type":"WebSite","@id":"https:\/\/ktromedia.com\/#website","url":"https:\/\/ktromedia.com\/","name":"Ktromedia","description":"KTRO MEDIA Crypto News","publisher":{"@id":"https:\/\/ktromedia.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ktromedia.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/ktromedia.com\/#organization","name":"Ktromedia","url":"https:\/\/ktromedia.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ktromedia.com\/#\/schema\/logo\/image\/","url":"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/11\/ktroicon.png","contentUrl":"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/11\/ktroicon.png","width":250,"height":250,"caption":"Ktromedia"},"image":{"@id":"https:\/\/ktromedia.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/KTROMedia\/","https:\/\/www.linkedin.com\/company\/ktro-media\/","https:\/\/t.me\/ktrogroup"]},{"@type":"Person","@id":"https:\/\/ktromedia.com\/#\/schema\/person\/612bf2fbac107722ea365932cdd35f5b","name":"KTRO TEAM","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ktromedia.com\/#\/schema\/person\/image\/","url":"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/10\/cropped-Untitled-design-7-1-150x150.png","contentUrl":"https:\/\/ktromedia.com\/wp-content\/uploads\/2025\/10\/cropped-Untitled-design-7-1-150x150.png","caption":"KTRO TEAM"},"description":"KTRO MEDIA \u662f\u4e00\u5bb6\u5168\u7403\u6027\u7684\u534e\u6587WEB3\u5a92\u4f53\u516c\u53f8\u3002\u6211\u4eec\u81f4\u529b\u4e8e\u4e3a\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u9886\u57df\u63d0\u4f9b\u6700\u65b0\u7684\u65b0\u95fb\u3001\u89c1\u89e3\u548c\u8d8b\u52bf\u5206\u6790\u3002\u6211\u4eec\u7684\u5b97\u65e8\u662f\u4e3a\u5168\u7403\u7528\u6237\u63d0\u4f9b\u9ad8\u8d28\u91cf\u3001\u5168\u9762\u7684\u8d44\u8baf\u670d\u52a1\uff0c\u8ba9\u4ed6\u4eec\u66f4\u597d\u5730\u4e86\u89e3\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u884c\u4e1a\u7684\u6700\u65b0\u52a8\u6001\u3002\u6211\u4eec\u4e5f\u5e0c\u671b\u80fd\u5e2e\u5230\u66f4\u591a\u4f18\u79c0\u7684WEB3\u4ea7\u54c1\u627e\u5230\u66f4\u591a\u66f4\u597d\u7684\u8d44\u6e90\u597d\u8ba9\u8fd9\u9886\u57df\u53d8\u5f97\u66f4\u6210\u719f\u3002 \u6211\u4eec\u7684\u62a5\u9053\u8303\u56f4\u6db5\u76d6\u4e86\u533a\u5757\u94fe\u3001\u52a0\u5bc6\u8d27\u5e01\u3001\u667a\u80fd\u5408\u7ea6\u3001DeFi\u3001NFT \u548c Web3 \u751f\u6001\u7cfb\u7edf\u7b49\u9886\u57df\u3002\u6211\u4eec\u7684\u62a5\u9053\u4e0d\u4ec5\u6765\u81ea\u884c\u4e1a\u5185\u7684\u4e13\u5bb6\uff0c\u5148\u950b\u8005\u4e5f\u5305\u62ec\u4e86\u6211\u4eec\u81ea\u5df1\u7684\u5206\u6790\u548c\u89c2\u70b9\u3002\u6211\u4eec\u5728\u5404\u4e2a\u56fd\u5bb6\u548c\u5730\u533a\u90fd\u8bbe\u6709\u56e2\u961f\uff0c\u4e3a\u8bfb\u8005\u63d0\u4f9b\u672c\u5730\u5316\u7684\u62a5\u9053\u548c\u5206\u6790\u3002 \u9664\u4e86\u65b0\u95fb\u62a5\u9053\uff0c\u6211\u4eec\u8fd8\u63d0\u4f9b\u5e02\u573a\u7814\u7a76\u548c\u54a8\u8be2\u670d\u52a1\u3002\u6211\u4eec\u7684\u4e13\u4e1a\u56e2\u961f\u53ef\u4ee5\u4e3a\u60a8\u63d0\u4f9b\u6709\u5173\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u884c\u4e1a\u7684\u6df1\u5165\u5206\u6790\u548c\u5e02\u573a\u8d8b\u52bf\uff0c\u5e2e\u52a9\u60a8\u505a\u51fa\u66f4\u660e\u667a\u7684\u6295\u8d44\u51b3\u7b56\u3002 \u6211\u4eec\u7684\u4f7f\u547d\u662f\u6210\u4e3a\u5168\u7403\u534e\u6587\u533a\u5757\u94fe\u548c\u91d1\u878d\u79d1\u6280\u884c\u4e1a\u6700\u53d7\u4fe1\u8d56\u7684\u4fe1\u606f\u6765\u6e90\u4e4b\u4e00\u3002\u6211\u4eec\u5c06\u7ee7\u7eed\u4e0d\u65ad\u52aa\u529b\uff0c\u4e3a\u8bfb\u8005\u63d0\u4f9b\u6700\u65b0\u3001\u6700\u5168\u9762\u3001\u6700\u53ef\u9760\u7684\u4fe1\u606f\u670d\u52a1\u3002","sameAs":["https:\/\/ktromedia.com"],"url":"https:\/\/ktromedia.com\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/posts\/180466","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ktromedia.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=180466"}],"version-history":[{"count":1,"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/posts\/180466\/revisions"}],"predecessor-version":[{"id":180468,"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/posts\/180466\/revisions\/180468"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ktromedia.com\/index.php?rest_route=\/wp\/v2\/media\/180467"}],"wp:attachment":[{"href":"https:\/\/ktromedia.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=180466"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ktromedia.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=180466"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ktromedia.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=180466"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}