网页抓取 资源

The World's First Complete Media Testing Platform with Revolutionary Audio & Video Capabilities

An AI-powered browser automation platform that converts natural language test steps into executable browser automation using the Model Context Protocol (MCP) and Playwright.

Grasp is an open-source and self-hosted agentic browser. With built-in support for MCP and A2A support, it can seamlessly integrate with any other AI apps or agents.

A Model Context Protocol (MCP) server for WaterCrawl, built with FastMCP. This package provides AI systems with web crawling, scraping, and search capabilities through a standardized interface.

This project implements a Model Context Protocol (MCP) server that provides a tool for fetching YouTube video transcripts in various formats. Leveraging the youtube-transcript-api, the server allows Large Language Models (LLMs) to access YouTube transcripts securely and efficiently.

Capture real browser network requests (URLs, methods, status, headers, optional bodies/traces) and save them locally as newline-delimited JSON (JSONL). This repo wires Playwright MCP (browser automation) with a Filesystem MCP (safe writes) and a small headless Python client.

mcp-stock-scanner

A Model Context Protocol server that provides browser automation capabilities using Playwright Enable LLMs to interact with web pages, take screenshots, and execute JavaScript in a real browser environment

Make your AI tools 10x more aware and capable of interacting with your browser

I wanted to try MCP and used the repository MCP-searxng by SecretiveShell as a starting point.

Built on a 195+ country proxy network, Thordata MCP breaks through web data barriers, delivering pure, structured, globally unlimited real-time information streams to AI models

A Node.js MCP Server for capturing screenshots using Safari on macOS.

A protocol that enables AI models to control Chrome browser and perform web automation.

An MCP (Model Context Protocol) server that can scrape web pages and extract content using CSS selectors. Built with deno-dom for fast HTML parsing.

Cloudflare Worker-based batch processing system for Apple Developer Documentation with intelligent content comparison and automated scheduling.

The Cross-Platform Browser That Actually Works in Authorized Testing Environments 99.7% Success Rate Zero Configuration Perfect Mobile Emulation Windows profiles run flawlessly on macOS Android simulation on desktop

A Model Context Protocol (MCP) server implementation that integrates with Firecrawl for web scraping capabilities.

A Model Context Protocol server that provides web content fetching capabilities. This server enables LLMs to retrieve and process content from web pages, converting HTML to markdown for easier consumption.

A simple CLI project that combines Bright Datas Browser API (MCP) with OpenAI to scrape hotel listings and generate a 3-day travel itinerary using AI (which is just the example used, but you can use it for any scraping task).

Automate your browser with natural language commands - The open source browser-use solution

This project is a web scraper designed to collect information about government support projects from various websites.

MCP server to give client the ability to extract a web page

A Model Context Protocol (MCP) server that enables Claude Desktop to interact with web browsers using WebDriverIO. This allows Claude to perform web automation tasks like clicking elements, filling forms, taking screenshots, and more.

Capture real browser network requests (URLs, methods, status, headers, optional bodies/traces) and save them locally as newline-delimited JSON (JSONL). This repo wires Playwright MCP (browser automation) with a Filesystem MCP (safe writes) and a small headless Python client.

A Model Context Protocol (MCP) server that provides web search and content extraction capabilities.

The Apify Model Context Protocol (MCP) server at mcp.apify.com enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.

English |

A Model Context Protocol (MCP) server for web crawling and content extraction from web pages with multiple output formats.

The Cool Browser Automation MCP Server

The crawl4ai-mcp-server is a lightweight server that allows you to access web scraping and crawling tools easily. It provides similar capabilities to Firecrawl's API but offers a self-hosted and free option. This server integrates seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code. You can use it for various AI workflows, making it a valuable resource for anyone needing web data.

A lightweight MCP (Model Context Protocol) server for LLM orchestration that provides efficient web content search and extraction capabilities. This CLI tool enables LLMs to search DuckDuckGo and extract clean, LLM-friendly content from web pages.

Docfork MCP pulls @latest documentation and code examples straight from the source - and adds them right into your context.

This repository hosts an AI agent that runs daily and collects the most interesting English-language articles on:

Fetch data from German Marktstammdatenregister (marktstammdatenregister.de)

The URL-Context-MCP MCP Server provides a tool to analyze and summarize the content of URLs using Google Gemini's URL Context capability via the Gemini API.

Easy, effortless Web Scraping as it should be!

The first remote Model Context Protocol (MCP) server that enables Claude AI to extract transcripts from YouTube videos. This server offers zero-setup access for users on any platform including mobile devices.

Welcome to the geo-ai-agent! This tool helps you audit and enhance your website content. It uses AI to crawl your URLs, analyze H1 tags, and provide GEO recommendations. With this tool, you can improve your sites SEO and user engagement without any technical knowledge.

A Model Context Protocol (MCP) server for controlling and automating Google Chrome browser operations using Puppeteer. This server provides tools for web automation, element interaction, and browser management.

A Model Context Protocol (MCP) server for YouTube operations, demonstrating core MCP concepts including tools and logging.

A Model Context Protocol (MCP) server for interacting with YouTube data. This server provides resources and tools to query YouTube videos, channels, comments, and transcripts through a stdio interface.

A Model Context Protocol (MCP) server implementation that integrates with Supadata for video & web scraping capabilities.

For an explanation and POC of what this mcp server does, please visit my article on medium : Mypost. But globally, it's an MCP server for performing XSS tests with AI.

A Model Context Protocol (MCP) server that provides Claude Code with comprehensive browser automation capabilities through Puppeteer. This server allows Claude to interact with web pages, take screenshots, execute JavaScript, and perform various browser automation tasks.

https://github.com/NanmiCoder/MediaCrawler

A comprehensive competitor analysis tool built for HeroForge.ai to analyze Aloa.co and other competitors, with advanced MCP integration, context management, and future SEO capabilities.

A powerful system for crawling documentation websites, extracting code snippets, and providing fast search capabilities via MCP (Model Context Protocol) integration.

Xenopus MCP is a tool that provides an interface to the Screaming Frog SEO Spider. It allows you to automate crawling and data extraction from websites.

Enhanced Model Context Protocol (MCP) server for browser automation with improved element selection, tab management, and token optimization. Built on top of the original BrowserMCP with significant improvements for AI-powered browser interaction.

Manage your GoLogin browser profiles and automation directly through AI conversations. This MCP server connects to the GoLogin API, letting you create, configure, and control browser profiles using natural language.

A tool to extract content and highlights from Apple Books for LLM analysis.

Advanced web scraping with AI-powered challenge solving and stealth capabilities.

The most feature-rich, production-ready web fetching MCP server available

yutu is a fully functional MCP server and CLI for YouTube to automate your YouTube workflows. It can manipulate almost all YouTube resources, like videos, playlists, channels, comments, captions, and more.

A self-hosted Puppeteer MCP (Model Context Protocol) server with remote SSE access, API key authentication, and Docker deployment. This server provides 16 comprehensive Puppeteer tools including advanced mouse interactions and authentication cookie management, with enhanced security, monitoring, and production-ready features.