mcp-server-fetch-python MCP Server: Data Extraction and Transformation

Overview: What is mcp-server-fetch-python MCP Server?

The mcp-server-fetch-python serves as an essential component for AI applications seeking to fetch, extract, and transform web content into structured formats. This server leverages both traditional URL scraping techniques and advanced headless browser rendering capabilities, ensuring comprehensive data capture even from complex modern web pages that rely on JavaScript execution.

🔧 Core Features & MCP Capabilities

Tools Suite:

get-raw-text

The get-raw-text tool is designed to extract raw text content directly from URLs without the need for JavaScript rendering. It supports various data formats such as plain text, JSON, XML, CSV, and TSV files.

• Arguments:
  • url: The URL of the target web page (text, JSON, XML, csv, tsv, etc.).
• Usage: Best suited for structured data formats where fast and direct access is required.

get-rendered-html

This tool fetches fully rendered HTML content using a headless browser, ensuring that JavaScript-rendered elements are also included in the output.

• Arguments:
  • url: The URL of the target web page (required).
• Usage: Essential for modern web applications and Single Page Applications (SPAs) where JavaScript rendering is critical.

get-markdown

The get-markdown tool converts web page content to well-formatted Markdown, preserving structural elements for easier readability.

• Arguments:
  • url: The URL of the target web page (required).
• Usage: Ideal for creating clean, readable output that maintains the original structure of the web page.

get-markdown-from-media

This tool performs AI-powered content extraction from media files such as images and videos.

• Arguments:
  • url: The URL of the target media file (images, videos).
• Usage: Utilizes computer vision and Optical Character Recognition (OCR) for analyzing visual content. Requires a valid OPENAI_API_KEY to be set in environment variables.

⚙️ MCP Architecture & Protocol Implementation

The architecture of the mcp-server-fetch-python is designed to seamlessly integrate with various AI applications using the Model Context Protocol (MCP). The server components are implemented as part of the broader MCP ecosystem, facilitating standardized data exchange between different application clients and upstream services.

MCP Protocol Flow Diagram:

graph TD
    A[AI Application] -->|MCP Client| B[MCP Protocol]
    B --> C[MCP Server]
    C --> D[Data Source/Tool]
    style A fill:#e1f5fe
    style C fill:#f3e5f5
    style D fill:#e8f5e8

This diagram illustrates the flow of data within the MCP protocol, from an AI application to the server and ultimately to a data source or tool.

🚀 Getting Started with Installation

Local Installation:

To run mcp-server-fetch-python in a local environment, follow these steps:

1. Clone the Repository:

   git clone https://github.com/tatn/mcp-server-fetch-python.git
   cd mcp-server-fetch-python
   

2. Build and Start the Server:

   uv sync
   uv build
   

Then, configure your MCP client to use this server.

Integration with MCP Clients

MCP Client	Resources	Tools	Prompts	Status
Claude Desktop	✅	✅	✅	Full Support
Continue	✅	✅	✅	Full Support
Cursor	❌	✅	❌	Tools Only

This table outlines the compatibility of mcp-server-fetch-python with various MCP clients.

💡 Key Use Cases in AI Workflows

1. Web Content Analysis:

   • Scenario: A research analyst needs to gather data from a series of web pages containing reports and charts.

     # Example Python code using the get-markdown tool
     import mcp_server_fetch_python
     
     url = "https://example.com/report"
     response = mcp_server_fetch_python.get_markdown(url)
     print(response.text)
     

2. Automated Content Extraction from Images and Videos:

   • Scenario: A marketing team wants to extract text data from a series of promotional images for automated sentiment analysis.

     # Example Python code using the get-markdown-from-media tool
     import mcp_server_fetch_python
     
     url_image = "https://example.com/image"
     url_video = "https://example.com/video"
     
     response_image = mcp_server_fetch_python.get_markdown_from_media(url_image)
     print(response_image.text)
     
     response_video = mcp_server_fetch_python.get_markdown_from_media(url_video)
     print(response_video.text)
     

🔌 Integration with MCP Clients

To configure mcp-server-fetch-python for use with an MCP client like Claude Desktop, add the following to your .claude_desktop_config.json file:

{
  "mcpServers": {
    "mcp-server-fetch-python": {
      "command": "uv",
      "args": [
        "--directory",
        "path\\to\\mcp-server-fetch-python",  # Replace with actual path to the cloned repository
        "run",
        "mcp-server-fetch-python"
      ],
      "env": {
        "OPENAI_API_KEY": "sk-***",  # Set your API key here
        "PYTHONIOENCODING": "utf-8",
        "MODEL_NAME": "gpt-4o"
      }
    }
  }
}

📊 Performance & Compatibility Matrix

The performance of mcp-server-fetch-python is optimized for various use cases, ensuring that data fetching and transformation are both efficient and effective. Here’s a compatibility matrix highlighting the performance across different AI clients.

[Performance Data Available on Request]

🛠️ Advanced Configuration & Security

1. Security Environment Variables:
   • OPENAI_API_KEY: Required for using the get-markdown-from-media tool.

     {
       "env": {
         "OPENAI_API_KEY": "sk-***"
       }
     }
     

2. Character Encoding:
   • Ensure correct character encoding is handled with PYTHONIOENCODING, especially when dealing with non-ASCII characters.

❓ Frequently Asked Questions (FAQ)

1. How can I ensure my server integration works correctly?
   • Use the MCP Inspector to debug and verify the server’s functionality.
2. What are the system requirements for running this server locally?
   • Ensure that the necessary dependencies and Python environment are set up properly.
3. Can I use third-party tools with this server?
   • Yes, but make sure they are compatible with the MCP protocol.
4. How do I handle large volumes of data extraction?
   • Implement batch processing or parallel execution to manage large datasets efficiently.
5. What if my media files require additional processing steps?
   • Check the documentation for the get-markdown-from-media tool and adjust as needed.

👨‍💻 Development & Contribution Guidelines

Contributions are welcome! To contribute, follow these guidelines:

1. Fork the repository on GitHub.
2. Create a pull request with your changes.
3. Document any new features or modifications thoroughly.

🌐 MCP Ecosystem & Resources

Explore more MCP-related documentation and resources at MCP Documentation.

By integrating mcp-server-fetch-python into your AI applications, you can achieve robust data extraction and transformation capabilities, enhancing the overall functionality of your solutions.