CoinParseApi - Cryptocurrency Data Parser & API

A C# / .NET web service that parses cryptocurrency data from CoinMarketCap and provides a REST API with pagination, filtering, and sorting capabilities.

Features:

• Web Scraping: Parses cryptocurrency data from CoinMarketCap using Selenium WebDriver
• REST API: Complete API with pagination, filtering, and sorting
• Database Storage: Uses PostgreSQL/SQLite for data persistence
• Batch Processing: Efficient database updates with batched operations
• Error Handling: Comprehensive logging and exception handling
• Flexible Architecture: Modular design with clear separation of concerns

Tech Stack:

• Framework: .NET 8.0 / ASP.NET Core
• Database: PostgreSQL
• Web Scraping: Selenium WebDriver with Chrome headless
• ORM: Entity Framework Core
• Logging: Built-in .NET logging

API Endpoints

Get Cryptocurrencies

GET /api/crypto Parameters:

• offset - Starting position (default: 0)
• limit - Items per page (default: 100, max: 1000)
• name - Filter by name (partial match)
• symbol - Filter by symbol (partial match)
• price_min / price_max - Price range filters
• marketcap_min / marketcap_max - Market cap range filters
• volume_min / volume_max - Volume range filters
• change_positive - Filter by positive/negative change (true/false)
• sort_by - Sort field: name, symbol, price, marketcap, volume, change, lastupdated
• sort_order - Sort direction: asc, desc

Update data

POST /api/crypto/update

Triggers manual data refresh from CoinMarketCap.

API Information

GET /api/crypto/api-info

Returns API documentation and usage examples.

Response format

{
  "data": [
    {
      "id": "guid",
      "name": "Bitcoin",
      "symbol": "BTC",
      "price": "$45,234.56",
      "marketCap": "884.2B",
      "volume24H": "28.5B",
      "priceChange24H": "2.34%",
      "lastUpdated": "2025-09-23T20:42:00Z"
    }
  ],
  "meta": {
    "totalCount": 9734,
    "offset": 0,
    "limit": 100,
    "hasMore": true,
    "currentPage": 1,
    "totalPages": 98
  }
}

Database Schema

CREATE TABLE cryptocurrencies (
    Id UUID PRIMARY KEY,
    Name VARCHAR(100) NOT NULL,
    Symbol VARCHAR(20),
    Price DECIMAL(18,8),
    MarketCap DECIMAL(18,2),
    Volume24H DECIMAL(18,2),
    PriceChange24H REAL,
    LastUpdated TIMESTAMP,
    CreatedAt TIMESTAMP
);

Setup & Installation

Prerequisites:

• NET 8.0 SDK
• PostgreSQL

Configuration

Database Connection

{
  "ConnectionStrings": {
    "DefaultConnection": "Host=localhost;Port=5432;Database=coin_parse_db;Username=postgres;Password=postgres_super_user;"
  }
}

Environment Setup

dotnet restore
dotnet build
dotnet run

If you run from console, try path {BaseURL from console}/swagger

Implementation notes

Web Scraping Strategy

• Progressive Loading: Scrolls through CoinMarketCap's infinite scroll interface
• Traffic Minimization: Single page load with intelligent scrolling
• Duplicate Prevention: Uses name+symbol combination for uniqueness
• Error Resilience: Handles page structure changes gracefully

Performance Considerations

• Batch Database Operations: 1000 records per batch for optimal performance
• Efficient Queries: Indexed fields for fast filtering and sorting
• Memory Management: Disposes of WebDriver resources properly
• Parsing Time: ~15-25 minutes for full dataset extraction