Google Sheets Scraper

Scraper for the public Google Sheets document containing daily Russia-Ukraine equipment loss data.

Overview

This scraper fetches and parses data from: https://docs.google.com/spreadsheets/d/1bngHbR0YPS7XH1oSA1VxoL4R34z60SJcR3NxguZM9GI/edit

The sheet contains daily time-series data with comprehensive metrics for both Russia and Ukraine including:

• Total losses and daily changes
• Destroyed, damaged, abandoned, and captured counts
• Equipment categories: Tanks, AFV, IFV, APC, IMV, Engineering, Communications, Vehicles, Aircraft, Infantry, Logistics, Armor, Antiair, Artillery
• UNHCR refugee data

Installation

pip install -r requirements.txt

Usage

Command Line

# Print all data to stdout
python scraper.py

# Save to file
python scraper.py -o output.json

# Get only the latest day's data
python scraper.py --latest -o latest.json

# Filter by date range
python scraper.py --start-date 2022-02-24 --end-date 2022-03-01 -o filtered.json

# Custom indentation
python scraper.py -o output.json --indent 4

# Debug mode (shows column names)
python scraper.py --debug

# Show available columns
python scraper.py --show-columns

Python API

from scraper import GoogleSheetsScraper

with GoogleSheetsScraper() as scraper:
    # Get all data
    data = scraper.scrape()
    print(f"Total days: {data['total_days']}")

    # Get latest day
    latest = scraper.get_latest()
    print(f"Latest date: {latest['date']}")

    # Get date range
    filtered = scraper.get_by_date_range('2022-02-24', '2022-03-01')
    print(f"Days in range: {len(filtered)}")

    # Save to JSON
    scraper.scrape_to_json('output.json')

Output Format

The scraper returns a JSON structure:

{
  "url": "https://docs.google.com/spreadsheets/d/...",
  "sheet_id": "1bngHbR0YPS7XH1oSA1VxoL4R34z60SJcR3NxguZM9GI",
  "total_days": 100,
  "date_range": {
    "start": "2022-02-24",
    "end": "2022-06-02"
  },
  "data": [
    {
      "date": "2022-02-24",
      "russia_total": 0,
      "russia_change": 0,
      "ukraine_total": 0.0,
      "ukraine_change": 0,
      "ratio_ru_ua": 0,
      "russia_destroyed": 0,
      "ukraine_destroyed": 0,
      "russia_damaged": 0,
      "ukraine_damaged": 0,
      "russia_tanks": 5,
      "ukraine_tanks": 2,
      ...
    },
    ...
  ]
}

Data Columns

The scraper parses all columns from the sheet:

Totals

• russia_total, ukraine_total - Total equipment losses
• russia_change, ukraine_change - Daily change
• ratio_ru_ua - Ratio of Russia to Ukraine losses

Status Breakdown

• russia_destroyed, ukraine_destroyed
• russia_damaged, ukraine_damaged
• russia_abandoned, ukraine_abandoned
• russia_captured, ukraine_captured

Equipment Categories

• Tanks: russia_tanks, ukraine_tanks, russia_tank_capture, ukraine_tank_capture
• AFV (Armoured Fighting Vehicles): russia_afv, ukraine_afv, russia_afv_capture, ukraine_afv_capture
• IFV (Infantry Fighting Vehicles): russia_ifv, ukraine_ifv
• APC (Armoured Personnel Carriers): russia_apc, ukraine_apc
• IMV (Infantry Mobility Vehicles): russia_imv, ukraine_imv
• Engineering: russia_engineering, ukraine_engineering
• Communications: russia_coms, ukraine_coms
• Vehicles: russia_vehicles, ukraine_vehicles
• Aircraft: russia_aircraft, ukraine_aircraft
• Infantry: russia_infantry, ukraine_infantry
• Logistics: russia_logistics, ukraine_logistics
• Armor: russia_armor, ukraine_armor
• Antiair: russia_antiair, ukraine_antiair
• Artillery: russia_artillery, ukraine_artillery

Refugee Data

• unhcr_ukraine_border - Refugees at border
• unhcr_ukraine_refugees - Total refugees
• unhcr_returning_ukraine_refugees - Returning refugees

Notes

• The scraper uses Google Sheets CSV export (no authentication required for public sheets)
• All numeric values are parsed as integers or floats
• Empty cells are handled gracefully (default to 0 or None)
• Dates are validated to ensure proper format (YYYY-MM-DD)
• The scraper respects rate limits with a 30-second timeout

Requirements

• Python 3.8+
• httpx (for HTTP requests)