diff --git a/docs/providers/alternative_data.md b/docs/providers/alternative_data.md new file mode 100644 index 0000000..e1ea582 --- /dev/null +++ b/docs/providers/alternative_data.md @@ -0,0 +1,210 @@ +# Alternative Data Sources + +This page maps sources of alternative data: news and text, social and web attention, app and +web usage, job postings, card transactions, foot traffic, satellite and shipping data, ESG +ratings, supply-chain relationships, and prediction markets, plus the marketplaces where such +datasets are found. For each source it lists what the data measures, how far back it goes, what +is known about its point-in-time behavior, the access tier, and whether `ml4t-data` wraps it. + +!!! note "Verified September 2026" + Alternative data vendors are acquired, merged and repackaged often. Every entry below was + checked in September 2026 against the vendor's own site, a press release, or published reporting. Few + vendors publish their timestamp and revision policy; where it is not public, the table says + "not documented" and the question belongs in your due diligence. Confirm the current terms + before you build on any of them. + +--- + +## Choosing a Source + +### When was each record first observable? + +A backtest is only valid if every record is used after the time it could first have been seen. +Ask the vendor for the timestamp that marks first availability, not the event date, and whether +history was backfilled when the dataset launched or when coverage expanded. A backfilled history +looks continuous but was assembled later, often by a method that did not exist at the time. + +### Revisions and methodology changes + +Panels change composition (card and device panels gain and lose users), models are re-estimated, +and rating methodologies are revised. ESG ratings are the clearest case: providers disagree with +each other and revise their own methods over time. Ask whether old values are restated when the +method changes and whether earlier vintages are kept. + +### Coverage and representativeness + +Check which names, sectors and regions a dataset actually covers, and whether coverage is stable +across the sample. A signal that exists only for large US consumer companies is a narrower +signal than its marketing suggests. + +### Legal and privacy + +Provenance, permitted use and redistribution rights are hard constraints. Narrow panels can carry +material non-public information, and granular location or device data carries privacy risk even +without names. These questions are answered before any backtest, not after. + +### In the book + +- Chapter 4, "Understanding alternative data" (section 4.4): "The alternative data landscape", + "The evaluation framework", "Build versus buy for alternative data", and "Data sources and + implementation". +- Chapter 4, "Using text data for NLP features" (section 4.5): extracting and storing text from + filings point-in-time. Chapter 10 covers turning text into features. +- Chapter 2, "A modern taxonomy of financial data" (section 2.1), subsection "Alternative data - + High variety, high validation burden". + +--- + +## News, Text and Sentiment + +| Source | What it measures | History | Point-in-time and revisions | Access | ml4t-data | +|--------|------------------|---------|-----------------------------|--------|-----------| +| [GDELT Project](https://gdeltproject.org/data.html) | Global news events and tone (Event database); themes, entities and sentiment from full text (Global Knowledge Graph) | Events from 1979; Knowledge Graph from April 2013 | Published in 15-minute batches; the batch time is a usable first-availability time. Older history was built retrospectively. | Free and open | No | +| [RavenPack](https://www.ravenpack.com/) | Entity-level news analytics: relevance, sentiment and event classification | From 2000 (Dow Jones edition), 2007 (web edition) | Records carry the news timestamp; revision policy not documented publicly | Institutional; academic via WRDS | No | +| [LSEG MarketPsych Analytics](https://www.lseg.com/en/data-analytics/market-data/quantitative-economic-data-solutions/marketpsych-analytics-and-models) | Sentiment and emotion scores from news and social media across equities, macro, FX, crypto and commodities | From 1998 | Minute-level to daily series; revision policy not documented publicly | Institutional | No | +| [Dow Jones Factiva DNA](https://www.dowjones.com/business-intelligence/factiva/) | Licensed premium news archive with metadata, via API, snapshots and streams | Archive depth varies by publication | Publication timestamps per article | Institutional | No | +| [Benzinga Newsfeed API](https://docs.benzinga.com/api-reference/news-api/overview) | Financial news headlines and articles, Wilshire 5000 and TSX coverage | Not documented | `created` and `updated` timestamps per article; `updatedSince` returns changes | Paid | No | +| [SEC EDGAR full-text search](https://www.sec.gov/edgar/search/efts-faq.html) | Full text of SEC filings | From 2001 | Every filing carries its acceptance timestamp | Free | No | + +--- + +## Social Media and Web Attention + +Access to social-media data has narrowed since 2023, so check the current terms before planning +a project around any of these. + +| Source | What it measures | Status | Access | ml4t-data | +|--------|------------------|--------|--------|-----------| +| [Stocktwits API](https://api.stocktwits.com/developers) | Investor message stream with cashtags and self-reported sentiment | Closed to new developer registrations | Not open to new developers | No | +| [Reddit Data API](https://support.reddithelp.com/hc/en-us/articles/16160319875092-Reddit-Data-API-Wiki) | Posts and comments, including investing subreddits | Paid for commercial use since July 2023 | Commercial license required | No | +| [X API](https://docs.x.com/x-api/getting-started/pricing) | Posts and engagement | Pay-per-use credits since February 2026, replacing subscription tiers | Paid | No | + +--- + +## Web, App and Workforce Data + +| Source | What it measures | History | Point-in-time and revisions | Access | ml4t-data | +|--------|------------------|---------|-----------------------------|--------|-----------| +| [Similarweb](https://www.similarweb.com/corp/ourdata/) | Website and app traffic and engagement estimates | About 10 years | Modeled estimates; revision policy not documented publicly | Free tools, paid, enterprise | No | +| [Sensor Tower](https://sensortower.com/product/mobile-app/app-performance-insights) | App downloads, revenue, rankings and usage estimates; includes the former data.ai | Not documented | Modeled estimates; revision policy not documented publicly | Enterprise; free top charts | No | +| [Thinknum](https://www.thinknum.com/) | Web-collected company data: job listings, headcount, product prices, store locations | Not documented | Not documented | Institutional | No | +| [YipitData](https://www.yipitdata.com/) | Company KPI and revenue estimates from web, app and transaction data | Not documented | Not documented | Institutional | No | +| [Revelio Labs](https://www.reveliolabs.com/data) | Workforce composition and flows, job postings, employee sentiment, layoff notices | Workforce composition from 2007, transitions from 2008, postings from 2021 | Not documented | Paid; academic via WRDS | No | +| [LinkUp](https://www.linkup.com/data) | Job postings collected directly from employer websites | From 2007 | New, updated and removed listings are captured nightly | Institutional | No | + +--- + +## Card Transactions + +Card panels measure the spending of a sample of consumers. Panel composition changes over time, +so ask how the vendor weights the panel and whether history is reweighted when it changes. + +| Source | What it measures | History | Access | ml4t-data | +|--------|------------------|---------|--------|-----------| +| [Bloomberg Second Measure](https://secondmeasure.com/) | US consumer card spending by merchant and company | 8+ years; 2 to 7 day reporting lag | Bloomberg Terminal and enterprise | No | +| [Consumer Edge Transact](https://www.consumeredge.com/products/transact/) | Card transactions by merchant, company and demographic; now includes Earnest Analytics | About 9 years | Institutional | No | +| [Facteus](https://facteus.com/) | Debit and credit card transactions, synthesized for privacy | Not documented; about 1-day lag | Institutional feeds (API, AWS, Snowflake) | No | + +--- + +## Foot Traffic, Satellite and Shipping + +| Source | What it measures | History | Access | ml4t-data | +|--------|------------------|---------|--------|-----------| +| [Placer.ai](https://www.placer.ai/products/api) | Visits to stores and venues from a mobile-device panel | Not documented | Free tools, paid platform, enterprise API | No | +| [Advan Research Patterns+](https://advanresearch.com/products/patternsplus) | Foot traffic to points of interest, US and Canada; took over SafeGraph's Patterns product | Weekly updates; history from 2019 | Institutional | No | +| [Planet Labs](https://www.planet.com/industries/education-and-research/) | Satellite imagery: near-daily medium resolution and tasked high resolution | Archive from 2009 (RapidEye, 2009 to 2020) | Commercial; education and research program for university users | No | +| [RS Metrics](https://rsmetrics.com/) | Asset-level signals from satellite imagery: metals production at smelters, retail parking lots | Not documented | Institutional; data for research | No | +| [Kpler](https://www.kpler.com/product/commodities) | Commodity flows from ship tracking (AIS), satellite, customs and port data; now owns MarineTraffic and Spire Maritime | Not documented | Institutional | No | + +--- + +## ESG + +ESG ratings disagree across providers and change when methodologies change. Treat a rating +history as a sequence of vintages, and check which methodology produced each value. + +| Source | What it measures | History and methodology | Access | ml4t-data | +|--------|------------------|-------------------------|--------|-----------| +| [MSCI ESG Ratings](https://www.msci.com/data-and-analytics/sustainability-solutions/esg-ratings) | Industry-relative ESG ratings (AAA to CCC), about 17,000 issuers | Time series since 2007 | Institutional; free public lookup for a subset | No | +| [Morningstar Sustainalytics ESG Risk Ratings](https://www.sustainalytics.com/esg-data) | Unmanaged ESG risk score, 16,000+ companies | A score change log exists; vintage policy not documented publicly | Institutional | No | +| [LSEG ESG Scores](https://www.lseg.com/en/media-centre/press-releases/2026/lseg-launches-new-suite-esg-scores-sustainability-analytics) | ESG scores from 220+ indicators, plus controversies | New suite launched March 2026 on 220 standardised indicators; how it relates to earlier LSEG scores is not documented publicly | Institutional | No | +| [S&P Global ESG Scores](https://www.marketplace.spglobal.com/en/datasets/s-p-global-esg-scores-(171)) | Scores from the annual Corporate Sustainability Assessment | Annual assessments; history not documented publicly | Institutional | No | +| [RepRisk](https://www.reprisk.com/insights/resources/methodology) | Reputational and conduct risk from news and stakeholder sources | Daily series with a consistent methodology since January 2007 | Institutional | No | + +--- + +## Supply Chain and Trade + +| Source | What it measures | History | Access | ml4t-data | +|--------|------------------|---------|--------|-----------| +| [FactSet Supply Chain Relationships](https://www.factset.com/marketplace/catalog/product/factset-supply-chain-relationships) (formerly Revere) | Customer, supplier, partner and competitor links between companies | North America from 2003; other regions from 2011 to 2016 | Institutional; academic via WRDS | No | +| [Bloomberg Supply Chain (SPLC)](https://professional.bloomberg.com/institutions/corporations/supply-chain/) | Supplier and customer relationships with revenue and cost exposure, 100,000+ companies | From 2006 | Bloomberg Terminal and enterprise feed | No | +| [S&P Global Panjiva](https://www.marketplace.spglobal.com/en/datasets/panjiva-supply-chain-intelligence-(22)) | Shipment-level customs records, 2 billion+ records | Not documented | Institutional | No | +| [ImportGenius](https://www.importgenius.com/pricing) | US and international bill-of-lading records | US imports from 2006, exports from January 2014 on the Pro and Enterprise tiers | Self-serve subscriptions; enterprise plans | No | + +--- + +## Prediction Markets + +Prediction-market prices are market data: each trade and quote has an exchange timestamp and is +never revised. The caveats are thin liquidity outside headline contracts and markets that are +listed and resolved over time, so a set of markets picked today is a survivor sample. + +| Source | What it measures | History | Access | ml4t-data | +|--------|------------------|---------|--------|-----------| +| [Kalshi](https://docs.kalshi.com/welcome) | CFTC-regulated event contracts: economics, politics, weather, sports | From the July 2021 launch | Public market-data API; trading needs a verified account | Yes: [`KalshiProvider`](kalshi.md) | +| [Polymarket](https://docs.polymarket.com/) | Event contracts on a crypto platform; CFTC approval for intermediated US access, November 2025 | From 2020 | Public market data | Yes: [`PolymarketProvider`](polymarket.md) | + +--- + +## Marketplaces and Discovery + +Marketplaces simplify procurement; they do not validate point-in-time behavior or methodology. + +| Source | What it offers | Access | +|--------|----------------|--------| +| [Nasdaq Data Link](https://data.nasdaq.com/) (formerly Quandl) | Catalog of financial, economic and alternative datasets, some free | Free account for free datasets; paid for premium | +| [Snowflake Marketplace](https://docs.snowflake.com/en/collaboration/collaboration-marketplace-about) | Datasets shared into a Snowflake account and queried in place | Free, trial and paid listings | +| [AWS Data Exchange](https://aws.amazon.com/data-exchange/) | Datasets delivered into AWS, billed through AWS | Per-product subscriptions | +| [Eagle Alpha](https://www.eaglealpha.com/) | Alternative data discovery and advisory, 2,500+ dataset profiles | Institutional | +| [Neudata](https://www.neudata.co/) | Alternative data discovery and research, 7,000+ datasets in its catalog | Institutional | +| [Datarade](https://datarade.ai/platforms) | Directory of data providers and data marketplaces | Free to browse; each provider sets its own terms | + +Of the free datasets that made Quandl popular, the WIKI prices table is frozen at April 2018 +(`ml4t-data` wraps it as [Wiki Prices](wiki_prices.md)), and the Zillow real-estate database +no longer has a catalog page and was last refreshed in July 2025. + +--- + +## Alternative Data in ml4t-data + +`ml4t-data` wraps two prediction-market sources, [Kalshi](kalshi.md) and +[Polymarket](polymarket.md). The FXMacroData provider also exposes news, risk-sentiment and CFTC +positioning endpoints alongside its release-timestamped macro data. Everything else on this page +needs its own client. + +--- + +## Name Changes and Closures + +| Then | Now | +|------|-----| +| data.ai (formerly App Annie) | Part of Sensor Tower since 2024 | +| Earnest Analytics | Part of Consumer Edge | +| SafeGraph Patterns (foot traffic) | Sold to Advan Research; SafeGraph now sells places, geometry and spend data | +| Orbital Insight | Acquired by Privateer, May 2024 | +| MarineTraffic | Acquired by Kpler, March 2023 | +| Spire Maritime | Acquired by Kpler, April 2025 | +| Refinitiv ESG scores | LSEG ESG data; a new LSEG ESG Scores suite launched March 2026 | +| Reddit and X APIs | Paid access (Reddit since July 2023; X pay-per-use since February 2026) | +| Stocktwits API | Closed to new developers | +| Polymarket (closed to US users from 2022) | [Amended CFTC order of designation](https://www.prnewswire.com/news-releases/polymarket-receives-cftc-approval-of-amended-order-of-designation-enabling-intermediated-us-market-access-302625833.html) for intermediated US access, November 2025 | +| PredictIt | [CFTC approval to operate a regulated exchange](https://www.bloomberg.com/news/articles/2025-09-05/predictit-gains-cftc-approval-to-launch-regulated-exchange), September 2025 | +| Quandl | Nasdaq Data Link (September 2021) | + +## See Also + +- [Provider comparison](index.md) +- [Market data sources](market_data.md) +- [Fundamental data sources](fundamentals.md) diff --git a/docs/providers/index.md b/docs/providers/index.md index da77dc0..5a76718 100644 --- a/docs/providers/index.md +++ b/docs/providers/index.md @@ -2,6 +2,10 @@ ML4T Data supports 20+ live and specialized data providers, plus synthetic and testing providers. +For the wider vendor landscape, including sources the library does not wrap, see +[Market Data Sources](market_data.md), [Fundamental Data Sources](fundamentals.md) and +[Alternative Data Sources](alternative_data.md). + ## Provider Comparison | Provider | Asset Class | Free Tier | Async | API Key | diff --git a/docs/providers/market_data.md b/docs/providers/market_data.md new file mode 100644 index 0000000..2b67c84 --- /dev/null +++ b/docs/providers/market_data.md @@ -0,0 +1,175 @@ +# Market Data Sources + +This page maps the market data vendor landscape: US equities and ETFs (daily and intraday), +futures, options, FX and crypto, including the institutional and academic sources that +`ml4t-data` does not wrap. For the providers the library does wrap (free tiers, async support, +credentials), the [provider comparison](index.md) is the reference; this page links to it rather +than repeating it. + +!!! note "Verified September 2026" + Vendor coverage, history, access terms and ownership change often. Every entry below was + checked in September 2026 against the vendor's own documentation or a published announcement. + Where a vendor does not document a property, the table says "not documented" instead of + guessing. Confirm the current terms before you build a dataset on any of them. + +--- + +## Choosing a Source + +### Survivorship + +A research universe must contain the securities that later delisted, merged or went bankrupt. +Free price sources usually carry only what trades today, so any cross-sectional backtest built +on them overstates returns. The "Delisted" columns below say whether a vendor keeps dead +securities. + +### Adjustments and corporate actions + +Splits, dividends and spin-offs break raw price series. Vendors differ in what they deliver: +unadjusted prices, adjusted prices, or both with the adjustment factors. Keep the unadjusted +series and the factors if you can: adjusted history is rewritten every time a new corporate +action occurs, so an adjusted series downloaded today differs from one downloaded last year. +Futures have the equivalent problem at each roll; a continuous contract is a construction choice, +not a market price. + +### Point-in-time universe membership + +An index strategy must trade the constituents as of each date, not today's members. Only a few +vendors sell historical index membership; without it, "S&P 500 stocks" silently means today's +survivors. + +### Granularity and timestamps + +Daily bars, minute bars, trades and quotes, and full order-book messages are different products. +For US equities, a consolidated (SIP) feed and a single-venue feed (for example IEX only) report +different volumes and prices. Check which timestamp a vendor records (exchange, SIP or receipt) +and how it defines the trading session. + +### Licensing + +Exchange data carries its own license terms. Redistribution, display and derived-data rights +vary by vendor and tier, and a dataset licensed for research may not be licensed for production. + +### In the book + +- Chapter 2, "The asset-class market data landscape" (section 2.2): what each asset class lets + you observe, its failure modes, and the engineering decisions it forces. +- Chapter 2, "A due diligence framework for data sourcing" (section 2.3): survivorship, + corporate actions, identifier integrity, and a vendor checklist. +- Chapter 3, "The anatomy of modern market data feeds" (section 3.2) and "Microstructure data + quality and sessionization" (section 3.6): feeds, timestamps and session handling for intraday + data. + +--- + +## Providers ml4t-data Wraps + +Yahoo Finance, Alpaca, EODHD, Tiingo, Twelve Data, Massive (formerly Polygon.io), Finnhub, +Databento, Oanda, Binance, Binance Public, OKX, CoinGecko, CryptoCompare, Wiki Prices and NASDAQ +ITCH samples all have `ml4t-data` providers. Their asset classes, free tiers and credentials are +in the [provider comparison](index.md) and on each provider's page. Three of them appear in the +tables below as well, because they are also the reference source for a data class: Databento +(order-book and CME data), Alpaca (US equities with a stated history start) and the NASDAQ ITCH +samples. + +--- + +## US Equities and ETFs: Daily Research Panels + +| Source | Coverage and history | Delisted | Adjustments | Finest granularity | Point-in-time index membership | Access | ml4t-data | +|--------|----------------------|----------|-------------|--------------------|--------------------------------|--------|-----------| +| [CRSP US Stock Databases (on WRDS)](https://wrds-www.wharton.upenn.edu/pages/about/data-vendors/center-for-research-in-security-prices-crsp/) | NYSE monthly from December 1925 and daily from July 1962; NASDAQ from December 1972 | Yes, keyed by PERMNO and PERMCO | Returns with and without dividends; full corporate-action history | Daily and monthly | Yes, index constituent files on WRDS | Academic and institutional, mainly via WRDS | No | +| [Norgate Data](https://norgatedata.com/data-content-tables.php) | US, Australian and Canadian stocks and ETFs; US listed and delisted coverage essentially complete from late 1992 | Yes, on the Platinum and Diamond tiers | Four modes: unadjusted, capital-reconstruction adjusted, plus special distributions, total return | Daily | Yes, on the Platinum and Diamond tiers | Retail paid subscription | No | +| [Sharadar Equity Prices (SEP)](https://data.nasdaq.com/databases/SEP) | About 21,000 active and delisted US tickers, from 1998 | Yes | OHLCV adjusted for splits and stock dividends; an unadjusted close; a close also adjusted for cash dividends and spin-offs | Daily | Separate S&P 500 constituents table (`SHARADAR/SP500`) | Paid; direct at sharadar.com since July 2026, or Nasdaq Data Link | No | +| [FirstRate Data](https://firstratedata.com/about/FAQ) | 16,000+ US tickers from 2000, including 7,000+ delisted | Partly: included where the vendor could source them | Unadjusted, split-adjusted, and split and dividend adjusted | 1-minute bars | No | Retail paid, one-off purchase | No | +| [Kibot](https://www.kibot.com/faq.html) | US stocks and ETFs; minute bars from 1998 | Partly: active and delisted lists, stated as incomplete | Split and dividend adjusted series plus an adjustments API | Tick with bid and ask | No | Free guest access (daily only), paid tiers | No | +| [Alpaca](https://docs.alpaca.markets/us/docs/about-market-data-api) | US equities and ETFs, from 2016 | Not documented | `adjustment` parameter: raw, split, dividend, spin-off or all | Trades and quotes | No | Free (IEX feed, delayed SIP); paid for real-time SIP | Yes: `AlpacaProvider` | + +The free wrapped sources (Yahoo Finance, Tiingo, EODHD free tier) carry currently listed +securities and vendor-adjusted prices; treat them as prototyping data, not as a survivorship-free +panel. Wiki Prices stopped updating in April 2018. + +--- + +## US Equities: Trades, Quotes and Order Books + +| Source | Coverage and history | Delisted | Adjustments | Finest granularity | Point-in-time index membership | Access | ml4t-data | +|--------|----------------------|----------|-------------|--------------------|--------------------------------|--------|-----------| +| [NYSE TAQ](https://wrds-www.wharton.upenn.edu/pages/about/data-vendors/nyse-trade-and-quote-taq/) | All US exchanges; Monthly TAQ 1993 to 2014, Daily TAQ (millisecond timestamps) from September 2003 | Not applicable: daily files of every security traded that day | Raw prices; adjust with CRSP factors | Trades and NBBO quotes | No | Academic and institutional via WRDS | No | +| [AlgoSeek](https://algoseek.com/us-equities-package/) | Every listed US equity from 2007 (SIP), including delisted securities; adjustment factors from 2007 | Yes | Raw and adjusted prices plus adjustment-factor files | Trades and quotes with nanosecond timestamps; bars from 1 second | Yes, Index Components dataset | Institutional; free sandbox with up to one year of data | No | +| [Databento](https://databento.com/datasets/XNAS.ITCH) | US equities across exchanges and ATSs; Nasdaq TotalView-ITCH from 2018 | Not applicable: raw exchange feeds | Unadjusted; corporate actions sold as separate reference data | Full order book (market by order) | No | Usage-based; see the [Databento page](databento.md) | Yes: `DataBentoProvider` | +| [LSEG Tick History](https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history) (formerly Refinitiv Tick History) | Cross-asset, 580+ venues and contributors, from January 1996 | Not documented | Not documented | Trades, quotes and order-book depth | No | Institutional; some universities license it | No | +| [Nasdaq TotalView-ITCH samples](https://emi.nasdaq.com/ITCH/Nasdaq%20ITCH/) | Selected full-day files for Nasdaq-listed stocks | Not applicable | Raw messages | Every order add, cancel and execution | No | Free samples; the full archive is a paid Nasdaq product | Yes: [NASDAQ ITCH provider](nasdaq_itch.md) | +| [DTN IQFeed](https://www.iqfeed.net/dev/) | Equities, futures, options, FX; tick history rolling 180 days, minute bars from 2005 to 2007 depending on asset | Not documented | Not documented | Trades and quotes | No | Retail paid | No | +| [Kinetick](https://kinetick.com/) | Market data service bundled with NinjaTrader; stocks, futures, FX | Not documented | Not documented | Tick | No | Retail; free end-of-day tier | No | + +--- + +## Futures + +| Source | Coverage and history | Expired contracts | Continuous series | Finest granularity | Access | ml4t-data | +|--------|----------------------|-------------------|-------------------|--------------------|--------|-----------| +| [CME DataMine](https://www.cmegroup.com/datamine.html) | CME, CBOT, NYMEX and COMEX futures and options; history varies by dataset, some back to the 1970s | Yes, data is per contract | No, individual contracts | Market depth and time and sales | Paid per dataset; academic discount | No | +| [Databento CME Globex (GLBX.MDP3)](https://databento.com/datasets/GLBX.MDP3) | CME Group futures and options from June 2010; full order-by-order depth from March 2017 | Yes, per contract | Continuous symbology by roll rule | Full order book | Usage-based | Yes: `DataBentoProvider.fetch_continuous_futures()` | +| [Norgate Data](https://norgatedata.com/futurespackage.php) | Global futures and cash commodities | Yes, individual contract histories | Unadjusted and back-adjusted continuous contracts | Daily | Retail paid | No | +| [FirstRate Data](https://firstratedata.com/it/futures) | 130 most active futures, intraday from 2007 to 2008 | Yes, individual contracts | Unadjusted, absolute-adjusted and ratio-adjusted continuous series | 1-minute bars | Retail paid | No | + +--- + +## Options + +| Source | Coverage and history | Delisted underlyings | Analytics | Finest granularity | Access | ml4t-data | +|--------|----------------------|----------------------|-----------|--------------------|--------|-----------| +| [OptionMetrics IvyDB](https://optionmetrics.com/united-states/) | US equity and index options from 1996; Europe from 2002 | Yes | Implied volatility, Greeks, volatility surfaces; adjusts for dividends, splits and spin-offs | Daily; a separate intraday product exists | Academic via WRDS, institutional | No | +| [ORATS](https://orats.com/data-api) | US equity, ETF and index options; end of day from 2007, 1-minute from August 2020 | Not documented | Smoothed implied volatility and Greeks | 1-minute snapshots | Retail paid, enterprise | No | +| [Cboe DataShop](https://datashop.cboe.com/documentation) (LiveVol) | Options trades, quotes and end of day from January 2012; VIX futures trades and quotes from April 2004 | Not documented | Greeks and implied volatility (Cboe Hanweck) as separate products | Trades and quotes | Usage-based | No | +| [Databento OPRA](https://databento.com/datasets/OPRA.PILLAR) | All US equity options from April 2013 | Not applicable | None: raw feed | Trades and quotes | Usage-based | Yes: `DataBentoProvider.fetch_option_chain()`, `fetch_option_quotes()` | +| [AlgoSeek](https://algoseek.com/us-equities-package/) | US equity options, full OPRA feed | Not documented | Greeks and implied volatility dataset | Trades and quotes | Institutional | No | + +--- + +## Foreign Exchange + +Spot FX trades over the counter, so there is no consolidated tape: every source reports one +dealer's or one venue's prices, and volumes are not comparable across sources. + +| Source | Coverage and history | Finest granularity | Access | ml4t-data | +|--------|----------------------|--------------------|--------|-----------| +| [Dukascopy](https://www.dukascopy.com/swiss/english/marketwatch/historical/) | FX, CFDs, commodities, indices from one Swiss bank's platform | Tick | Free download | No | +| [TrueFX](https://www.truefx.com/truefx-historical-downloads-2/) | Major and cross pairs, top of book | Tick, millisecond timestamps | Free historical downloads with registration | No | +| [LSEG Tick History](https://www.lseg.com/en/data-analytics/market-data/data-feeds/tick-history) | Contributed and venue FX quotes from 1996 | Tick | Institutional | No | +| [Oanda](oanda.md) | One broker's quotes | Seconds-level candles | Account required | Yes: `OandaProvider` | + +--- + +## Crypto + +Crypto trades on many venues around the clock, and reported volume on some venues is unreliable. +A source that keeps delisted pairs and closed exchanges (FTX, for example) matters for the same +survivorship reason as delisted stocks. + +| Source | Coverage and history | Delisted pairs and venues | Finest granularity | Access | ml4t-data | +|--------|----------------------|---------------------------|--------------------|--------|-----------| +| [Kaiko](https://www.kaiko.com/products/l1-l2-data) | 100+ centralized exchanges plus DeFi; trades from 2010, order-book snapshots from 2015 | Not documented | Trades, best bid and offer, full order book | Institutional | No | +| [Tardis.dev](https://docs.tardis.dev/historical-data-details/overview) | 50+ exchanges, spot, derivatives and options; most from March 2019 | Yes: closed venues such as FTX and BitMEX history retained | Raw exchange messages, full order-book depth | Paid; academic, solo and pro tiers cover the last four years, business tiers from March 2019 | No | +| [CoinAPI](https://www.coinapi.io/products/market-data-api) | 400+ exchanges; spot, futures, perpetuals, options | Not documented | Trades, quotes, order books | Free credits, paid, enterprise | No | +| Binance, Binance Public, OKX, CoinGecko, CryptoCompare | Venue or aggregator data; see the [provider comparison](index.md) | Venue-specific | Bars; funding rates and premium index for perpetuals | Free or free tier | Yes | + +--- + +## Name Changes and Closures + +| Then | Now | +|------|-----| +| CRSP (University of Chicago) | Owned by Morningstar since February 2, 2026; the CRSP Market Indexes were renamed Morningstar Market Indexes in 2026 | +| Thomson Reuters / Refinitiv Tick History | LSEG Tick History | +| Polygon.io | Massive (October 30, 2025); `api.polygon.io` still answers alongside `api.massive.com` | +| Sharadar on Nasdaq Data Link only | Also sold direct at sharadar.com since July 2026 | +| Quandl WIKI Prices | Frozen since April 2018; still available as a static dataset | +| IEX Cloud | Retired August 31, 2024 | + +## See Also + +- [Provider comparison](index.md) +- [Fundamental data sources](fundamentals.md) +- [Alternative data sources](alternative_data.md) diff --git a/mkdocs.yml b/mkdocs.yml index 83f771d..cc9b7fe 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -124,6 +124,8 @@ nav: - Providers: - providers/index.md - Fundamental Data Sources: providers/fundamentals.md + - Market Data Sources: providers/market_data.md + - Alternative Data Sources: providers/alternative_data.md - Yahoo Finance: providers/yahoo.md - Alpaca: providers/alpaca.md - CoinGecko: providers/coingecko.md