Reddit Posts & Comments Scraper in Python — No API Key, No OAuth

Pull Reddit posts, full comment trees, subreddit stats and user profiles into Python as clean JSON — without registering a Reddit app or paying for API access.

This repository is a ready-to-run Python project built around the Reddit Scraper All-in-One Apify actor. One actor covers keyword search, subreddit feeds, complete comment threads, user profiles and community metadata — the code here shows how to chain those modes into real research workflows.

What this does

Since 2023, Reddit's official Data API requires app registration, OAuth credentials and a paid agreement for anything beyond hobby volume.

The actor behind this repo acts as an unofficial Reddit API. You send a JSON input; it returns structured records:

• Posts — up to 80 fields each: title, body (text and HTML), author and flair, score, upvote ratio, comment count, subreddit and its subscriber count, permalink and old.reddit URL, NSFW/spoiler/locked/archived/edited flags, crosspost lineage and full media details.
• Comments — up to 44 fields each, with postId, parentId, depth, replyIds and hasMoreReplies, so you can rebuild the exact reply tree.
• Communities — members, active users, descriptions, creation date, language, icons and posting rules.
• Users — karma split, account age, moderator and employee flags, recent posts and comments, plus emails, phones and social links pulled from websites in their bio.

It also computes engagement metrics you would otherwise derive by hand: ageHours, scorePerHour, commentsPerHour, engagementTotal, commentToScoreRatio, wordCount.

Use cases

• Customer pain-point research — find the threads where your buyers complain, then read every reply. main.py does exactly this.
• Brand and competitor monitoring — search your product name sorted by new over the past day and push fresh mentions to a sheet every morning.
• Subreddit analysis — top posts of the month or year from any community, with top comments embedded, to see what format and topics win there.
• Training and evaluation data — export long discussion threads with nesting preserved for LLM summarisation, RAG or sentiment models.
• Lead generation — collect users active in a niche and enrich their profiles with emails and socials from linked websites.
• Media collection — Reddit-hosted video URLs (MP4 fallback, DASH, HLS), gallery images in author order and preview renditions.

Requirements

• Python 3.10+
• A free Apify account — the free plan includes $5 of monthly credit
• No Reddit account, no Reddit app, no client ID or secret

Quick start

git clone https://github.com/pro100chok/reddit-posts-comments-scraper-python.git
cd reddit-posts-comments-scraper-python
pip install -r requirements.txt
cp .env.example .env
# put your APIFY_API_TOKEN into .env
python main.py

main.py validates a product idea (invoicing software for freelancers): it searches Reddit for three problem phrases ranked by top score over the past year, de-duplicates threads that matched several phrases, opens the three busiest discussions with their full comment trees (collapsed branches included), prints the most frequent words and the highest-voted replies, and writes output.json and output.csv.

How it works

Every run is driven by two fields: scrapeType says what you want and items says where to start, one value per line.

scrapeType	Put in items	You get
search	keywords, e.g. standing desk	matching posts — and optionally communities and users
subreddit	r/python, Entrepreneur or a subreddit URL	a community record, then posts with topComments embedded
post	post URLs or IDs	the post record followed by one record per comment
user	spez, u/spez or a profile URL	one user record with recent activity and contacts
url	any mix of Reddit links	type detected per link

Every record carries a dataType field (post, comment, community, user), so a mixed dataset filters in one line of pandas. Pagination, parallelism (up to 50 sources), retries and proxies are handled server-side.

Two details worth knowing:

1. Embedded comments are free in subreddit mode. Comments land inside the post's topComments array and are not billed as separate results. Fifty posts with twenty comments each cost fifty results.
2. Deep threads need expandMoreComments. By default you get what Reddit shows on first load. Turn it on and the actor follows the "load more comments" stubs until maxCommentsPerPost is reached.

Example: top posts in a subreddit with their best replies

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_API_TOKEN"])
run = client.actor("pro100chok/reddit-scraper-all-in-one").call(run_input={
    "scrapeType": "subreddit",
    "items": ["r/personalfinance"],
    "sort": "top",
    "time": "month",
    "scrapeComments": True,
    "maxCommentsPerPost": 10,
    "maxPostsPerSource": 30,
    "maxItems": 31,
})

for rec in client.dataset(run["defaultDatasetId"]).iterate_items():
    if rec.get("dataType") != "post":
        continue
    print(rec["score"], rec["title"])
    for c in rec.get("topComments") or []:
        print("   ", c["score"], c["body"][:100])

Example output

A trimmed comment record from post mode:

{
  "dataType": "comment",
  "id": "mq8xk2p",
  "postId": "1kx9q2z",
  "parentId": "t3_1kx9q2z",
  "depth": 0,
  "author": "sales_veteran",
  "body": "This matches my experience exactly. Cold outreach beat ads 10:1.",
  "score": 312,
  "controversiality": 0,
  "isSubmitter": false,
  "numberOfReplies": 4,
  "replyIds": ["mq9a1zz", "mq9b7kk", "mq9c0pd", "mq9d4xa"],
  "hasMoreReplies": true,
  "createdAt": "2026-05-20T15:42:03+00:00",
  "scorePerHour": 17.8286
}

Post records add title, upvoteRatio, numberOfComments, subredditSubscribers, flair, mediaType, redditVideo, galleryImages, previewImages, crosspostParent and the derived metrics above. Empty fields are dropped, so rows stay compact.

Input parameters

Parameter	Type	Default	Description
scrapeType	string	search	search, subreddit, post, user or url
items	string[]	—	Keywords, subreddits, post URLs/IDs, usernames or mixed URLs
maxItems	integer	50	Hard cap on records saved — your main cost control
sort	string	new	relevance, hot, top, new, rising, comments
time	string	all	all, hour, day, week, month, year
searchWithinCommunity	string	—	Restrict a keyword search to one subreddit (name without r/)
searchPosts / searchCommunities / searchUsers	boolean	true / false / false	Which result types a search returns
scrapeComments	boolean	true	Fetch comments in post and subreddit modes
maxCommentsPerPost	integer	50	Comment cap per post
expandMoreComments	boolean	false	Follow collapsed reply chains
includeCommunityDetails	boolean	true	Push a community record before each subreddit's posts
includeNSFW	boolean	true	Keep over-18 content
includeUserPosts / includeUserComments	boolean	true	Recent activity on user records
maxUserPosts	integer	25	Recent posts and comments per user
extractContacts	boolean	true	Crawl bio websites for emails, phones, socials
maxPostsPerSource	integer	50	Posts per subreddit or search term
maxConcurrency	integer	10	Sources processed in parallel (1–50)
maxRetries	integer	5	Retries per request (1–15)

More examples

File	Demonstrates
examples/01_basic_usage.py	Newest posts for one keyword from the past week.
examples/02_subreddit_top_posts_with_comments.py	Top posts of the year from two subreddits, community stats and embedded comments.
examples/03_user_profiles_with_contacts.py	User karma, account age, recent posts, emails, phones and social links.
examples/04_export_to_csv.py	One thread with every comment to CSV; depth, commenter and score analysis in pandas.
examples/05_export_to_google_sheets.py	Daily brand-mention feed appended to a Google Sheet without duplicates.

Ready-made tasks

Prefer to start from a preset? These public tasks run the same actor with a pre-filled input — open one, change the keywords and press Start:

• Search Reddit Posts for Any Keyword
• Export Hot and New Posts from Any Subreddit
• Scrape a Reddit Thread with All Its Comments
• Export a Reddit User's Posts and Comments
• Find Reddit Leads with Emails and Socials
• Find the Best Subreddits for Any Topic

Pricing

The actor is pay-per-result: $2.00 per 1,000 results on the free Apify plan, dropping to $1.80 (Bronze), $1.70 (Silver) and $1.50 per 1,000 on Gold and above. A run start costs $0.00005 per GB of memory. Proxies and retries are included. A 100-post subreddit export with embedded comments is about $0.20; the free monthly credit covers roughly 2,500 results.

FAQ

Do I need Reddit API credentials? No. You need only an Apify API token. There is no Reddit app, client ID, secret or OAuth flow to set up.

How is this different from PRAW? PRAW is a wrapper around the official API, so it carries the same credential requirements and rate limits. This actor returns similar public data through a single call, with pagination, retries and proxies handled server-side.

Can I get every comment in a large thread? Use scrapeType: "post", set maxCommentsPerPost high and enable expandMoreComments. Each comment comes back with depth and parentId; hasMoreReplies marks any place Reddit still truncated the tree.

Can I search inside one subreddit only? Yes — set searchWithinCommunity to the subreddit name, e.g. "SaaS", and keep scrapeType: "search".

How are emails found? For user profiles, the actor reads links in the bio, visits up to five linked websites (home page plus likely contact pages) and extracts emails, phone numbers and links to social platforms. It only finds what people publish themselves.

Is scraping Reddit legal? The actor reads only publicly visible content without logging in. Usernames and contact details are personal data under GDPR and similar laws — store and use them accordingly.

Related actors

• Threads Scraper — posts, profiles and replies from Meta's Threads, no login.
• YouTube Scraper All-in-One — videos, Shorts, comments and transcripts for the same topics.
• Instagram Scraper All-in-One — profiles, posts, reels and comments.
• Website Contact Scraper — bulk emails, phones and socials for any list of domains.

License

MIT — see LICENSE.

---

Built on the Reddit Scraper All-in-One Apify actor by pro100chok.