How to Scrape Instagram Comments With an API (curl + Python to CSV)

Getting all the comments from a public Instagram post sounds like an afternoon of work. It usually is not. I build SocialHolmes, a social data API, so I deal with this every day. Here is why it is awkward, which options you really have, and a small script that saves any public post's comments to CSV.

Why Instagram comments are hard to get

• Login walls. Comments sit behind "View all N comments", and Instagram pushes anonymous visitors toward a login prompt quickly.
• Rate limits. Hit the web endpoints too hard from one IP and you get throttled or blocked.
• Changing markup. The web app talks to internal GraphQL queries whose names and shapes change. A scraper that worked in March can quietly break by June.
• Pagination. Comments load in small pages (around 15 at a time) as you scroll, so you need a loop, plus deduplication, because pages overlap on busy posts.

Your options, honestly

1. The official Instagram Graph API. If you want comments on your own business or creator account, use it. It is the right tool: free, supported, stable. The catch is that it only returns comments on accounts connected to your app. It cannot fetch comments on someone else's public post, which is what most research, competitor and giveaway use cases need.

2. DIY scraping (Playwright/Selenium, or replaying the GraphQL calls). You get full control and no per-row fee, but you own the whole maintenance burden: login sessions, proxies, selector fixes, retries. Fine for a one-off; painful as a pipeline.

3. A paid scraping API. Someone else maintains the scraper and you call an endpoint. You trade money for time. I use SocialHolmes for the rest of this post (disclosure: I build it), but the pattern of start a run, poll, read pages is common to most of these services.

Walkthrough: comments from one post

Grab the post's shortcode: the part after /p/ in the URL. In instagram.com/p/C1a2B3cDEfG/ it is C1a2B3cDEfG.

Sign up for an API key (1,000 free results, no card), then export it:

export SOCIALHOLMES_KEY="sh_your_key_here" export API="https://api.socialholmes.com/api/public/v1" 

Start a run

curl -X POST "$API/scrape/instagram/comments" \ -H "Authorization: Bearer $SOCIALHOLMES_KEY" \ -H "Content-Type: application/json" \ -d '{ "parameters": { "shortcode": "C1a2B3cDEfG" }, "limit": 500 }' 

You get back 202 Accepted and a run object with an id and a status. limit is the most comments you want. It is also the most you can be billed.

Poll, then read

curl "$API/runs/RUN_ID" -H "Authorization: Bearer $SOCIALHOLMES_KEY" curl "$API/runs/RUN_ID/comments?limit=100&offset=0" -H "Authorization: Bearer $SOCIALHOLMES_KEY" 

Statuses are queued, running, succeeded, empty, failed and cancelled. Comments come back as JSON, up to 100 per page; use offset to page.

The same thing in Python, saved to CSV

import csv import json import os import time import requests API = "https://api.socialholmes.com/api/public/v1" HEADERS = {"Authorization": "Bearer " + os.environ["SOCIALHOLMES_KEY"]} DONE = {"succeeded", "empty", "failed", "cancelled"} def call(method: str, path: str, **kwargs) -> dict: r = requests.request(method, f"{API}{path}", headers=HEADERS, timeout=30, **kwargs) if not r.ok: # Errors come back as {"error": {"code": ..., "message": ...}}  raise RuntimeError(f"{r.status_code}: {r.text}") return r.json() def get_comments(shortcode: str, limit: int = 500, max_wait: int = 600) -> list[dict]: run = call("POST", "/scrape/instagram/comments", json={"parameters": {"shortcode": shortcode}, "limit": limit}) deadline = time.time() + max_wait while run["status"] not in DONE: if time.time() > deadline: raise TimeoutError(f"run {run['id']} still {run['status']}") time.sleep(5) run = call("GET", f"/runs/{run['id']}") if run["status"] != "succeeded": print("Run ended as:", run["status"], run.get("reason"), run.get("message")) return [] comments, offset = [], 0 while True: page = call("GET", f"/runs/{run['id']}/comments", params={"limit": 100, "offset": offset}) comments += page["comments"] offset += page["page"]["returned"] if page["page"]["returned"] == 0 or offset >= page["page"]["total"]: return comments rows = get_comments("C1a2B3cDEfG", limit=500) with open("comments.csv", "w", newline="", encoding="utf-8") as f: w = csv.writer(f) w.writerow(["id", "username", "text", "likes", "replies", "parent_id", "created_at", "raw_json"]) for c in rows: # Empty fields are left out of the JSON, so use .get() rather than [].  w.writerow([ c["id"], c.get("author", {}).get("username"), c.get("text"), c.get("like_count"), c.get("reply_count"), c.get("parent_id"), c.get("created_at"), json.dumps(c.get("raw_json"), ensure_ascii=False), ]) print(f"Saved {len(rows)} comments") 

A few notes on what comes back:

• Each comment has id, text, created_at, like_count, reply_count and the author under author (author.username, author.id). Fields with no value are omitted, not sent as null.
• Each comment also carries raw_json: the comment exactly as Instagram returned it. The script saves it as a JSON string in the last CSV column, so if you later need a field we do not map (or Instagram adds one), it is already in your file.
• Replies arrive in the same list with a parent_id pointing at the comment they answer. There is no separate Instagram replies endpoint.
• Deduplicate on id if you stitch several runs together.

If you do not want to write code at all, the dashboard has a form (Instagram, Comments, paste the shortcode) and a Download CSV button.

The Claude / MCP angle

If you use Claude or another MCP client, SocialHolmes exposes an MCP server, so you can skip the script and ask "get the first 200 comments on instagram.com/p/C1a2B3cDEfG". It runs the same endpoint and costs the same.

What it costs

Pricing is $0.50 per 1,000 results on every endpoint, and you pay only for rows returned. A run that returns nothing costs nothing. Some numbers:

New accounts get 1,000 free results, which is enough to try this on a handful of posts.

Gotchas

• Private accounts and posts are out of reach. Public data only.
• Counts rarely match. The number under the post includes comments hidden from blocked accounts, filtered by region or held for moderation, and deleted comments are gone. Plan around the number you actually got. A script that retries until the counts match will never finish.
• limit caps your bill. Set it every time, especially in loops over many posts.
• Busy posts shift while you page. Ask for what you need in one run with a higher limit rather than several small runs.

A word on legality

Comments are public, but a stored record of who said what is personal data under GDPR and similar laws. Stick to public data, respect rate limits, think about whether you really need usernames (the text and timestamp are often enough), and check the law and Instagram's terms for your jurisdiction and use case. This is not legal advice.

Where to go next

• Comparing options and prices: the cheapest way to get Instagram data via API
• Building something on top: SocialHolmes for developers
• Full endpoint reference: API docs
• The original version of this post: How to scrape Instagram comments

If you have hit an Instagram scraping edge case I did not cover, tell me in the comments and I will try to reproduce it.