2.2 KiB
Executable File
Wellfound URL reference (recorded 2026-08-20)
robots.txt (https://wellfound.com/robots.txt)
Allowed: /role/..., /jobs/<id>-<slug>, company pages. Disallowed: /search, /_jobs/, /u/, /re/, /projects/, any ?jobId=/?jobSlug=/?role= query URLs, auth/profile paths. tools/robots_check.py returns ALLOWED for the role pages.
Search: SEO role landing pages
| Pattern | Example | Notes |
|---|---|---|
/role/l/<role-slug>/<location-slug> |
/role/l/senior-software-engineer/new-york |
200, ~800 KB HTML |
/role/r/<role-slug> |
/role/r/senior-software-engineer |
remote listings |
/role/<role-slug> |
404 (no location-less page) | |
?page=N / /<N> |
ignored / 404 - no server-side pagination |
Slugs: lowercase, non-alphanumerics → - (San Francisco → san-francisco, New York → new-york).
Data blob
<script id="__NEXT_DATA__" type="application/json" crossorigin="anonymous"> → props.pageProps.apolloState.data (a normalized Apollo cache):
ROOT_QUERY.talent.seoLandingPageJobSearchResults({"location":"new-york","page":1})→{ totalStartupCount, totalJobCount, perPage: 20, pageCount, startups: [{__ref: "StartupResult:<id>"}] }StartupResult:<id>:id, name, slug, companySize (SIZE_51_200), highConcept, logoUrl, badges[], highlightedJobListings: [{__ref: "JobListingSearchResult:<id>"}]JobListingSearchResult:<id>:id, title, slug, description (Markdown), compensation ("$172k – $253k"), liveStartAt (epoch seconds), locationNames[], remote (bool), remoteConfig, acceptedRemoteLocationNames[], jobType ("full-time"), primaryRoleTitle, yearsExperienceMin/Max, atsSource, autoPosted
Job URL: https://wellfound.com/jobs/<id>-<slug>.
Detail: /jobs/<id>-<slug> (slug optional; /jobs/<id> redirects)
200, ~85 KB. No __NEXT_DATA__; one <script type="application/ld+json"> block with a schema.org JobPosting: title, identifier, employmentType, hiringOrganization.name, description (HTML), datePosted, jobLocation.address{addressLocality, addressRegion, addressCountry}, baseSalary.value{minValue,maxValue}/currency, jobBenefits, directApply, url.
Failure modes seen
- Unknown role/location → 404 → CLI emits empty results.
- Cloudflare fronts the site; honest UA works at low volume.