Files
jobyssey/.agents/skills/wellfound-search/url-reference.md
T
2026-08-26 10:35:34 -04:00

2.2 KiB
Executable File
Raw Blame History

Wellfound URL reference (recorded 2026-08-20)

robots.txt (https://wellfound.com/robots.txt)

Allowed: /role/..., /jobs/<id>-<slug>, company pages. Disallowed: /search, /_jobs/, /u/, /re/, /projects/, any ?jobId=/?jobSlug=/?role= query URLs, auth/profile paths. tools/robots_check.py returns ALLOWED for the role pages.

Search: SEO role landing pages

Pattern Example Notes
/role/l/<role-slug>/<location-slug> /role/l/senior-software-engineer/new-york 200, ~800 KB HTML
/role/r/<role-slug> /role/r/senior-software-engineer remote listings
/role/<role-slug> 404 (no location-less page)
?page=N / /<N> ignored / 404 - no server-side pagination

Slugs: lowercase, non-alphanumerics → - (San Francisco → san-francisco, New York → new-york).

Data blob

<script id="__NEXT_DATA__" type="application/json" crossorigin="anonymous"> → props.pageProps.apolloState.data (a normalized Apollo cache):

  • ROOT_QUERY.talent.seoLandingPageJobSearchResults({"location":"new-york","page":1}) → { totalStartupCount, totalJobCount, perPage: 20, pageCount, startups: [{__ref: "StartupResult:<id>"}] }
  • StartupResult:<id>: id, name, slug, companySize (SIZE_51_200), highConcept, logoUrl, badges[], highlightedJobListings: [{__ref: "JobListingSearchResult:<id>"}]
  • JobListingSearchResult:<id>: id, title, slug, description (Markdown), compensation ("$172k – $253k"), liveStartAt (epoch seconds), locationNames[], remote (bool), remoteConfig, acceptedRemoteLocationNames[], jobType ("full-time"), primaryRoleTitle, yearsExperienceMin/Max, atsSource, autoPosted

Job URL: https://wellfound.com/jobs/<id>-<slug>.

Detail: /jobs/<id>-<slug> (slug optional; /jobs/<id> redirects)

200, ~85 KB. No __NEXT_DATA__; one <script type="application/ld+json"> block with a schema.org JobPosting: title, identifier, employmentType, hiringOrganization.name, description (HTML), datePosted, jobLocation.address{addressLocality, addressRegion, addressCountry}, baseSalary.value{minValue,maxValue}/currency, jobBenefits, directApply, url.

Failure modes seen

  • Unknown role/location → 404 → CLI emits empty results.
  • Cloudflare fronts the site; honest UA works at low volume.