Update: I scraped 5.9 million jobs with ChatGPT

I got sick and tired of how LinkedIn & Indeed is contaminated with ghost jobs and 3rd party offshore agencies, making it nearly impossible to navigate.

I discovered that most companies post jobs directly on their websites. Until recently, there was no way to scrape them at scale because each career page has different structure and format. After playing with ChatGPT's API, I realized that you can effectively dump raw domain names and ask it to give you all formatted jobs back in JSON (ex salary, yoe, etc).

Update: I’ve now used this technique to scrape 5.9 million jobs (with over 220k remote jobs) and built powerful filters. I made it publicly available here in case your'e interested Hiring.Cafe.

Pro tips:

• You can select multiple job titles and job functions (and even exclude them) under "Job Filters"
• Filter out or restrict to particular industries and sectors (Company -> Industry/Keywords)
• Select IC vs Management roles, and for each option you can select your desired YOE
• … and much more