Job Scraper
Overview
I created a system to automatically find, verify, and score these jobs while I do not have to manually go through the process of sifting through the job boards. This has saved me a ton of time of by discarding listings that were looking for senior-level positions, disguised commission-only gigs or ultimately, outright scams.
Services
- n8n
- AI/LLM Integration
- Autonomous
Core Features
- Multi-Platform Scraping
- LLM Fraud Detection
- Automated Scoring
Integrations
- Gemini 3.1 Flash Lite
- Google Sheets
- Gmail API
Status
- Personal Automation
- Ongoing
System Design
Architecture and Workflow
Trigger: Runs on a 15-minute schedule during active
hours.
Processing:
1. Fetches listings from LinkedIn and OLJ using injected
keywords.
2. Deduplicates against a historical Google Sheet to prevent
redundant processing.
3. Fetches the full job description.
4. Uses Google Gemini LLM to analyze the description for fraud,
scoring it from "Safe" to "High Risk".
5. Scores the listing based on custom keyword weighting (e.g., n8n =
20pts, Webhook = 15pts).
Output: High-scoring, safe listings are formatted
into a clean HTML email and sent via Gmail.
Sample Output
Curated Daily Digest
Instead of raw JSON or spreadsheet rows, the final output is a beautifully formatted HTML email. It highlights the matched keywords, displays the LLM's risk assessment badge (e.g., Safe or Moderate Risk), and provides a direct link to apply. The layout follows familiar UI conventions seen on major job sites.
Built to Handle Failure
Resilient by Design
Every point where this automation could break silently was accounted for. API calls to Gemini and Google Sheets automatically retry with backoff if they fail, a single blocked or unavailable job listing does not halt the entire run, the AI's risk assessment is validated against a strict schema before it is used, and even a day with zero matching listings still produces a clean, readable email instead of an empty one.
Behind the Scenes
Never Duplicate a Job Listing
Before anything gets scored or emailed, every newly scraped listing is automatically compared against the job history already logged in Google Sheets, so genuinely new postings move forward for fraud analysis while listings that are unchanged, updated, or no longer active are quietly filtered out and the same job never reaches your inbox twice.
The Outcome
The Job Scraper runs on a schedule, pulling automation-related listings from OnlineJobsPH and LinkedIn, filtering out duplicates against a Google Sheets history, then scoring and screening each new listing through keyword matching and an AI fraud-risk assessment before compiling the results into a digest email. It replaced manual job-board browsing with a pre-filtered, pre-screened shortlist, cutting out most of the repetitive searching and vetting that used to eat up the process.
The scraping logic depends on each platform's current HTML structure, so it can break silently if a site redesigns its layout. Keyword scoring is intentionally pattern-based rather than context-aware, and the AI risk assessment is a helpful signal rather than a guarantee. Overall, it is sufficient to what I want to need.