Volodymyr Shcherbyna
Back to Engineering

TalentBank live ↗

CV-processing pipeline: sequenced ETL steps, a resumable state machine and LLM extraction.

Table of Contents

Context

TalentBank is a DevBrother platform that turns uploaded CVs into structured candidate records. I built the processing backend: Python, Django, a relational database, an LLM (OpenAI API), Docker and Nginx.

Problem

A CV goes through several dependent stages: PDF ingestion, text extraction, structured extraction by a language model, and generation of the output document. Any stage can fail, and a model call can return garbage. Treating the whole thing as one request meant starting over on every failure and hours of work per CV.

Approach

  • A step-based ETL pipeline with forced sequencing, so a stage cannot run before the one it depends on.
  • A status field on every item acts as a state machine stored in the database. Each item resumes from the last good stage instead of from the start.
  • Structured extraction by an LLM (OpenAI API) with retries and a maximum-attempts limit, so a bad response is retried a bounded number of times and then surfaced instead of looping.
  • PDF ingestion, text extraction and PDF generation as separate, testable stages.
flowchart LR
  U[Upload] --> I[ingest PDF] --> T[extract text] --> L[LLM extraction] --> G[generate]
  L -. retry, max attempts .-> L

Outcome

Processing one CV went from hours to about half an hour, and a failed item no longer costs the work of the stages that already succeeded.

Details are limited to the technical side because the product belongs to DevBrother.