UNMAPPED, Skill mapping for the informal economy, across three engines

The problem

In most low and middle income countries the majority of young workers are in the informal economy. They have real skills, often substantial ones, and almost none of it is legible to any formal system. There is no transcript for running a repair shop, no certificate for three years of site work, no structured record a hiring platform or a training program can read.

That illegibility is the actual barrier. Programs designed to move people into better work have to start by guessing what those people can already do, and they usually guess badly.

UNMAPPED was built for the World Bank track of Hack-Nation's 5th Global AI Hackathon, with Amit Kumar Gupta. Around 300 teams competed in the track. We had 18 hours.

Constraints

The hackathon constraints turned out to be the same constraints the real problem has, which is unusual and useful.

No labelled data. There is no dataset of informal-economy skill profiles to fine-tune on. Whatever we built had to work from unstructured descriptions of what someone actually does day to day.

Cost per profile has to approach zero. A system meant for national-scale deployment in an LMIC cannot spend a dollar per person. That ruled out routing everything through a large hosted model.

18 hours. This is not a footnote. It eliminated every architecture that required provisioning infrastructure before writing any logic, and that elimination turned out to produce a better system rather than a worse one.

Architecture: three engines

The core decision was to stop treating this as one model call and split it into three engines with different cost profiles, so the expensive one runs least.

Semantic matching

fastembed with ONNX runtime generates embeddings locally, and pgvector on Neon stores and searches them. Running embeddings on CPU through ONNX rather than calling a hosted embedding API is what makes the per-profile cost approach zero. It also removes a rate-limited network dependency from the hot path.

Neon mattered for a boring reason that turned out to be the right one. pgvector inside ordinary Postgres means the vectors live next to the relational data, so a similarity search and a metadata filter are one query rather than two systems kept in sync.

The graph engine

This is the part that makes UNMAPPED something other than a search box.

The graph is built in NetworkX, held in memory, and pre-seeded at startup from ESCO, the European Skills, Competences and Occupations taxonomy. Roughly 500 skill nodes and 2,000 weighted edges.

The edges carry four different relationships:

• skill to adjacent skill
• skill to occupation
• occupation to wage
• occupation to automation risk, using Frey-Osborne scores

A traversal takes a person's current skills and walks one to two hops out. What comes back is a ranked list of adjacent skills, ranked by wage premium and automation resilience.

That ranking is the whole idea. Skill mapping is not a nearest-neighbour problem. The useful question is not "what is this person's skill", which they already know, but "what is worth learning next from where they are standing". Answering it means combining proximity, which the graph gives you, with economic value and durability, which the edge weights carry. A recommendation that is one hop away but heading into work that automates in five years is a bad recommendation, and only a graph that knows about automation risk can decline to make it.

Why NetworkX and not Neo4j

A graph database is the obvious tool and would have been the wrong one here.

Neo4j means provisioning, a connection, a schema, and a query language, all before the first traversal runs. NetworkX is pure Python, loads in seconds, and at 500 nodes and 2,000 edges traversal is effectively instant. There is no scale at which this graph benefits from a database, because the graph fits comfortably in memory and always will at this granularity.

In an 18-hour build the setup overhead alone decided it. But the decision holds outside the hackathon frame too: adding a database to hold two thousand edges is infrastructure you maintain forever in exchange for nothing.

Language understanding

NVIDIA Nemotron, accessed through OpenRouter, handles the part that genuinely needs a language model: turning a messy free-text description of someone's work into structured input the other two engines can act on.

Putting the LLM last, behind the cheap engines, is the cost argument. Most of the work never reaches it.

Decisions and trade-offs

Local embeddings over a hosted embedding API. Slower on cold paths, but the cost curve is flat rather than linear in users, and there is one fewer service that can rate-limit you mid-demo. For a system whose premise is national scale under cost constraints, the flat curve is the only defensible choice.

Postgres with pgvector over a dedicated vector database. A dedicated store would be marginally faster at retrieval and would have meant two systems, two consistency stories, and two things to provision. One Postgres was right at this size and stays right well past it.

A REST API rather than a demo UI. A judged hackathon rewards a visible interface, so this was a real trade. We built the API because skill mapping is infrastructure: what matters is whether a ministry or an NGO platform can call it, not whether we shipped a nice screen.

The decision I would reverse

Deploying too late.

We started deployment early by hackathon standards and it was still too late. The backend bundled the sentence embedding model directly into the service, which made it large, and that size caused repeated failures on free-tier hosting through the build-up. Every one of those failures came out of the 18 hours.

The fix is not subtle in hindsight: deploy a skeleton in the first hour, before there is anything worth deploying, so the hosting constraints are known while there is still time to design around them. The embedding model being in-process is exactly the kind of decision that looks free locally and is not free on a free tier.

Outcome

2nd place globally in the World Bank track, out of roughly 300 competing teams.

The judges pushed hardest on two things, and both were fair. The first was latency. The second was the gap between a working demo and real-world deployment, specifically how much on-the-ground implementation work would sit between this and anything running in an actual program. That is the right question to ask of a hackathon project aimed at development policy, and having a good architecture is not an answer to it.

What holds up months later is the shape rather than the code: three engines with different cost profiles, cheap ones first, the expensive one last and rarely. That structure is the reason it could plausibly run at the scale the problem actually has.