DarKnight: RAG Agent Solves Dark Web CAPTCHAs

Background and Context

Cyber threat intelligence (CTI) investigators monitor dark web markets to track leaked account credentials and data breach listings. Before a crawler can see any page, it must pass the CAPTCHAs that market operators set up to block automated collection. These challenges differ from site to site. The gate is therefore a daily operational hurdle, not a rare edge case.

Two approaches exist today. The first fine-tunes a model for each site. A new market means new data collection and new training, which costs too much and does not scale. The second calls a commercial vision-language model (VLM) API. That choice hands control to a vendor. A change in vendor policy, Tor exit blocking, government regulation or a silent model update can break the solver or change its behavior overnight. Every call and every retry also adds to the bill.

DarKnight, described in a new arXiv paper, targets this situation. It is a self-improving agentic framework built to run on a single workstation GPU beside the investigator's Tor crawler. It uses only a small, locally hosted open-weight VLM. It needs no per-market training and no third-party service, so no provider decision can take the solver away.

Deep Analysis

The core idea is to let a small model plan before it answers, instead of making the model larger. For each CAPTCHA, the system first retrieves earlier attempts on similar images. It pulls both successes and failures. The paper calls this dual-signal retrieval-augmented generation (RAG). Successful cases show what worked. Failed cases show what has already been proven not to work. Together they help the system avoid repeating known mistakes better than a success-only memory would. Next, the framework builds a preprocessing pipeline for that specific image. It selects a few tools from the available image-processing set and puts them in a useful order. It does not apply one fixed treatment to every image. This choice matters because uniform preprocessing can damage images that were already easy to read. The last step is a feedback loop. The target site returns a pass or fail result for each answer. DarKnight treats that result as a real learning signal and stores it for later retrieval on similar images. The source describes no weight updates and no human labeling in this loop. The system improves with use while it stays local and needs no per-market training.

The evidence comes from a case study on four real dark web markets with different levels of protection. On the best-protected market, a deployable open-weight VLM with DarKnight reached 83.2%. That is 37.2 percentage points above the same model without the framework. It is also above the frontier model Claude Opus 5, which scored 44%. In this study, agentic tool selection beat scaling up model size. The paper also reports how DarKnight changes individual outcomes. It fixed 48.2% of the baseline failures. It also caused 7.6% regressions, meaning cases the baseline had solved were now wrong. That regression cost is the price of acting on images at all. Its small size supports the authors' claim that choosing tools per image avoids most of the loss caused by uniform preprocessing.

Industry Impact

For security teams, the main gain is control. A solver that a vendor policy, a regional block or a model version change can interrupt is a weak link in a long-running intelligence pipeline. A local open-weight model removes that dependency. It also removes the per-call and per-retry fees that grow with crawl volume. Running next to the Tor crawler on one GPU keeps the setup within reach of a small investigative team.

The result also offers a pattern beyond this niche. The 83.2% versus 44% comparison suggests that a small model with retrieval and tools can beat a far larger commercial model on a narrow visual task. Teams that must work in restricted environments, or cannot depend on external APIs, can reuse the idea. The model gains skill at inference time from stored experience and external tools, not from more parameters.

There is a dual-use side that readers should note. The same capability that helps investigators pass operator-set gates is, in general, a CAPTCHA-solving method. The source frames the work around CTI monitoring, and this article follows that framing.

Outlook

The paper's evidence is limited to four markets. Whether the gains hold across more sites and over longer periods is not yet shown. Operators may change their CAPTCHAs in response, and the framework's reliance on site feedback means its memory must keep pace with those changes.

Several open questions follow from the design. How should the dual-signal memory choose and update its stored experience? How far can the tool set grow before selection becomes harder? Can the 7.6% regression rate fall further without lowering the 48.2% fix rate? Answers to these would show whether selective tool use can move from a promising case study to a dependable operating practice for CTI teams.