Protego CVE-2026-55520

Exploitation requires three concurrent conditions: (1) the target application uses protego <= 0.6.1 to parse robots.txt; (2) the application fetches robots.txt from a domain the attacker controls or can influence (e.g., via DNS poisoning or by simply operating the target domain); and (3) the application subsequently calls can_fetch() with a URL that is a near-miss for the crafted wildcard pattern. No authentication or privileges are required of the attacker - serving a valid HTTP robots.txt response is sufficient. Exploitation is bounded by the requirement that the crawler must actually initiate a request to the attacker's domain: deployments restricted to a fixed allow-list of trusted domains are not exploitable in practice. The CVSS 4.0 UI:P metric reflects this client-initiated-fetch dependency. There is no requirement for a non-default configuration; any default Scrapy/protego deployment that crawls external sites is structurally exposed.

The CVSS 4.0 vector (AV:N/AC:L/AT:N/PR:N/UI:P/VA:H, score 7.1) accurately models the threat: a remote attacker with no credentials can cause complete availability loss in the vulnerable component, but only after the target application fetches the attacker-controlled robots.txt - captured by UI:P. No EPSS score was included in the intelligence package, so exploitation probability cannot be independently quantified; however, the published PoC (requiring only 12 wildcard pairs to trigger an indefinite freeze) substantially lowers the exploitation bar. There is no CISA KEV listing, indicating no confirmed widespread exploitation at time of analysis. Risk is moderate-to-high for web-crawling applications that routinely fetch robots.txt from arbitrary external servers - a standard operational pattern for Scrapy-based crawlers - because those deployments are structurally exposed to attacker-controlled robots.txt content as a matter of normal function. Risk is low for deployments restricted to known, trusted domains, since the crawler must initiate contact with the attacker's server for the exploit to trigger.

An attacker operates a web server and serves a crafted robots.txt containing a Disallow directive with many consecutive * wildcards (e.g., /a/*a*a*a*...*b with 12 or more wildcard pairs). When a Scrapy-based crawler or any other protego-using application fetches this file and calls can_fetch() with a near-miss URL such as /a/aaa...c, Python's regex engine enters catastrophic backtracking and the calling thread freezes indefinitely. The published PoC in the GitHub advisory demonstrates this with 12 wildcard pairs and a 60-character URL, requiring no special privileges or existing access to the target system.