Scrapy
CVE-2026-105782
HIGH
Attack ChainAIDerived
Hypothetical attack flow derived from CVE metadata – not a confirmed exploit chain
-
AccessLure Scrapy crawler to malicious page
-
ExploitReturn 'Referrer-Policy: sys.exit' header
-
ExecutionMiddleware imports and calls the object
-
ImpactCrawler process terminates (DoS)
Vulnerability Assessment
| Weakness |
CWE
OWASP Top 10:2025
A05:2025 - Injection
ASVS 5.0
V15 - Secure Coding and Architecture Reflection safety, class allowlisting
|
| Exploitation Conditions | Exploitable only when a Scrapy crawler running versions 1.4.0 through 2.14.1 fetches a response from an attacker-controlled web server while the built-in RefererMiddleware is enabled (the default state, REFERER_ENABLED=True) and no per-request 'referrer_policy' meta key overrides policy evaluation. The attacker sets the Referrer-Policy response header to a value resembling a Python import path (e.g. 'sys.exit'), which Scrapy imports and calls. Limiting factors: the crawler must actually request the malicious page (crawlers confined to trusted domains are not exposed), and impact is constrained to objects that are importable and invokable with no arguments, so realistic effect is process termination/DoS rather than general arbitrary code execution. Available mitigations without upgrading include setting REFERER_ENABLED=False or pinning the referrer_policy request meta key. |
| Impact | The input CVSS 3.1 vector (AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H, base 7.5 HIGH) is internally consistent and matches the description: a network-reachable, unauthenticated, low-complexity flaw whose entire severity comes from availability impact (A:H) with no confidentiality or integrity effect. This is CWE-470 (unsafe reflection) - the RefererMiddleware imported and called an object named in an attacker-controlled Referrer-Policy response header. No EPSS score, no CISA KEV listing, and no public POC were provided in the intelligence set; SSVC decision data was also not supplied, so exploitation-probability and automatability signals are absent and cannot be weighed. Practically, exploitation requires that the Scrapy crawler fetch a response from an attacker-controlled site, which is trivial for broad/open crawlers but not for scrapers limited to trusted domains. The realistic impact is denial of service (crashing the crawler via callables like sys.exit), not arbitrary code execution or data theft - so despite the HIGH score this is a moderate operational priority rather than a critical remote-code-execution emergency. Note a signal conflict: the input Tags include 'Information Disclosure', which contradicts the C:N/A:H impact profile and appears mislabeled. |
| Exploit ScenarioDerived | An operator runs a Scrapy crawler that follows links across arbitrary or untrusted sites. An attacker stands up a web page that returns the HTTP response header 'Referrer-Policy: sys.exit'; when the crawler processes the response, the vulnerable RefererMiddleware imports and invokes that callable, terminating the crawler process. No public exploit code was identified at time of analysis, though the one-line payload is described explicitly in the vendor advisory. |
Get an e-mail when this CVE is exploited, patched or re-scored. Free account.
Alert meAnalysisAIv1 – Oct 05, 2026
Denial of service in Scrapy 1.4.0 through 2.14.1 allows any web server the crawler visits to terminate the crawler process by returning a crafted Referrer-Policy response header, such as 'sys.exit', which the built-in RefererMiddleware treats as a Python import path and invokes. The flaw is reachable over the network without authentication or user interaction (CVSS 3.1 AV:N/AC:L/PR:N/UI:N/S:U, base 7.5 High), and its impact is limited to availability because only importable, zero-argument callables can be triggered, making realistic effect process termination rather than general arbitrary code execution. Any crawler left in its default configuration (RefererMiddleware enabled via REFERER_ENABLED=True) that fetches attacker-controlled pages is exposed; no public exploit identified at time of analysis, and the vendor-released patch is Scrapy 2.14.2.
RemediationAI
Upgrade to Scrapy 2.14.2 or later (vendor-released patch: 2.14.2), which stops executing response-header values as Python callables and introduces the REFERRER_POLICIES setting for controlling which header values are honored; see https://github.com/scrapy/scrapy/security/advisories/GHSA-cwxj-rr6w-m6w7 and https://github.com/scrapy/scrapy/releases/tag/2.14.2. If upgrading immediately is not possible, the most effective mitigation is to set REFERER_ENABLED = False, which removes Referer headers from follow-up requests entirely - a functional trade-off for sites that validate referrers or for anti-bot evasion logic. Where a Referer header is still required, disable the middleware and set the header explicitly on the specific requests that need it, or pin the 'referrer_policy' request meta key (for example meta={'referrer_policy': 'scrapy.spidermiddlewares.referer.DefaultReferrerPolicy'}) so that untrusted response headers are never evaluated; applying that key per-request is error-prone at scale, so a custom spider middleware registered ahead of the built-in referrer policy middleware, or the scrapy-sticky-meta-params plugin to propagate the meta key to follow-up requests, keeps the control consistent. Teams that must keep honoring legitimate Referrer-Policy headers can set the built-in middleware entry to None in SPIDER_MIDDLEWARES and drop in the fixed implementation from Scrapy 2.14.2. Note that restricting crawl scope to trusted domains reduces, but does not eliminate, exposure for any crawler that follows external links.
Affected Packages
Source: OSV (Open Source Vulnerabilities)
Affects your stack
Affects 1 package in pypi
Checking this CVE against your own stacks and servers is part of the Basic plan and up.
Technical ContextAI · show details
The issue is an instance of CWE-470 (Use of Externally-Controlled Input to Select Classes or Code, i.e. unsafe reflection) inside scrapy/spidermiddlewares/referer.py. Scrapy's RefererMiddleware parses the W3C Referrer-Policy response header to decide how to populate the Referer header on follow-up requests. In the vulnerable implementation, the helper _load_policy_class() first attempted load_object(policy) on the raw header value before falling back to matching standard tokens such as 'no-referrer' or 'origin-when-cross-origin'; load_object() is Scrapy's generic dotted-path importer (scrapy.utils.misc), so any header value shaped like a Python import path - for example 'sys.exit' or a dotted path to a project class - was imported and, because the code assumed it referenced a ReferrerPolicy subclass, instantiated or called. A remote server controls that header, giving an attacker direct influence over which object Scrapy imports and executes; the technique is the same class of bug as unsafe reflection via user-controlled class names in Java or PHP. The fix in 2.14.2 changes the parsing flow so that response-header-derived values are resolved only against a fixed mapping of known policy names (allow_import_path=False), while import paths remain permitted exclusively for the trusted REFERRER_POLICY setting, and it adds a new REFERRER_POLICIES setting so operators can override or disable policy names. A _looks_like_import_path() utility in scrapy.utils.python was introduced alongside the change. The affected CPE, cpe:2.3:a:scrapy:scrapy:*:*:*:*:*:*:*:*, reflects the pip-distributed framework rather than a single application, so exposure scales with however many untrusted domains a given spider is allowed to crawl.
Know when Scrapy gets its next vulnerability
- Watch products and get an email when something new is published
- Check this CVE against your own stack and servers
- Exploit and PoC references, threat intelligence from 22 sources
No credit card · 7-day full trial
Severity by source
AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
A malicious remote response triggers it with no auth, privileges, or interaction (AV:N/AC:L/PR:N/UI:N); impact is limited to crashing the crawler process, so C:N/I:N/A:H.
Primary rating from Vendor (GitHub_M).
CVSS VectorVendor: GitHub_M
DescriptionCVE.org
Scrapy is a high-level web crawling and scraping framework for Python. From 1.4.0 until 2.14.2, RefererMiddleware in scrapy/spidermiddlewares/referer.py treated a Referrer-Policy response-header value that resembled a Python import path as a referrer policy class, imported the referenced object, and called it. A malicious website could supply a callable such as sys.exit and terminate a crawler processing the response. This issue is fixed in version 2.14.2.
Articles & Coverage 1
Technical POC Analysis
Evidence base: GitHub advisory GHSA-cwxj-rr6w-m6w7 only.
Full technical POC analysis - root cause, step-by-step exploitation and patch breakdown - is part of the Pro and Business plans.
Executive ViewAI
Scrapy, the open-source crawling framework used widely for threat-intelligence collection, market and price monitoring, and large-scale data pipelines, can be forcibly terminated by any website a crawler visits - no credentials, no user action, and no special network position required. Organizations running Scrapy in its default configuration between versions 1.4.0 and 2.14.1 therefore face unplanned outages of any business process that depends on continuous crawling, including security monitoring feeds and competitive data collection. The impact is limited to availability - no data exposure or arbitrary code execution has been demonstrated - but the low barrier to trigger it makes continuity, not confidentiality, the leadership concern until the framework is upgraded.
Vendor patch identified: upgrade to Scrapy 2.14.2; where immediate patching is not possible, disable the RefererMiddleware (REFERER_ENABLED=False), run crawlers under supervised restart policies or containers to contain process termination, and restrict crawling to trusted, allow-listed domains.
Threat Intelligence Overview
How this decision was reached
The engine reads each signal below and applies a fixed list of rules from top to bottom. The first rule that matches sets the recommended action – like a statement, every line is shown.
What every action level means
SSVC AssessmentENISA
Priority Score
Attack Technique
Denial of Service attacks render applications or systems unavailable by overwhelming resources or triggering failure conditions.
Affected Products (CPE)CVE.org
Scrapy 1.4 allows remote attackers to cause a denial of service (memory consumption) via large files because arbitrarily
In scrapy/scrapy, an issue was identified where the Authorization header is not removed during redirects that only chang
In scrapy version 2.10.1, an issue was identified where the Authorization header, containing credentials for server auth
The scrapy/scrapy project is vulnerable to XML External Entity (XXE) attacks due to the use of lxml.etree.fromstring for
A Regular Expression Denial of Service (ReDoS) vulnerability exists in the XMLFeedSpider class of the scrapy/scrapy proj
Exposure of Sensitive Information to an Unauthorized Actor in GitHub repository scrapy/scrapy prior to 2.6.1. Rated medi
Scrapy's S3DownloadHandler transmitted s3:// requests - including signed AWS Authorization headers, temporary session to
Scrapy is a high-level web crawling and scraping framework for Python. Rated medium severity (CVSS 6.5), this vulnerabil
GHSA-cwxj-rr6w-m6w7
NVD
CVE.org
EPSS
Vulnrichment
Exploit-DB