Scrapy
CVE-2021-41125
MEDIUM
Vulnerability Assessment
| Weakness |
CWE
OWASP Top 10:2025
A01:2025 - Broken Access Control
ASVS 5.0
V14 - Data Protection Data exposure prevention
MITRE ATT&CK
2 techniques across 2 tactics: Collection, Discovery The technique list with ATT&CK links and the kill-chain view is part of the Basic plan and up. |
| Impact | CVSS score of 6.5 (MEDIUM). Attack characteristics: network attack vector, low complexity, low privileges required, no user interaction not required. |
| Exploit ScenarioDerived | A remote attacker could access sensitive data that should not be disclosed by sending crafted requests to the affected application. |
Get an e-mail when this CVE is exploited, patched or re-scored. Free account.
Alert meAnalysisAI
Scrapy is a high-level web crawling and scraping framework for Python. Rated medium severity (CVSS 6.5), this vulnerability is remotely exploitable, low attack complexity. This Exposure of Sensitive Information vulnerability could allow attackers to access sensitive data that should not be disclosed.
RemediationAI
A vendor patch is available. Apply the latest security update as soon as possible. Minimize information in error messages, implement proper access controls, encrypt sensitive data at rest and in transit.
Affected Packages
Source: OSV (Open Source Vulnerabilities)
Affects your stack
Affects 1 package in pypi
Checking this CVE against your own stacks and servers is part of the Basic plan and up.
Technical ContextAI · show details
This vulnerability is classified as Exposure of Sensitive Information (CWE-200), which allows attackers to access sensitive data that should not be disclosed. Scrapy is a high-level web crawling and scraping framework for Python. If you use HttpAuthMiddleware (i.e. the http_user and http_pass spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to True, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new http_auth_domain spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the w3lib.http.basic_auth_header function to convert your credentials into a value that you can assign to the Authorization header of your request, instead of defining your credentials globally using HttpAuthMiddleware. Affected products include: Scrapy, Debian Debian Linux.
Know when Scrapy gets its next vulnerability
- Watch products and get an email when something new is published
- Check this CVE against your own stack and servers
- Exploit and PoC references, threat intelligence from 22 sources
No credit card · 7-day full trial
Severity by source
AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N
Primary rating from NVD · only source for this CVE.
CVSS VectorNVD
DescriptionNVD
Scrapy is a high-level web crawling and scraping framework for Python. If you use HttpAuthMiddleware (i.e. the http_user and http_pass spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to True, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new http_auth_domain spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the w3lib.http.basic_auth_header function to convert your credentials into a value that you can assign to the Authorization header of your request, instead of defining your credentials globally using HttpAuthMiddleware.
Threat Intelligence Overview
NVD
CVE.org
EPSS
GitHub
How this decision was reached
The engine reads each signal below and applies a fixed list of rules from top to bottom. The first rule that matches sets the recommended action – like a statement, every line is shown.
What every action level means
SSVC AssessmentENISA
Priority Score
Attack Technique
Information disclosure occurs when an application unintentionally exposes sensitive data that aids attackers in reconnaissance or directly compromises security.
Affected Products (CPE)NVD
Scrapy 1.4 allows remote attackers to cause a denial of service (memory consumption) via large files because arbitrarily
In scrapy/scrapy, an issue was identified where the Authorization header is not removed during redirects that only chang
In scrapy version 2.10.1, an issue was identified where the Authorization header, containing credentials for server auth
The scrapy/scrapy project is vulnerable to XML External Entity (XXE) attacks due to the use of lxml.etree.fromstring for
A Regular Expression Denial of Service (ReDoS) vulnerability exists in the XMLFeedSpider class of the scrapy/scrapy proj
Exposure of Sensitive Information to an Unauthorized Actor in GitHub repository scrapy/scrapy prior to 2.6.1. Rated medi
Denial of service in Scrapy 1.4.0 through 2.14.1 allows any web server the crawler visits to terminate the crawler proce
Scrapy's S3DownloadHandler transmitted s3:// requests - including signed AWS Authorization headers, temporary session to
Exploit-DB